Engineering Approximate Computations

There’s a new ecosystem of applications that integrates machine learning into a variety of tasks. Typical domains have included image recognition and natural language processing. However, these techniques have also spread to computer systems domains, such as program compilation, resource scheduling, and database query optimization.

With the success of these systems, we must grapple with the reality that they model and compute with objects that are inherently approximate — real numbers (only computable up to a given precision), neural networks (only validated on a given dataset), and probabilistic computations (results only computable up to a given probability). This reality presents many engineering questions about interpreting, debugging, validating, verifying, and optimizing these systems.

As an illustrative example of these issues, I’ll present our work on deep learning systems for performance modeling of modern computer processors. Using data and simple models, our system predicts the performance of assembly code on modern Intel CPUs better than state-of-the-art, handcrafted techniques from LLVM and Intel.

Guided by the resulting engineering questions, I’ll also present our systems for manipulating and reasoning about such approximate computations. I’ll present our work on the Lottery Ticket Hypothesis, a set of techniques for producing small trainable neural networks that are 10-20% of the size of standard architectures for MNIST, CIFAR10, and ImageNet. The promise of this latter work is not only faster inference and training, but also smaller neural networks that are more amenable to reasoning, such as verifying their robustness.

I’ll conclude with a discussion of our results on new programming systems for sound real-valued, differentiable, probabilistic computation, which is the core computational model behind these systems.

Prof. Michael Carbin

Assistant Professor, MIT on February 21, 2020 at 11:45 AM in EB2 1230

Michael Carbin is an Assistant Professor of Electrical Engineering and Computer Science at MIT and leads the Programming Systems Group. Carbin obtained his Ph.D. in EECS from MIT. Carbin works on investigating the semantics, design, and implementation of systems that operate in the presence of uncertainty in their environment (perception), implementation (neural networks or approximate transformations), or execution (unreliable hardware). Carbin has received an NSF Career Award and many best paper awards at top-tier conferences such as ICLR2019 and OOPSLA 2013, 2014.

The Department of Electrical and Computer Engineering hosts a regularly scheduled seminar series with preeminent and leading reseachers in the US and the world, to help promote North Carolina as a center of innovation and knowledge and to ensure safeguarding its place of leading research.