CMOR 537: COMPUTER-ASSISTED ALG DESIGN
Spring 2026 · Section 002 · CRN 26077
In a traditional first course in continuous optimization, students learn to use and analyze existing algorithms, such as applying Stochastic Gradient Descent to train neural networks, using Accelerated Gradient Descent to solve image processing problems, or study their convergence analysis. In contrast, this course methodically addresses the fundamental questions: Where do these algorithms come from? How do we know they are any good? And, most importantly, could we design something provably better? The core of this course is a transition from merely using and analyzing optimization algorithms to actively designing them, treating the algorithm design process itself as an optimization problem. The goal is to construct algorithms that are the provably fastest methods for specific classes of problems in consideration. In particular, we will focus on designing the optimal first-order methods (FOMs), which are methods that rely solely on gradient or subgradient information. These are the most commonly used optimization algorithms today, given their efficacy in solving high-dimensional problems, compatibility with large-scale datasets, and applicability to machine learning. The curriculum is structured in two main parts. First, we will learn to build a mathematical model (an optimization problem in itself) that calculates the absolute worst-case performance for a given algorithm. This provides a rigorous framework for computing tight convergence guarantees for a known algorithm. Subsequently, we will formulate the search for the provably fastest algorithm as a new, higher-level optimization problem. The solution to this problem is, in fact, the optimal algorithm we seek. This modern, computer-assisted approach to algorithm design is known as Performance Estimation Problem (PEP). This is a hands-on course with a strong emphasis on implementation, where students will learn to build the tools that discover these algorithms from scratch using PEP. By the end, students will not only understand the theory behind cutting-edge algorithm design but will also have the skills to discover and analyze new, high-performance algorithms for a variety of challenges in optimization and data science.
- Credits
- 3
- Instructors
- Shuvomoy Das Gupta