The summer school will be helf druing August 10-12, 2018 @ Lehigh University.
It will be taught by
- Frank E. Curtis
- Francesco Orabona
- Martin Takac
There is no registration fee to attend the summer school. However, only selected participant will be allowed
to participate (due to limited space).
To nominate a student for a summer school, please fill this
The summer school does not provide a travel support for the students, however,
breakfact and lunch will be provided and inexpensive shared hotel accomodation is available.
More information will be forthcoming.
Outline of summer school
The summer school will cover three topics:
Python & Tensor-flow tutorial
We will discuss the basics of Python (needed for this summer school as every lecture will have a coding part)
and the TensorFlow framework.
During the summer school, we will implement various algorithms, compare their performance, etc.
We will also explain the benefits of GPUs for deep learning and use a cloud platform (e.g. AWS) to run the code.
Online learning and Stochastic Gradient Descent
Online learning is a popular framework for designing and analyzing iterative optimization algorithms,
including stochastic optimization algorithms or algorithms operating on large data streams.
The emphasis in online learning algorithms is on adapting on the unknown characteristics of the data stream,
aiming at designing algorithms with optimal guarantees and no hyperparameters to tune. In this lecture, we will review the basis of online learning,
its connection with stochastic optimization, and the latest advancements. In particular, we will show how it is easy to design first-order
stochastic methods that do not require the tuning of step sizes, yet they achieve practical and theoretical optimal performance.
Beyond SG: Second-order methods for nonconvex optimization
Users of optimization methods for machine learning have been fascinated by the success of stochastic gradient (SG) algorithms for solving large-scale problems. This interest extends even into settings in which first-order methods have been known to falter in the context of deterministic optimization, namely, when the objective function is nonconvex and negative curvature is present. While interesting theoretical results can be proved about SG for nonconvex optimization, there remain various interesting ways to move beyond SG that are worth exploring for the next generation of optimization methods for machine learning. In this segment, we discuss these opportunities along with new second-order-type techniques for solving stochastic nonconvex optimization problems, including inexact Newton, trust region, cubic regularization, and related techniques.