Singular Diffusion Limit of Tagged Particle Dynamics in a 1-dimensional Zero Range Process with Sinai-type Medium and the Convergence Rate for the Last Iterate of Stochastic Gradient Descent Schemes

Abstract: We present two works in applied stochastic processes. First, consider an unconstrained optimization problem involving a convex objective with H¨older gradient.
This problem arises in the training of a linear classifier for text classification using log-loss objective. When the risk is differentiable with respect to the training parameters, gradient-based optimization schemes can be used. However, when the gradient is small, relying entirely on the gradient to ‘descend’ to a minimizer is slow. In 1964, Polyak introduced an algorithm called the Heavy Ball to accelerate convergence by adding a second order difference term scaled by a positive parameter less than 1. In the first part of the dissertation defense, I will present almost sure and in-probability convergence rate results for the last iterate of the Stochastic Heavy Ball (SHB) in the said setting.
For the second part of the dissertation defense, imagine a message in a bottle floating on an ocean surrounded by similar sized debris. Can we describe the dynamics of this bottle? This is a situation similar to an interacting particle system in a random environment. The case when there is only one particle in 1-dimension is studied by Solomon, Kesten, Kozlov, Spitzer in 1975, Ya. Sinai in 1982, and Brox in 1986 among others. I will present a derivation of tagged particle dynamics in a ‘zero range’ particle system on a 1-dimensional flat torus with Sinai-type random environment. Starting from the dynamics in a mollified environment, we take the paracontrolled limit of the dynamics as mollification vanishes. We define such singular diffusion limit as the random environment dynamics of the tagged particle.