Regression of Functions on Low-dimensional Manifolds by Neural Networks

Wenjing Liao
Georgia Institute of Technology
Tuesday, February 2, 2021 - 1:25pm to 2:25pm

Many data in real-world applications lie in a high-dimensional space but are concentrated on or near a low-dimensional manifold. Our goal is to estimate functions on the manifold from finite samples of data. This talk focuses on an efficient approximation theory of deep ReLU networks for functions supported on low-dimensional manifolds. We construct a ReLU network for such function approximation where the size of the network grows exponentially with respect to the intrinsic dimension of the manifold. When the function is estimated from finite samples, we proved that the mean squared error for the function approximation converges as the training samples increases with a rate depending on the intrinsic dimension of the manifold instead of the ambient dimension of the space. These results demonstrate that deep neural networks are adaptive to low-dimensional geometric structures of data. This is a joint work with Minshuo Chen, Haoming Jiang, Tuo Zhao (Georgia Institute of Technology).

Dr. Wenjing Liao is an assistant professor in the School of Mathematics at Georgia Tech. She obtained her Ph.D. in mathematics at University of California, Davis in 2013. She was a visiting assistant professor at Duke University from 2013 to 2016, as well as a postdoctoral fellow at Statistical and Applied Mathematical Sciences Institute from 2013 to 2015. She worked at Johns Hopkins University as an assistant research scientist from 2016 to 2017. She works on theory and algorithms in the intersection of applied math, machine learning and signal processing. Her current research interests include multiscale methods for dimension reduction, statistical learning theory, and deep learning.