LOS Seminar

Cristian Rusu (LOS)
Accelerated Coordinate Descent Methods for SVMs

Abstract: We study a particular class of constrained linear least-square optimization problems and propose new accelerated coordinate descent techniques to solve them. We prove convergence results and show experimental simulation for problems in machine learning, particularly Support Vector Machines (SVMs).

Ion Necoară (University Politehnica Bucharest)
Higher order methods for nonlinear least-squares

Abstract: Nonlinear least-squares minimization involves a collection of nonlinear functions which are aggregated in a nonsmooth manner, often via a norm. We present a higher-order majorization-minimization algorithm for solving such (nonconvex) problems, where the nonlinear functions are assumed to have smooth higher-order derivatives. Our algorithm replaces each component in the composite model with a higher-order Taylor approximation and adds a proper regularization term, leading to a higher-order Gauss-Newton type method. We present convergence guarantees for this algorithm under different assumptions on problem’s data, including rates. In special cases, where complexity bounds are known for some particular (first-order) algorithms, our convergence results recover the existing bounds. Applications to non-linear least-squares (including phase retrieval), optimal control, and functional constrained minimization are presented. Finally, some open questions related to higher-order optimization are discussed.

Andrei Pătrașcu (LOS)
Learning Explicitly Conditioned Sparsifying Transforms

Abstract: Sparsifying transforms became in the last decades widely known tools for finding structured sparse representations of signals in certain transform domains. Despite the popularity of classical transforms such as DCT and Wavelet, learning optimal transforms that guarantee good representations of data into the sparse domain has been recently analyzed in a series of papers. Typically, the conditioning number and representation ability are complementary key features of learning square transforms that may not be explicitly controlled in a given optimization model. Unlike the existing approaches from the literature, we consider a new sparsifying transform model that enforces explicit control over the data representation quality and the condition number of the learned transforms. The numerical experiments show that our model presents better numerical behavior than the state-of-the-art.

• LOS Seminar in 2023/2024
• LOS Seminar in 2022/2023
• LOS Seminar in 2021/2022
• LOS Seminar in 2020/2021