Novel minimization algorithms and applications to image restoration problems and data prediction

Abstract

One kind of the most prevalent minimization problem which has been studied extensively is the least absolute shrinkage and selection operator problem (LASSO). Such problem arises in a wide range of applications such as statistical analysis, image processing, machine learning, etc. To solve this problem, several algorithms were introduced and studied by many authors with the challenges of computational speed and accuracy of the algorithms such as the proximal forward-backward splitting method, the Douglas-Rachford splitting method, and the alternating direction method of multipliers, and the primal-dual splitting type methods. In the first results part of this thesis, we present the novel minimization algorithms for solving the lasso problem and also analyze the theoretical weak convergence behavior of the proposed algorithms under some suitable conditions by using a specific technique in fixed point theory. In the part of the application, we apply our main results to solve image restoration problems, and some supervised learning problems. It is shown by some numerical experiments that our algorithms have a good behavior compared with some classical and popular methods such as the forward-backward algorithm (FBA), a new accelerated proximal gradient algorithm (nAGA), and a fast iterative shrinkage-thresholding algorithm (FISTA), etc. In the second results part, the specific type of bi-level convex optimization so called hierarchical convex optimization with primal-dual splitting constrain will be studied and discussed. We propose a new algorithm to solve such problem. A strong convergence and rate of convergence theorems are also proved and investigated. In the final experiment part, our algorithm will be applied to train a specific deep convolution neural networks that are able to solve image denoising and image super-resolution problems.