3

We propose two variants of Newton method for solving unconstrained minimization problem. Our method leverages optimization techniques such as penalty and augmented Lagrangian method to generate novel variants of the Newton method namely the Penalty Newton method and the Augmented Newton method.

In this work, we shed light on the so-called Kaczmarz method for solving Linear System (LS) and Linear Feasibility (LF) problems from a optimization point of view. We introduce well-known optimization approaches such as Lagrangian penalty and Augmented Lagrangian in the Randomized Kaczmarz (RK) method. In doing so, we propose two variants of the RK method namely the Randomized Penalty Kacmarz (RPK) method and Randomized Augmented Kacmarz (RAK) method.

We develop two greedy sampling rules for the Sketch & Project method for solving linear feasibility problems. The proposed greedy sampling rules generalize the existing max-distance sampling rule and uniform sampling rule and generate faster variants of Sketch & Project methods. We also introduce greedy capped sampling rules that improve the existing capped sampling rules. Moreover, we incorporate the so-called heavy ball momentum technique to the proposed greedy Sketch & Project method.

In this work, we first propose a Stochastic Steepest Descent (SSD) framework that connects SP methods with the well-known Steepest Descent (SD) method for solving positive-definite linear system of equations. We then introduce two greedy sampling strategies in the SSD framework that we generalize the existing sampling rules into one framework and develop an efficient version of SP methods. Furthermore, we incorporated the Polyak momentum technique into the SSD method to accelerate the resulting algorithms.

In this work, we propose a momentum induced algorithm called Momentum Sampling Kaczmarz Motzkin (MSKM). The MSKM algorithm is developed by integrating the heavy ball momentum to the SKM algorithm.

This work deals with a systematic review of logarithmic integrals starting from $\textit{Malmsten}$ integrals to classical collection of Integrals, Series and Products by $\textit{I. S. Gradshteyn}$ and $\textit{I. M. Ryzhik}$ to recent times. The evaluation of these types of integrals involves higher transcendental functions (i.e., Hurwitz Zeta function, Polylogarithms, Lerch Transcendental, Orthogonal Polynomials, PolyGamma functions).