Neural Networks & Deep Learning

We propose two variants of Newton method for solving unconstrained minimization problem. Our method leverages optimization techniques such as penalty and augmented Lagrangian method to generate novel variants of the Newton method namely the Penalty Newton method and the Augmented Newton method.

We develop two greedy sampling rules for the Sketch & Project method for solving linear feasibility problems. The proposed greedy sampling rules generalize the existing max-distance sampling rule and uniform sampling rule and generate faster variants of Sketch & Project methods. We also introduce greedy capped sampling rules that improve the existing capped sampling rules. Moreover, we incorporate the so-called heavy ball momentum technique to the proposed greedy Sketch & Project method.