In this thesis, our main focus is on the design, analysis, and implementation of such algorithms. In particular, we are interested in developing stochastic iterative methods for solving large-scale linear systems, linear feasibility, and linear programming problems.