We review various subspace techniques that are used in constructing of numerical methods for nonlinear optimization. The subspace techniques are getting more and more important as the optimization problems we have to solve are getting larger and larger in scale. The applications of subspace techniques have the advantage of reducing both computation cost and memory size. Actually in many standard optimization methods (such as conjugate gradient method, limited memory quasi-Newton method, projected gradient method, and null space method) there are ideas or techniques that can be viewed as subspace techniques. For constrained optimization, by using subspace approach, we can have a better understanding of some numerical methods, such as the null space method.

The essential part of a subspace method is how to choose the subspace in which the trial step or the trust region should belong. Model subspace algorithms for unconstrained optimization and constrained optimization will be discussed respectively. We will also consider subspace techniques for solving nonlinear equations and nonlinear least squares.