Non-cooperative games are closely associated with multi-agent optimization wherein a large number of selfish players compete non-cooperatively to optimize their individual objectives under various constraints. Unlike centralized algorithms that require a certain system mechanism to coordinate the players' actions, distributed algorithms have the advantage that the players, either individually or in subgroups, can each make their best responses without full information of their rivals' actions. These distributed algorithms by nature are particularly suited for solving huge size games where the large number of players in the game makes the coordination of the players almost impossible. The distributed algorithms are distinguished by several features: parallel versus sequential implementations, scheduled versus randomized player selections, synchronized versus asynchronous transfer of information, and individual versus multiple player updates. There are two general approaches to establish the convergence of distributed algorithms: contraction versus potential based, each requiring different properties of the players' objective functions. We present convergence results based on these two approaches and discuss randomized extensions of the algorithms that require less coordination and hence are more suitable for big data problems.