More on matrix products and inversion

Recall that we gave the formula for matrix-vector products:

Definition (matrix-vector multiplication)

Let $A = (a_{ij})$ be an $n \times m$ matrix and let $x$ be a $m \times 1$ column vector. The product $y = Ax$ is a $n \times 1$ vector $y$ given by

$$y_i = \sum_{j=1}^m a_{ij} x_j, \quad 1 \leq i \leq n.$$

Definition (matrix-matrix multiplication)

Let $A = (a_{ij})$ be an $n \times m$ matrix and let $B = (b_{jk})$ be and $m \times p$ matrix. The product $C = AB$, $C = (c_{ik})$ is given by

$$c_{ik} = \sum_{j=1}^m a_{ij} b_{jk}.$$

The easiest way to remember this formula is:

$$C =AB = [ Ab_1, A b_2, \ldots, Ab_p ]$$

where $b_k$ is the $k$th column of $b$.

This has to be the case because of the rules of matrix multiplication:

• Associative $A(BC) = (AB)C$
• Distributive $A(B+D) = AB + AD$
• Scalar multiplication $\lambda AB = (\lambda A) B = A(\lambda B)$

So $$B = [b_1,0,0,0,\ldots,0] + [0,b_2,0,0,\ldots 0] + [0,0,b_3,0,\ldots 0] + \cdots$$

and (check this!)

$$AB = [Ab_1,0,0,0,\ldots,0] + [0,Ab_2,0,0,\ldots 0] + [0,0,Ab_3,0,\ldots 0] + \cdots\\ = [ Ab_1, A b_2, \ldots, Ab_p ]$$

Matrix/vector transpose

If $$A = (a_{ij})_{1\leq i \leq n, 1 \leq j \leq m}$$ is an $n \times m$ matrix then $$A^T = (a'_{ij})_{1 \leq i \leq m, 1 \leq j \leq n}$$ is an $m \times n$ matrix with

$$a'_{ij} = a_{ji}.$$

Properties of the transpose

• $(A^T)^T = A$
• $(AB)^T = B^T A^T$
• $(A+B)^T = A^T + B^T$
• $(A^{-1})^T = (A^{T})^{-1}$ if $A^{-1}$ exists

Linear transformations

A linear transformation $y =T(x)$ from $\mathbb R^n$ to $\mathbb R^m$ is a function that takes in a vector $x \in R^n$ and returns a vector $y \in \mathbb R^m$. In addition, it must satisfy the rule of linearity:

• $T(\alpha x_1 + \beta x_2) = \alpha T(x_1) + \beta T(x_2)$ for any $x_1,x_2 \in R^n$ and $\alpha,\beta \in \mathbb R$

Theorem

Every linear transformation (from $\mathbb R^n$ to $\mathbb R^m$) has a matrix representation $A$ such that $T(x) = Ax$ for all $x \in \mathbb R^n$. In addition

$$A = [\underbrace{a_1,a_2,\ldots,a_n}_{\text{columns of } A}], \quad a_i = T(e_i), \quad e_i = [\underbrace{0,0,\ldots,0}_{i-1 \text{ zeros}},1,0,0,\ldots]^T$$

Determinants

Definition

Suppose that $A$ is a square matrix

• If $A = [a]$ is a $1 \times 1$ matrix then $\det A = a$.
• If $A$ is an $n\times n$ matrix with $n > 1$ the minor $M_{ij}$ is the determinant of the $(n-1)\times(n-1)$ submatrix of $A$ obtained by deleting the $i$th row and $j$th column of the matrix $A$.
• The cofactor $A_{ij}$ is given by $A_{ij} = M_{ij} (-1)^{i+j}$.
• The determinant of the $n \times n$ matrix $A$ when $n > 1$ is given by either

$$\det A = \sum_{j=1}^n a_{ij}A_{ij}, \quad \det A = \sum_{i=1}^n a_{ij}A_{ij}$$

for any $1 \leq i \leq n$ or $1 \leq j \leq n$.

Example

Find $\det A$ using the cofactor expansion

$$A = \begin{bmatrix} 1 & 1 & 3 & 0 \\ 2 & 1 &0 & 3 \\ 1 & 0 & 1 & 0 \\ 0 & -1 & -1 & 0 \end{bmatrix}.$$

Properties of the determinant

1. If $\tilde A$ is obtained from $A$ by the operation $R_i \leftrightarrow R_j$ with $i \neq j$ then $\det \tilde A = - \det A$.
2. If $\tilde A$ is obtained from $A$ by the operation $\lambda R_j \to R_j$ then $\det \tilde A = \lambda \det A$.
3. If $\tilde A$ is obtained from $A$ by the operation $R_i + \lambda R_j \to R_i$ with $i \neq j$ then $\det \tilde A = \det A$.
4. If any row or column of $A$ has only zero entries, then $\det A = 0$.
5. If $A$ has two rows or two columns the same, then $\det A = 0$.

Properties of the determinant (cont.)

1. If $A$ and $B$ are both $n\times n$ matrices then $\det AB = \det A \det B$.
2. $\det A = \det A^T$
3. If $A^{-1}$ exists then $\det A^{-1} = 1/\det A$.
4. If $A$ is either upper or lower triangular then $\det A = \prod_{i=1}^n a_{ii}$.

Example

Find $\det A$ using Gaussian elimination

$$A = \begin{bmatrix} 1 & 1 & 3 & 0 \\ 2 & 1 &0 & 3 \\ 1 & 0 & 1 & 0 \\ 0 & -1 & -1 & 0 \end{bmatrix}.$$