Lecture 3: Homogeneous Systems, Independence, and Linear Maps

Let $$A = \begin{pmatrix} 1 & 2 & 0 & 3 \\ 2 & 4 & 1 & 8 \\ 3 & 6 & 1 & 11 \end{pmatrix} \qquad\text{with columns } \mathbf{a}_1, \mathbf{a}_2, \mathbf{a}_3, \mathbf{a}_4 .$$ Row reduction ($R_2 - 2R_1$, $R_3 - 3R_1$, then $R_3 - R_2$) brings the augmented matrix of $A\mathbf{x} = \mathbf{0}$ to reduced echelon form: $$\begin{pmatrix} 1 & 2 & 0 & 3 & 0 \\ 2 & 4 & 1 & 8 & 0 \\ 3 & 6 & 1 & 11 & 0 \end{pmatrix} \xrightarrow{\substack{R_2 - 2R_1 \\ R_3 - 3R_1 \\ R_3 - R_2}} \begin{pmatrix} 1 & 2 & 0 & 3 & 0 \\ 0 & 0 & 1 & 2 & 0 \\ 0 & 0 & 0 & 0 & 0 \end{pmatrix}$$ Pivots in columns $1$ and $3$; the variables $x_2 = s$ and $x_4 = t$ are free. The parametric vector form of the solutions of $A\mathbf{x} = \mathbf{0}$ is $$\mathbf{x} = s \begin{pmatrix} -2 \\ 1 \\ 0 \\ 0 \end{pmatrix} + t \begin{pmatrix} -3 \\ 0 \\ -2 \\ 1 \end{pmatrix}, \qquad s, t \in \mathbb{R}$$ — a plane through the origin inside $\mathbb{R}^4$, with one direction vector per free variable. Note there is no ``particular solution'' vector out front: for a homogeneous system the trivial solution plays that role. Keep this example; it works overtime in this lecture.

Continue with the matrix $A$ of the homogeneous example, and take $\mathbf{b} = \mathbf{a}_1 + \mathbf{a}_3 = \begin{pmatrix} 1 & 3 & 4 \end{pmatrix}^{T}$. By the column view, $\mathbf{p} = \begin{pmatrix} 1 & 0 & 1 & 0 \end{pmatrix}^{T}$ is a particular solution of $A\mathbf{x} = \mathbf{b}$, and by the structure theorem the complete solution set is $$\mathbf{x} = \begin{pmatrix} 1 \\ 0 \\ 1 \\ 0 \end{pmatrix} + s \begin{pmatrix} -2 \\ 1 \\ 0 \\ 0 \end{pmatrix} + t \begin{pmatrix} -3 \\ 0 \\ -2 \\ 1 \end{pmatrix}, \qquad s, t \in \mathbb{R}$$ — the same plane of homogeneous solutions, shifted by $\mathbf{p}$. Compare with Lecture 2's parametric example: the “particular solution plus direction vectors” shape we computed there was not an accident; it is the structure theorem in action.

Return to the homogeneous example. Each direction vector of the solution set is, by the proposition, a dependence relation among the columns of $A$. Taking $s = 1, t = 0$: $-2\mathbf{a}_1 + \mathbf{a}_2 = \mathbf{0}$, i.e. $\mathbf{a}_2 = 2\mathbf{a}_1$. Taking $s = 0, t = 1$: $-3\mathbf{a}_1 - 2\mathbf{a}_3 + \mathbf{a}_4 = \mathbf{0}$, i.e. $\mathbf{a}_4 = 3\mathbf{a}_1 + 2\mathbf{a}_3$. This is the general pattern: the parametric vector form of $A\mathbf{x} = \mathbf{0}$ shows exactly how each column of $A$ without a pivot is a linear combination of the pivot columns — one direction vector per non-pivot column, with the dependence weights visible in its entries.

For any $m \times n$ matrix $A$, the function $T(\mathbf{x}) = A\mathbf{x}$ is a linear map from $\mathbb{R}^n$ to $\mathbb{R}^m$: the two requirements are precisely the linearity theorem of Lecture 1. So every matrix defines a linear map. The rest of this section shows the converse.

Fix an angle $\theta$ and let $T\colon \mathbb{R}^2 \to \mathbb{R}^2$ rotate every vector counterclockwise by $\theta$ about the origin. This is a linear map, for geometric reasons: rotating a parallelogram rotates its diagonal, so $T(\mathbf{u} + \mathbf{v}) = T(\mathbf{u}) + T(\mathbf{v})$; and rotating a stretched arrow is the same as stretching the rotated arrow, so $T(c\mathbf{u}) = c\,T(\mathbf{u})$. (An algebraic proof will be immediate once we know its matrix.)

Let $T\colon \mathbb{R}^3 \to \mathbb{R}^3$ flip space around the $z$-axis — that is, rotate by $180^\circ$ about the line of all $\begin{pmatrix} 0 & 0 & z \end{pmatrix}^{T}$. A point's $z$-coordinate is untouched while its $(x,y)$-part is sent to the opposite side: $$T\begin{pmatrix} x \\ y \\ z \end{pmatrix} = \begin{pmatrix} -x \\ -y \\ z \end{pmatrix}.$$ From this formula, linearity is an entrywise check: negating and copying entries both respect addition and scaling. Flips around other lines are also linear maps; this one has the friendliest formula because its line is a coordinate axis.

Apply the recipe to rotation by $\theta$. Rotating $\mathbf{e}_1$ (which points along the positive $x$-axis) by $\theta$ lands on $\begin{pmatrix} \cos\theta & \sin\theta \end{pmatrix}^{T}$; rotating $\mathbf{e}_2$ lands on $\begin{pmatrix} -\sin\theta & \cos\theta \end{pmatrix}^{T}$. By the theorem, the matrix of the rotation is $$R_\theta = \begin{pmatrix} \cos\theta & -\sin\theta \\ \sin\theta & \cos\theta \end{pmatrix},$$ and rotation by $\theta$ is simply $\mathbf{x} \mapsto R_\theta \mathbf{x}$ — trigonometry packed into four entries. (This also delivers the promised algebraic proof that rotation is linear: it is a matrix map.)

For the flip around the $z$-axis: $T(\mathbf{e}_1) = -\mathbf{e}_1$, $T(\mathbf{e}_2) = -\mathbf{e}_2$, and $T(\mathbf{e}_3) = \mathbf{e}_3$. So its matrix is $$F = \begin{pmatrix} -1 & 0 & 0 \\ 0 & -1 & 0 \\ 0 & 0 & 1 \end{pmatrix},$$ and indeed $F \begin{pmatrix} x & y & z \end{pmatrix}^{T} = \begin{pmatrix} -x & -y & z \end{pmatrix}^{T}$.