Lecture 1: Vectors, Matrices, and Systems of Linear Equations

$\mathbb{R}^n$ denotes the set of all column vectors with $n$ real entries. For $$\mathbf{u} = \begin{pmatrix} u_1 \\ u_2 \\ \vdots \\ u_n \end{pmatrix}, \qquad \mathbf{v} = \begin{pmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \end{pmatrix} \qquad\text{in } \mathbb{R}^n$$ and a scalar $c \in \mathbb{R}$, define vector addition and scalar multiplication entrywise: $$\mathbf{u} + \mathbf{v} = \begin{pmatrix} u_1 + v_1 \\ \vdots \\ u_n + v_n \end{pmatrix}, \qquad c\,\mathbf{u} = \begin{pmatrix} c\,u_1 \\ \vdots \\ c\,u_n \end{pmatrix}.$$ We write $\mathbf{0}$ for the zero vector (all entries $0$) and $-\mathbf{u} = (-1)\mathbf{u}$.

A row vector with $n$ real entries is a list written horizontally, $$\mathbf{a} = \begin{pmatrix} a_1 & a_2 & \cdots & a_n \end{pmatrix},$$ and it is added and scaled entrywise, exactly as for columns. The transpose operation turns one shape into the other: for a column vector $\mathbf{u}$ and a row vector $\mathbf{a}$ as above, $$\mathbf{u}^{T} = \begin{pmatrix} u_1 \\ u_2 \\ \vdots \\ u_n \end{pmatrix}^{T} = \begin{pmatrix} u_1 & u_2 & \cdots & u_n \end{pmatrix},$$ $$\mathbf{a}^{T} = \begin{pmatrix} a_1 & a_2 & \cdots & a_n \end{pmatrix}^{T} = \begin{pmatrix} a_1 \\ a_2 \\ \vdots \\ a_n \end{pmatrix}.$$ So the transpose of a column vector is a row vector, and the transpose of a row vector is a column vector; the entries are unchanged, only the shape is. Applying the operation twice returns the original vector: $$(\mathbf{u}^{T})^{T} = \mathbf{u}, \qquad (\mathbf{a}^{T})^{T} = \mathbf{a}.$$ Transposing also respects the two vector operations: $(\mathbf{u} + \mathbf{v})^{T} = \mathbf{u}^{T} + \mathbf{v}^{T}$ and $(c\,\mathbf{u})^{T} = c\,\mathbf{u}^{T}$, since both sides are formed from the same entries.

Two vectors are equal when they have the same shape, the same number of entries, and every pair of corresponding entries is equal. That is, for column vectors $\mathbf{u}, \mathbf{v}$, $$\mathbf{u} = \mathbf{v} \quad\Longleftrightarrow\quad u_i = v_i \ \text{ for } i = 1, 2, \dots, n,$$ and likewise for two row vectors. In particular vectors of different lengths are never equal: $\begin{pmatrix} 1 & 2 \end{pmatrix}^{T}$ and $\begin{pmatrix} 1 & 2 & 0 \end{pmatrix}^{T}$ are different vectors, one in $\mathbb{R}^2$ and one in $\mathbb{R}^3$. And a row vector is never equal to a column vector, no matter what its entries are: $$\begin{pmatrix} 1 & 2 & 3 \end{pmatrix} \;\neq\; \begin{pmatrix} 1 \\ 2 \\ 3 \end{pmatrix} ,$$ because the two sides have different shapes. They are related by the transpose, not by equality: the left-hand side is the transpose of the right-hand side.

Let $\mathbf{a} = \begin{pmatrix} a_1 & a_2 & \cdots & a_n \end{pmatrix}$ be a row vector of length $n$. The row–column product of $\mathbf{a}$ with a column vector $\mathbf{x} \in \mathbb{R}^n$ of the same length is the scalar $$\mathbf{a}\,\mathbf{x} = \begin{pmatrix} a_1 & a_2 & \cdots & a_n \end{pmatrix} \begin{pmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{pmatrix} = a_1 x_1 + a_2 x_2 + \cdots + a_n x_n .$$

An $m \times n$ matrix $A$ is a rectangular array of numbers $a_{ij}$ with $m$ rows and $n$ columns. Its rows $\mathbf{r}_1, \dots, \mathbf{r}_m$ are row vectors of length $n$; its columns $\mathbf{a}_1, \dots, \mathbf{a}_n$ are vectors in $\mathbb{R}^m$. For $\mathbf{x} \in \mathbb{R}^n$, the matrix–vector product $A\mathbf{x}$ is the column vector in $\mathbb{R}^m$ whose $i$-th entry is the row–column product of the $i$-th row with $\mathbf{x}$: $$A\mathbf{x} = \begin{pmatrix} \mathbf{r}_1\,\mathbf{x} \\ \mathbf{r}_2\,\mathbf{x} \\ \vdots \\ \mathbf{r}_m\,\mathbf{x} \end{pmatrix}, \qquad\text{i.e.}\qquad (A\mathbf{x})_i = a_{i1}x_1 + a_{i2}x_2 + \cdots + a_{in}x_n .$$

A linear combination of the vectors $\mathbf{v}_1, \dots, \mathbf{v}_p \in \mathbb{R}^m$ with weights $c_1, \dots, c_p \in \mathbb{R}$ is the vector $$c_1 \mathbf{v}_1 + c_2 \mathbf{v}_2 + \cdots + c_p \mathbf{v}_p .$$

A linear equation in the variables $x_1, \dots, x_n$ is an equation of the form $$a_1 x_1 + a_2 x_2 + \cdots + a_n x_n = b,$$ where the coefficients $a_1, \dots, a_n$ and the right-hand side $b$ are real numbers. A system of linear equations (or linear system) is a finite collection of linear equations in the same variables. (Note the left-hand side is exactly a row–column product: the row of coefficients times the column of unknowns.)

A solution of a linear system in $x_1,\dots,x_n$ is a list $(s_1, \dots, s_n)$ of numbers that makes every equation of the system true when we substitute $x_i = s_i$. The solution set is the set of all solutions. A system is consistent if it has at least one solution, and inconsistent if it has none.

Two linear systems in the same variables are equivalent if they have the same solution set.

The elementary operations on a linear system are:

1. (Swap) interchange the positions of two equations;
2. (Scale) multiply one equation by a nonzero constant $c$;
3. (Replace) add a multiple of one equation to another equation.

Consider the linear system $$\left\{ \begin{aligned} a_{11}x_1 + a_{12}x_2 + \cdots + a_{1n}x_n &= b_1 \\ a_{21}x_1 + a_{22}x_2 + \cdots + a_{2n}x_n &= b_2 \\ &\;\;\vdots \\ a_{m1}x_1 + a_{m2}x_2 + \cdots + a_{mn}x_n &= b_m \end{aligned} \right.$$ Collect its coefficients, its unknowns, and its right-hand sides into $$A = \begin{pmatrix} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n} \\ \vdots & \vdots & & \vdots \\ a_{m1} & a_{m2} & \cdots & a_{mn} \end{pmatrix}, \qquad \mathbf{x} = \begin{pmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{pmatrix}, \qquad \mathbf{b} = \begin{pmatrix} b_1 \\ b_2 \\ \vdots \\ b_m \end{pmatrix}.$$ $A$ is the coefficient matrix of the system; $\mathbf{x}$ and $\mathbf{b}$ are the column vectors of unknowns and of right-hand sides. Since the left-hand side of the $i$-th equation is exactly the $i$-th entry of the matrix–vector product $A\mathbf{x}$, the system asks precisely for those $\mathbf{x}$ with $$A\mathbf{x} = \mathbf{b},$$ called the matrix equation of the system. Finally, the augmented matrix $(\,A \mid \mathbf{b}\,)$ is $A$ with the column $\mathbf{b}$ attached as an extra last column.

The elementary row operations on a matrix are the mirror images of the elementary operations on equations: swap two rows, scale a row by a nonzero constant, and add a multiple of one row to another row. Two matrices are row equivalent if one can be turned into the other by a chain of row operations. By the theorem on elementary operations, row-equivalent augmented matrices represent equivalent systems.