MATRIX

The verses in the magma algebra were given linear representations (their coordinate vectors or versors) as solutions to linear systems of equations. Linear systems of equations, in general, are among the most widespread and fundamental tools. Matrices are the standard tool to represent and solve linear systems of equations. A matrix can be viewed as a structured grid that formalizes and encodes a linear system of equations. To illustrate the concept, a small linear system of equations will be considered along with its matrix representation. The process applies equally well to larger linear systems.

The simplest linear equation consists of a single equation with one unknown:

$a\; x\; =\; b,$

where $a$ and $b$ are the known quantities and $x$ is the unknown. If $a\; \ne \; 0$, then the unique solution is given by

$x\; =\; b/a={\mathrm{a}}^{\mathrm{-1}} b$, where ${\mathrm{a}}^{\mathrm{-1}}$ is the multiplicative inverse of $a$ (its reciprocal).

If $a\; =\; 0$, then there are two possibilities: if $b\; =\; 0$, the equation is of the form $0\; x\; =\; 0$ and $x$ can be any number, therefore, there are infinitely many solutions. If, on the other hand, $b\; \ne \; 0$, then the equation $0\; x\; =\; b$ is impossible and has no solution. The possible cases therefore, are: no solution, one solution or infinitely many.

Next, consider a slightly more elaborate case of a coupled linear system of two equations and two unknowns:

$a\; x\; +\; b\; y\; =\; c$
$d\; x\; +\; e\; y\; =\; f,$

where the coefficients $a,\; b,\; c,\; d,\; e,\; f$ are the known quantities, with $x$ and $y$ as the unknowns. It will be instructive to express the two coupled equations compactly in a form that resembles the previous case involving one equation with one unknown. Accordingly, the four known quantities on the LHS are collected and arranged into a square array of two rows and two columns:

$A\; =$ $\left(\begin{array}{cc}\mathrm{a}& \mathrm{b}\\ \mathrm{d}& \mathrm{e}\end{array}\right)$.

Similarly, the two unknowns on the LHS, and the two known quantities on the RHS, each, forms a column of length two:

$X\; =$ $\left(\begin{array}{c}\mathrm{x}\\ \mathrm{y}\end{array}\right)$

$B\; =$ $\left(\begin{array}{c}\mathrm{c}\\ \mathrm{f}\end{array}\right)$.

Putting them all together, the linear system of equations can now be expressed in the following compact form:

$AX\; =\; B$.

These rectangular arrays of numbers are the matrices. The dimension of an arbitrary matrix is m × n, where m and n are the number of rows and columns resp. A matrix M of dimension m × n can be written in shorthand notation M_m,n or M_m×n. M_n×n is a square matrix with equal number (n) of rows and columns. In the extreme case, a rectangular matrix can be of dimension 1 × n or m × 1, a row or a column matrix, also known as row or column vector. A matrix can be represented in a standard compact notation (a_ij) where a_ij is a general entry located at row i and column j. The following is A expressed in (a_ij) form:

$A\; =$ $\left(\begin{array}{}{a}_{\mathrm{11}} a_{\mathrm{12}}\\ a_{\mathrm{21}} a_{\mathrm{22}}\end{array}\right)$

where, $a_{\mathrm{11}}=\; a$, $a_{\mathrm{12}}=\; b$, $a_{\mathrm{21}}=\; d$, $a_{\mathrm{22}}=\; e$. The original linear system of equations is thus transformed into an equivalent compact matrix equation.

The matrix equation reveals an important property of matrices: they can be multiplied just like numbers as indicated on the LHS. The product of the two matrices A and X is defined in the only natural way possible: it is defined precisely to reproduce the LHS of the original linear system of equations it was obtained from:

$\left(\begin{array}{cc}\mathrm{a}& \mathrm{b}\\ \mathrm{d}& \mathrm{e}\end{array}\right)$ $\left(\begin{array}{c}\mathrm{x}\\ \mathrm{y}\end{array}\right)$ = $\left(\begin{array}{c}\mathrm{a\; x\; +\; b\; y}\\ \mathrm{d\; x\; +\; e\; y}\end{array}\right)$.

It is readily apparent that the product of the two matrices on the LHS is only possible if the number of columns of A matches the number of rows of X. This is the general rule for matrix multiplication. The product of two matrices requires the number of columns of the first matrix to match the number of rows of the second matrix. In the matrix equation the 2 × 2 matrix A is multiplied by a 2 × 1 column matrix X resulting in the 2 × 1 column matrix B. The multiplication can be extended by replacing the single column vector with multiple column vectors placed side-by-side. In other words, matrix multiplication is viewed as a series of matrix-vector multiplications bundled together side-by-side. To see this consider the product of A with a 2 × 3 matrix M:

$M\; =$ $\left(\begin{array}{ccc}\mathrm{t}& \mathrm{v}& \mathrm{x}\\ \mathrm{u}& \mathrm{w}& \mathrm{y}\end{array}\right)$

The product AM simply is the product of A and the columns of M. That is M is viewed as a collection of column vectors placed next to each other. Let

$C$₁ = $\left(\begin{array}{c}\mathrm{t}\\ \mathrm{u}\end{array}\right)$

$C$₂ = $\left(\begin{array}{c}\mathrm{v}\\ \mathrm{w}\end{array}\right)$

$C$₃ = $\left(\begin{array}{c}\mathrm{x}\\ \mathrm{y}\end{array}\right)$

be the three columns of M. Then

M = (C₁, C₂, C₃).

Then

AM = (AC₁, AC₂, AC₃)

The product is a 2 × 3 matrix whose i-th column AC_i is the product of A and C_i. The multiplication of larger matrices are carried out exactly the same way. In general, the product of matrices of dimensions m × k and k × n is a matrix of dimension m × n. In particular, the products of square matrices of a fixed dimension n are always defined and the result is square matrices of the same dimension. Matrix multiplication is also associative: (AB)C = A(BC) for any three matrices of compatible dimensions.

Matrices like vectors can be multiplied by scalars (numbers):

$kA\; =\; k$ $\left(\begin{array}{cc}\mathrm{a}& \mathrm{b}\\ \mathrm{d}& \mathrm{e}\end{array}\right)$ = $\left(\begin{array}{cc}\mathrm{ka}& \mathrm{kb}\\ \mathrm{kd}& \mathrm{ke}\end{array}\right)$

where k is a scalar in some suitable field. The scalar product is commutative: kA = Ak.

In the special case of a matrix of dimension 1 × 1, the square matrix consists of a single entry. The products of 1 × 1 matrices reduce to the commutative products of numbers. Let A_1,1 = (a) and B_1,1 = (b) with single entries a and b resp. Then AB = (a)(b) = (ab) = (ba) = (b)(a) = BA. Viewed this way, matrices can be considered generalizations of numbers. It should be noted that in the multiplication of a matrix by the scalar k, the scaling factor k simply is an element of a field. It is not considered a 1 × 1 matrix since that would require dimension restriction. As a matrix, K_1,1 = (k) can only be multiplied by row or column vectors: K_1,1R_1,m or C_n,1K_1,1, where R_1,m is a row-vector and C_n,1 is a column-vector. As generalizations of numbers, matrices share many common properties with numbers. Like numbers, they can also be added. The addition is performed componentwise:

$\left(\begin{array}{cc}\mathrm{a}& \mathrm{b}\\ \mathrm{c}& \mathrm{d}\end{array}\right)$ + $\left(\begin{array}{cc}\mathrm{e}& \mathrm{f}\\ \mathrm{g}& \mathrm{h}\end{array}\right)$ = $\left(\begin{array}{cc}\mathrm{a\; +\; e}& \mathrm{b\; +\; f}\\ \mathrm{c\; +\; g}& \mathrm{d\; +\; h}\end{array}\right)$

Stated compactly, if A = (a_ij) and B = (b_ij), them A + B = (a_ij) + (b_ij) = (a_ij + b_ij). The corresponding entries at row i and column j are added together. The dimension compatibility for addition is more rigid. The two matrices must strictly be of the same dimension. The sum is a matrix also of the same dimension.

An m × n matrix can be viewed through its vector components. It can be treated as an ordered stack of vectors: m row-vectors of length n or equivalently n column-vectors of length m. This motivates matrix transposition as an operation that transforms horizontal row-vectors into vertical column-vectors (and vice versa). Accordingly, the matrix is flipped over its diagonal. That is row i becomes column i, or equivalently, column j becomes row j. The entry a_ij is interchanged with a_ji. If the original matrix was of dimension m × n, its transpose is of dimension n × m. A square matrix retains its dimension. The elements on the diagonal a_ii remain intact during transposition. Let

$M$_2,3 = $\left(\begin{array}{ccc}\mathrm{a}& \mathrm{b}& \mathrm{c}\\ \mathrm{d}& \mathrm{e}& \mathrm{f}\end{array}\right)$

Then, its transpose M^T is a flipped rectangular matrix of dimension 3 × 2:

$MT=$ $\left(\begin{array}{cc}\mathrm{a}& \mathrm{d}\\ \mathrm{b}& \mathrm{e}\\ \mathrm{c}& \mathrm{f}\end{array}\right)$

Transposing a 1 × 1 matrix A = (a) is trivial since the single entry a can be both a row and a column: A^T = A. Since, numbers can be viewed as 1 × 1 matrices, transposition of numbers is trivial as well. Matrices that stay fixed under transposition are called symmetric. They are necessarily square since dimension m × n must be the same as n × m, forcing m = n. Their entries are mirrored across the main diagonal (a_ij = a_ji). Let a and b be numbers, then (ab)^T = ab = a^Tb^T. Matrices nearly follow a similar behavior: (MN)^T = N^TM^T for matrices M and N. Transposition is an involution. If repeated once more it returns the original matrix: (M^T)^T = M. It respects addition and scalar multiplication: (M + N)^T = M^T + N^T and (kM)^T = kM^T. These properties make transposition a linear map from m × n matrices to n × m matrices.

Similar to numbers, there are two distinguished matrices of great significance: the zero and the identity matrices. They are the direct counterparts of the numbers 0 and 1, resp. The following matrix

$$0_2×2 = $\left(\begin{array}{cc}0& 0\\ 0& 0\end{array}\right)$

consists of all zero entries a_ij = 0. It behaves exactly like the number 0. It is the additive neutral element: 0 + M = M + 0 = M for all M_2×2. It is also the multiplicative annihilator: 0M = M0 = 0. The identity matrix:

$I\; =$ $\left(\begin{array}{cc}1& 0\\ 0& 1\end{array}\right)$

is the straightforward analogue of the number 1. It is the multiplicative neutral element: IM = MI = M for M_2×2. It has 1s on the diagonal and 0s for the off-diagonals: a_ii = 1 and a_ij = 0 for i ≠ j. Due to their extra structures, matrices also differ significantly from familiar numbers. Whereas, any two numbers can be added or multiplied, matrices require dimension compatibility. Furthermore, multiplication of matrices, in general, is not commutative: MN ≠ NM as can be seen:

$\left(\begin{array}{cc}1& 2\\ 3& 4\end{array}\right)$ $\left(\begin{array}{cc}5& 6\\ 7& 8\end{array}\right)$ = $\left(\begin{array}{cc}19& 22\\ 43& 50\end{array}\right)$

While

$\left(\begin{array}{cc}5& 6\\ 7& 8\end{array}\right)$ $\left(\begin{array}{cc}1& 2\\ 3& 4\end{array}\right)$ = $\left(\begin{array}{cc}23& 34\\ 31& 46\end{array}\right)$

Two matrices A = (a_ij) and B = (b_ij) are equal if both are of the same dimension m × n and with identical corresponding entries: a_ij = b_ij for 1 ≤ i ≤ m and 1 ≤ j ≤ n. Another significant difference is the existence of zero-divisors. The product of two non-zero matrices can be zero:

$\left(\begin{array}{cc}1& 0\\ 0& 0\end{array}\right)$ $\left(\begin{array}{cc}0& 0\\ 0& 1\end{array}\right)$ = $\left(\begin{array}{cc}0& 0\\ 0& 0\end{array}\right)$

Numbers other than zero have multiplicative inverses (reciprocals): for n ≠ 0, the inverse n⁻¹ = 1/n where n × n⁻¹ = 1. Matrices in general, do not share this property. The two zero-divisor matrices above have no inverses. They are singular. Matrices with multiplicative inverses are called invertible or nonsingular. The product of an invertible matrix M and its inverse M⁻¹ is the identity matrix: MM⁻¹ = M⁻¹M = I. The product of two non-zero numbers a and b is non-zero and therefore has an inverse: (ab)⁻¹ = 1/(ab) = (1/a)(1/b) = a⁻¹ b⁻¹. Matrices share a comparable property. Let M and N be two invertible matrices, then: (MN)⁻¹ = N⁻¹M⁻¹. This can be easily confirmed: (MN)(MN)⁻¹ = I = (MN)(N⁻¹M⁻¹) = M(NN⁻¹)M⁻¹ = MIM⁻¹ = MM⁻¹ = I. And similarly: (MN)⁻¹(MN) = I = (N⁻¹M⁻¹)(MN) = N⁻¹(M⁻¹M)N = N⁻¹IN = N⁻¹N = I.

Solving the resulting matrix equation $AX\; =\; B$ follows a similar procedure to that of $ax\; =\; b$. When $a\; \ne \; 0$ the unique solution was obtained by multiplying both sides by its multiplicative inverse, $x\; =a^{\mathrm{-1}}b$. Analogously, if the coefficient matrix $A$ is nonsingular, then, both sides of the matrix equation can be left-multiplied by $A^{\mathrm{-1}}$ to obtain the solution vector $X$:

$AX\; =\; B$
${\mathrm{A}}^{\mathrm{-1}}AX\; =A^{\mathrm{-1}}B$,   (multiplying from the left on both sides by ${\mathrm{A}}^{\mathrm{-1}}$)
$IX\; =A^{\mathrm{-1}}B$
$X\; =A^{\mathrm{-1}}B$.

Note that the multiplication on both sides of the equation by $A^{\mathrm{-1}}$ must be done from the left (due to noncommutativity) to cancel out $A$. Multiplication on the right is not permissible since it is not defined due to incompatibility of the dimensions. When the coefficient matrix $A$ is singular it is similar to the case $a\; =\; 0$; there may be infinitely many or no solution at all.

Although, matrices are closely related to representing and solving linear systems of equations they are independent abstract algebraic objects with their own specific characteristics and rules. They can be algebraically manipulated through addition, multiplication, scalar-multiplication, and transposition, while sharing many of the familiar properties with plain numbers such as additive and multiplicative neutral elements, distributivity, associativity, etc. They also diverge significantly from standard arithmetic due to dimension compatibility requirements, noncommutativity, transposition and existence of zero-divisors.