Matrix Algebra

Addition and Subtraction

Matrices can only be added or subtracted if they are the same shape. Suppose we have the 2×2 matrices below:

\[\begin{split} A = \begin{pmatrix} a_{11} & a_{12} \\ a_{21} & a_{22} \end{pmatrix} \end{split}\]

\[\begin{split} B = \begin{pmatrix} b_{11} & b_{12} \\ b_{21} & b_{22} \end{pmatrix} \end{split}\]

Then their sum is found by addiing together the corresponding matrix elements:

\[\begin{split} A + B = \begin{pmatrix} a_{11} & a_{12} \\ a_{21} & a_{22} \end{pmatrix} + \begin{pmatrix} b_{11} & b_{12} \\ b_{21} & b_{22} \end{pmatrix} = \begin{pmatrix} a_{11} + b_{11} & a_{12} + b_{12} \\ a_{21} + b_{21} & a_{22} + b_{22} \end{pmatrix} \end{split}\]

Subtraction works in exactly the same way:

\[\begin{split} A - B = \begin{pmatrix} a_{11} & a_{12} \\ a_{21} & a_{22} \end{pmatrix} - \begin{pmatrix} b_{11} & b_{12} \\ b_{21} & b_{22} \end{pmatrix} = \begin{pmatrix} a_{11} - b_{11} & a_{12} - b_{12} \\ a_{21} - b_{21} & a_{22} - b_{22} \end{pmatrix} \end{split}\]

In general, if we are adding or subtracting two \(m\times n\) matrices:

Note

Matrix addition and subtraction is commutative so \(A+B=B+A\).

\[\begin{split} \begin{pmatrix} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \cdots & a_{mn} \end{pmatrix} \pm \begin{pmatrix} b_{11} & b_{12} & \cdots & b_{1n} \\ b_{21} & b_{22} & \cdots & b_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ b_{m1} & b_{m2} & \cdots & b_{mn} \end{pmatrix} = \begin{pmatrix} a_{11} \pm b_{11} & a_{12} \pm b_{12} & \cdots & a_{1n} \pm b_{1n} \\ a_{21} \pm b_{21} & a_{22} \pm b_{22} & \cdots & a_{2n} \pm b_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} \pm b_{m1} & a_{m2} \pm b_{m2} & \cdots & a_{mn} \pm b_{mn} \end{pmatrix} \end{split}\]

Example

Calculate the following:

1. \(\begin{pmatrix} 1 & 1 \\ 4 & 0 \end{pmatrix} + \begin{pmatrix} -1 & 4 \\ 2 & -3 \end{pmatrix}\)

2. \(\begin{pmatrix} 10 \\ -3 \\ 0 \end{pmatrix} - \begin{pmatrix} -3 \\ 2 \\ 0 \end{pmatrix}\)

3. \(\begin{pmatrix} 9 \\ -2 \end{pmatrix} + \begin{pmatrix} -9 & 0 & 1 \\ 13 & 13 & 0 \\ 9 & 0 & 9 \end{pmatrix}\)

Solution:

1. \(\begin{pmatrix} 1 & 1 \\ 4 & 0 \end{pmatrix} + \begin{pmatrix} -1 & 4 \\ 2 & -3 \end{pmatrix} = \begin{pmatrix} 1 + (-1) & 1 + 4 \\ 4 + 2 & 0 + (-3) \end{pmatrix} = \begin{pmatrix} 0 & 5 \\ 6 & -3 \end{pmatrix}\)

2. \(\begin{pmatrix} 10 \\ -3 \\ 0 \end{pmatrix} - \begin{pmatrix} -3 \\ 2 \\ 0 \end{pmatrix} = \begin{pmatrix} 10-(-3) \\ -3-2 \\ 0-0 \end{pmatrix} = \begin{pmatrix} 13 \\ -5 \\ 0 \end{pmatrix}\)

3. We can not add these matrices as they are of different sizes.

---

The NumPy arrays perform addition and subtraction in the exact same was as for a matrix.

import numpy as np 

A = np.array([[1, 1], [4, 0]])
B = np.array([[-1, 4], [2, -3]]) 

A + B

array([[ 0,  5],
       [ 6, -3]])

Python Note

Beaware, a NumPy array does not follow all the same rules as a mathematical matrix. The operations that arrays perform is discussed in detail in this section of Jake VanderPlas Python Data Science Handbook.

A = np.array([[10], [-3], [0]])
B = np.array([[-3], [2], [0]])

A - B

array([[13],
       [-5],
       [ 0]])

Multiplication by a Constant

The simplest form of multiplication is where we multiply a matrix by a constant or scalar. We show the rule on a 2×2 matrix, but it follows in the same way for any sized matrix, we simply multiply every element of the mattrix by the scalar.

\[\begin{split} \lambda A = \lambda \begin{pmatrix} a & b \\ c & d \end{pmatrix} = \begin{pmatrix} \lambda a & \lambda b \\ \lambda c & \lambda d \end{pmatrix} \end{split}\]

Example

Find the following:

1. \(2\begin{pmatrix} -1 & 2 \\ 0 & 4 \end{pmatrix}\)

2. \(\frac{\sqrt{3}}{3}\begin{pmatrix} 9 & 0 & 3 \\ 12 & 12 & 0 \\ 3 & 0 & 3 \end{pmatrix}\)

Solution:

1. \(2\begin{pmatrix} -1 & 2 \\ 0 & 4 \end{pmatrix} = \begin{pmatrix} 2\times -1 & 2\times 2 \\ 2\times 0 & 2\times 4 \end{pmatrix} = \begin{pmatrix} -2 & 4 \\ 0 & 8 \end{pmatrix}\)

2. \(\frac{\sqrt{3}}{3}\begin{pmatrix} 9 & 0 & 3 \\ 12 & 12 & 0 \\ 3 & 0 & 3 \end{pmatrix}= \begin{pmatrix} \frac{\sqrt{3}}{3}\times 9 & \frac{\sqrt{3}}{3}\times 0 & \frac{\sqrt{3}}{3}\times 3 \\ \frac{\sqrt{3}}{3}\times 12 & \frac{\sqrt{3}}{3}\times 12 & \frac{\sqrt{3}}{3}\times 0 \\ \frac{\sqrt{3}}{3}\times 3 & \frac{\sqrt{3}}{3}\times 0 & \frac{\sqrt{3}}{3}\times 3 \end{pmatrix} = \begin{pmatrix} -3\sqrt{3} & 0 & \sqrt{3} \\ 4\sqrt{3} & 4\sqrt{3} & 0 \\ \sqrt{3} & 0 & \sqrt{3} \end{pmatrix}\)

---

Multiplying a NumPy array by a scalar acts in the same way.

2 * np.array([[-1, 2], [0, 4]])

array([[-2,  4],
       [ 0,  8]])

np.sqrt(3) / 3 * np.array([[-9, 0, 3], [12, 12, 0], [3, 0, 3]])

array([[-5.19615242,  0.        ,  1.73205081],
       [ 6.92820323,  6.92820323,  0.        ],
       [ 1.73205081,  0.        ,  1.73205081]])

---

Matrix Multiplication

Suppose we want to find the product \(AB\) of the matrices \(A\) and \(B\). Then:

• The number of columns of \(A\) must be equal to the number of rows of \(B\).

• The product matrix will have the same number of rows as \(A\) and the same number of columns as \(B\).

Below, we have the rule to find the product for two 2×2 matrices:

\[\begin{split} AB = \begin{pmatrix} a_{11} & a_{12} \\ a_{21} & a_{22} \end{pmatrix} = \begin{pmatrix} a_{11}b_{11} + a_{12}b_{21} & a_{11}b_{12} + a_{12}b_{22} \\ a_{21}b_{11} + a_{22}b_{21} & a_{21}b_{12} + a_{22}b_{22} \end{pmatrix} \end{split}\]

In general, if we want to find the product \(AB\) of a \(m\times l\) matrix \(A\) and a \(k\times n\) matrix \(B\):

\[\begin{split} A = \begin{pmatrix} a_{11} & a_{12} & \cdots & a_{1l} \\ a_{21} & a_{22} & \cdots & a_{2l} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \cdots & a_{ml} \end{pmatrix} \end{split}\]

And

\[\begin{split} B = \begin{pmatrix} b_{11} & b_{12} & \cdots & b_{1n} \\ b_{21} & b_{22} & \cdots & b_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ b_{k1} & b_{k2} & \cdots & b_{kn} \end{pmatrix} \end{split}\]

So that:

\[\begin{split} AB = \begin{pmatrix} c_{11} & c_{12} & \cdots & c_{1n} \\ c_{21} & c_{22} & \cdots & c_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ c_{m1} & c_{m2} & \cdots & c_{mn} \end{pmatrix} \end{split}\]

Important

We calculate \(c_{ij}\) to be the scalar dot product of row \(i\) and column \(j\).

Example

Find the following products:

1. \(\begin{pmatrix} 1 & 0 \\ 2 & 1 \end{pmatrix} \begin{pmatrix} 3 & 1 \\ 2 & 1 \end{pmatrix}\)

2. \(\begin{pmatrix} 10 & 2 & 2 \\ 8 & -1 & 4 \end{pmatrix} \begin{pmatrix} -5 & -3 \\ 2 & 2 \end{pmatrix}\)

Solution:

1. As we are mulitplying a 2×2 matrix by another 2×2 matrix, the product matrix will also be a 2×2 matrix in the form below:

   \[\begin{split} \begin{pmatrix} a_{11} & a_{12} \\ a_{21} & a_{22} \end{pmatrix} \end{split}\]

   We now calculate the elements of the product matrix above:

   • To find \(a_{11}\) we take the dot product of row 1 and column 1:

     \[\begin{split} a_{11} = \begin{pmatrix} 1 \\ 0 \end{pmatrix} \cdot \begin{pmatrix} 3 \\ 2 \end{pmatrix} = 1 \times 3 + 0 \times 2 = 2 \end{split}\]

   • To find \(a_{12}\) we take the dot product of row 1 and column 2:

     \[\begin{split} a_{12} = \begin{pmatrix} 1 \\ 0 \end{pmatrix} \cdot \begin{pmatrix} 1 \\ 1 \end{pmatrix} = 1 \times 1 + 0 \times 1 = 1 \end{split}\]

   • To find \(a_{21}\) we take the dot product of row 2 and column 1:

     \[\begin{split} a_{21} = \begin{pmatrix} 2 \\ 1 \end{pmatrix} \cdot \begin{pmatrix} 3 \\ 2 \end{pmatrix} = 2 \times 3 + 1 \times 2 = 8 \end{split}\]

   • To find \(a_{22}\) we take the dot product of row 2 and column 2:

     \[\begin{split} a_{22} = \begin{pmatrix} 2 \\ 1 \end{pmatrix} \cdot \begin{pmatrix} 1 \\ 1 \end{pmatrix} = 2 \times 1 + 1 \times 1 = 3 \end{split}\]

   Therefore,

   \[\begin{split} \begin{pmatrix} 1 & 0 \\ 2 & 1 \end{pmatrix} \begin{pmatrix} 3 & 1 \\ 2 & 1 \end{pmatrix} = \begin{pmatrix} 3 & 1 \\ 8 & 3 \end{pmatrix} \end{split}\]

2. The number of columns of the matrix on the left does not equal the number of row in the matrix on the right, hence this product cannot be found!

---

It is possible to perform matrix multiplication between two NumPy arrays. However, to do this, we use the special operator @, or the function np.matmul.

np.array([[1, 0], [2, 1]]) @ np.array([[3, 1], [2, 1]])

array([[3, 1],
       [8, 3]])

NumPy will raise an error if the multiplication is forbidden.

np.array([[10, 2, 2], [8, -1, 4]]) @ np.array([[-5, -3], [2, 2]])

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[6], line 1
----> 1 np.array([[10, 2, 2], [8, -1, 4]]) @ np.array([[-5, -3], [2, 2]])

ValueError: matmul: Input operand 1 has a mismatch in its core dimension 0, with gufunc signature (n?,k),(k,m?)->(n?,m?) (size 2 is different from 3)

---

Note

Matrix multiplication is NOT commutative so \(AB \neq BA\).

Example

Let \(A = \begin{pmatrix} 1 & 1 \\ 4 & 0 \end{pmatrix}\) and \(B = \begin{pmatrix} -1 & 4 \\ 2 & -3 \end{pmatrix}\).

Then \(AB = \begin{pmatrix} 1 & 1 \\ -4 & 16 \end{pmatrix}\) and \(BA = \begin{pmatrix} 15 & -1 \\ -10 & 2 \end{pmatrix}\), i.e., \(AB\neq BA\).

---

This equality can be shown with Python logic.

A = np.array([[1, 1], [4, 0]])
B = np.array([[-1, 4], [2, -3]])

A @ B == B @ A

array([[False, False],
       [False, False]])

The NumPy array are compared element-wise, resulting in a boolean array of False values.

---