linear algebra notes

chapter 1

a symmetric matrix is a any matrix $A$ such that $A^t = A$.

$\text{span}(\empty) = \{0\}$ by definition

the span of any subset $S$ of a vector space $V$ is a subspace of $V$ that also contains $S$.

also, any subspace of $V$ that contains $S$ must also contain $\text{span}(S)$.

given $S_1 \subseteq S_2 \subseteq V$, if $S_2$ is linearly independent, then $S_1$ is linearly independent.

😱💯:

• the vector space of complex numbers over the field of complex numbers has dimension $1$. a basis is $\{1\}$.
• however, when taken over the field of real numbers, the same vector space has dimension $2$. a basis is $\{1,i\}$.

the vector space of polynomials has infinite dimension: $\{1, x, x^2, \dots\}$.

if $W$ is a subspace of a finite dimensional vector space $V$, then $\dim(W) \leq \dim(V)$. if $\dim(W) = \dim(V)$, then $W=V$.

if $W$ is a subspace of a finite dimensional vector space $V$, then any basis of $W$ can be extended to form a basis for $V$.

chapter 2

let $V$and $W$ be vector spaces. let $T:V \rightarrow W$ be a linear operator. the kernel or the null space of $T$ is defined as:

$$\ker(T):=\{\vec{x} \in V:T(\vec{x}) =0\}$$

the range or the image of $T$ is defined as:

$$\text{Im}(T):=\{T(\vec{x}) : \vec{x} \in V\}$$

both the kernel and the image are subspaces of $T$.

the image is a span of the basis vectors of $V$.

let $V$and $W$ be vector spaces. let $T:V \rightarrow W$ be a linear operator. if $V$ is finite dimensional, then:

$$\dim(\ker(T)) + \dim(\text{Im}(T)) = \dim(V)$$ $$\text{nullity}(T) + \text{rank}(T) = \dim(V)$$

let $V$and $W$ be vector spaces. let $T:V \rightarrow W$ be a linear operator. $T$ is one-to-one or injective $\iff$ $\ker(T) =\{0\}$.

let $V$and $W$ be vector spaces of equal dimension. let $T:V \rightarrow W$ be a linear operator. $T$ is one-to-one $\iff$ $T$ is onto $\iff$ $\dim(\text{Im}(T)) = \dim(V)$

let $V$and $W$ be vector spaces. let $T:V \rightarrow W$ be a linear operator. $T$ is isomorphic if $T$ is linear and invertible.

also, $V$ is isomorphic to $W \iff$ $\dim{V} = \dim{W}$.

let $V$ be a vector space over $\mathbb{F}$. then $V$ is isomorphic to $\mathbb{F}$ $\iff$ $\dim(V) = n$

$$V^* := \mathcal{L}(V, \mathbb{F})$$

where $V^*$ is known as the dual space of $V$.

given an ordered basis $\{\vec{v_1}, \vec{v_2}, \dots, \vec{v_n}\}$ for a finite dimensional vector space $V$, we can define an ordered basis$\{\vec{v_1}^*, \vec{v_2}^*, \dots, \vec{v_n}^*\}$ for the dual $V^*$. given any $f \in V^*$,

$$f=f(\vec{v_1})\vec{v_1}^* + \dots + f(\vec{v_n})\vec{v_n}^*$$

let $V$ and $W$ be finite dimensional vector spaces with bases $\beta_V$ and $\beta_W$ respectively. for any linear transformation $T:V \rightarrow W$, the map $T^*:W^* \rightarrow V^*$ is defined by $T^*(g)=g \circ T$ for all $g \in W^*$. this also satisfies the property $[T^*]_{\beta_W^*}^{\beta_V^*} = ([T]_{\beta_V}^{\beta_W})^t$, which is probably why friedberg calls $T^* = T^t$ 😡

chapter 5

2 $n \times n$ matrices $A$ and $B$ are said to be similar if there exists $P$ such that:

$$B=P^{-1}AP$$

• $A$ and $B$ have the same determinant
• $A$ and $B$ have the same eigenvalues
• $A$ and $B$ have the same trace
• $A$ and $B$ have the same rank

$P$ is usually known as the change of coordinates matrix. essentially, $A$ and $B$ are representing the same linear transformation, but in different bases. the choice of basis is completely arbitrary. we just decided to work in the standard basis from the start of time. but similar matrices tell us that if we chose a completely random other basis, and did the “same” linear transformation on that, we would be doing the same thing. 🤯

a linear operator $T$ is on a vector space $V$ is called diagonalizable:

• if there is an ordered basis $\beta$ for which $[T]_{\beta}$ is a diagonal matrix.
• if and only if there exists an ordered basis $\beta$ consisting of eigenvectors of $T$.
• if and only if $p_T(t)$ splits over $\mathbb{F}$ AND for each eigenvalue, algebraic multiplicity $=$ geometric multiplicity $=$ $\dim(\ker(T-\lambda I))$

if $T$ is diagonalizable and $\beta = \{\vec{v_1}, \dots, \vec{v_n}\}$ is an ordered basis of eigenvectors for $T$, then the matrix $[T]_{\beta}=D$ is a diagonal matrix where entries on the diagonal are eigenvalues. also, for an eigenvalue $d_{jj}$, it corresponds to the $j$th eigenvector $v_j$ for $1 \leq j \leq n$.

side note:

• let’s consider the linear transformation in $\mathbb{R}^2$ that rotates every vector in the plane $90$ degrees CCW (CCW or CW doesn’t actually matter for this example)
• given any $\vec{v} \in \mathbb{R^2}$, $T(\vec{v})$ is never gonna be a scalar multiple of $\vec{v}$ since it’s being rotated.
• so, there are transformations that don’t have any eigenvectors/eigenvalues, and so, they’re also not diagonalizable.

let $A \in \text{Mat}_{n \times n}(\mathbb{F})$.

• $t \in \mathbb{F}$ is an eigenvalue $\iff \det(A- tI_n) = 0$. this determinant thing is called the characteristic polynomial of $A$.
• $p_A(t)$ has degree $n$ with leading coefficient $(-1)^n$ (monic polynomial).
• $A$ has at most $n$ distinct eigenvalues.
• given $t \in \mathbb{F}$ is an eigenvalue, $\vec{v} \in \mathbb{F}^n$ is an eigenvector of $A$ $\iff$ $\vec{v} \neq 0$ and $(A-tI_n)\vec{v}=0$

let $T$ be a linear operator. let $\lambda_1, \dots, \lambda_k$ be distinct eigenvalues of $T$. for $1 \leq i \leq k$, let $S_i$ be a linearly independent finite set of eigenvectors corresponding to $\lambda_i$. then, the set $S_1 \cup \dots \cup S_k$ is also linearly independent.

let $T$ operate on a finite dimensional vector space with dimension $n$. if $T$ has $n$ distinct eigenvalues, then $T$ is diagonalizable. not the other way around though

the algebraic multiplicity of an eigenvalue $\lambda$ is the largest positive integer $m$ for which $(t-\lambda)^m$ is a factor of $p_T(t)$.

an eigenspace is defined as:

$$E_{\lambda}:=\{\vec{v} \in V:T(\vec{v})=\lambda \vec{v}\}=\ker(T-\lambda I)$$

imma define $\dim(E_\lambda)$ as the geometric multiplicity of $\lambda$. then $1 \leq \dim(E_{\lambda})\leq m$, where $m$ is the algebraic multiplicity of $\lambda$

let $T$ be a linear operator on a finite dimensional vector space. let’s say $T$ has distinct eigenvalues $\lambda_1, \dots, \lambda_k$ and that $p_T(t)$ splits. then we know that:

• $T$ is diagonalizable $\iff$for each eigenvalue, the algebraic multiplicity $=$ geometric multiplicity.
• given an ordered basis $\beta_i$ for $E_{\lambda_i}$, $\beta_1 \cup \dots \cup \beta_k$ forms an ordered basis for $V$ consisting of eigenvectors.

direct sums

given subspaces $W_1, \dots, W_k$ and $W$ of a vector space $V$ such that $W_i \subseteq W$ for $1 \leq i \leq k$. we call $W$ the direct sum of the subspaces $W_1, \dots, W_k$ and write $V=W_1 \oplus \dots \oplus W_k$ if:

• $V= \sum_{i=1}^k W_i$
• $W_j \cap \sum_{i \neq j} W_i = \{0\}, \forall j : 1 \leq j \leq k$

these are equivalent:

• $V=W_1 \oplus \dots \oplus W_k$.
• if $\vec{v} = \vec{v_1} + \dots + \vec{v_k}=0$, where $\vec{v_i} \in W_i$, then $\vec{v_1} = \dots = \dots{v_k} = 0$.
• for any $\vec{v} \in V$, $\vec{v}$ can uniquely be written as a sum: $\vec{v} = \vec{v_1} + \dots + \vec{v_k}$, where $\vec{v_i} \in W_i$.
• for any choice of bases $\gamma_i$ for $W_i$, their union $\gamma_1 \cup \dots \cup \gamma_k$ forms a basis for $V$.
• there exists at least one choice of bases $\gamma_i$ for $W_i$, such that their union $\gamma_1 \cup \dots \cup \gamma_k$ forms a basis for $V$.

a linear operator $T$ on a finite dimensional vector space is diagonalizable $\iff$ $V$ is the direct sum of the eigenspaces of $T$

invariant subspaces

given a linear operator $T$ on a finite dimensional vector space $V$, a subspace $W$ is called $T$ invariant if $T(W) \subseteq W$, that is, $T(\vec{v}) \in W, \forall \vec{v} \in W.$

here are some examples of $T$ invariant subspaces:

• $\{\vec{0}\}$
• $V$
• $\text{Im}(T)$
• $\ker(T)$
• $E_{\lambda}$, where $\lambda$ is any eigenvalue of $T$

cyclic subspaces

$$W:= \text{span}\{\vec{x}, T(\vec{x}), T^2(\vec{x}), \dots\}$$

$W$ is called a $T$ cyclic subspace generated by $\vec{x}$, $\vec{x} \in V, \vec{x} \neq \vec{0}$.

apparently, any $T$ invariant subspace of $V$ containing $\vec{x}$ must also contain $W$.

let $W$ be a $T$ invariant subspace of $V$. then, $p_{T|_W}(t)$ divides $p_T(t)$.

let $W$ be a $T$ cyclic subspace of $V$ generated by $\vec{x} \in V, \vec{x} = \vec{0}$. let $\dim(V) = k$. the set $\{\vec{x}, T(\vec{x}), T^2(\vec{x}), \dots, T^{k-1}(\vec{x})\}$ forms a basis for $W$.

cayley hamilton theorem

given $p_T(t)$, $p_T(T) = T_0$, the zero transformation.

given $p_A(t)$, $p_A(A)=O$, the zero matrix.

chapter 6

inner products

an inner product is a function that takes an ordered pair of vectors $\vec{x}, \vec{y} \in V$ and maps it to a scalar in $\mathbb{F}$. these properties also hold (axioms):

• $\langle x + z, y \rangle = \langle x,y \rangle + \langle z,y \rangle$
• $\langle c\vec{x}, \vec{y} \rangle = c \langle \vec{x}, \vec{y} \rangle$
• $\overline{\langle x, y \rangle} = \langle y, x \rangle$

$c \in \mathbb{F}; x,y \in V$

standard inner product

$$\langle x, y \rangle = \sum_{i=1} ^n x_i \overline{y_i}$$

when $\mathbb{F} = \mathbb{R}$ or $\mathbb{C}$.

---

given $A$ is an $m \times n$ matrix, the conjugate transpose or adjoint of $A$ is the $n \times m$ matrix $A^*$ such that $(A^*)_{ij}=\overline{A_{ji}}$.

---

other properties that can be proven (theorem 6.1): $\vec{x}, \vec{y}, \vec{z} \in V, c \in \mathbb{F}$

• $\langle x, y+z \rangle = \langle x, y \rangle + \langle x, z \rangle$
• $\langle x, cy \rangle = \overline{c} \langle x, y \rangle$
• $\langle x, 0 \rangle = \langle 0, x \rangle = 0_{\mathbb{F}}$

** first component linear, second component conjugate linear **

---

norm: $||x||=\sqrt{\langle x, x \rangle}$

---

theorem 6.2:

• $||cx|| = |c| \cdot ||x||$
• $||x|| \geq 0$
• $|\langle x,y \rangle| \leq ||x|| \cdot ||y||$ (cauchy-schwarz inequality)
• $||x+y|| \leq ||x|| + ||y||$ (triangle inequality)

---

theorem 6.3:

given an orthogonal subset $S = \{v_1, \dots, v_k\}$ of $V$ and $y \in \text{span}(S)$, then:

$$y=\sum_{i=1}^k \dfrac{\langle y, v_i \rangle}{||v_i||^2}v_i$$

---

theorem 6.4:

gram-schmidt

basically you can take a linearly independent subset of a finite dimensional inner product space $V$, and extend it to a orthogonal/orthonormal basis of $V$.

---

theorem 6.5:

given an orthonormal basis $\beta = \{v_1, \dots, v_n\}$ and $x \in V$:

$$x=\sum_{i=1}^n \langle x, v_i \rangle v_i$$

---

theorem 6.6:

given a finite dimensional subspace $W$ of $V$ and $y \in V$, there exists unique vectors $u \in W, v \in W^\perp$ such that $y = u + v$.

if $\beta=\{v_1, \dots, v_k\}$ is an orthonormal basis for $W$, then:

$$u=\sum_{i=1}^k\langle y, v_i \rangle v_i$$

---

theorem 6.7:

given $S = \{v_1, \dots, v_k\}$ is an orthonormal set:

• $S$ can be extended to an orthonormal basis $\{v_1, \dots, v_k, v_{k+1}, \dots, v_n\}$ for $V$.
• if $W = \text{span}(S)$, then the set $\{v_{k+1}, \dots, v_n\}$ is an orthonormal basis for $W^\perp$.
• if $W$ is any subspace of $V$, then $\dim(V) = \dim(W)+ \dim(W^\perp)$.

---

theorem 6.8:

let $g: V \rightarrow \mathbb{F}$ be a linear transformation. then, there exists a unique $y \in V$ such that $g(x) = \langle x, y \rangle, \forall x \in V$.

---

theorem 6.9:

given $T:V \rightarrow V$, there exists a unique linear function $T^* : V\rightarrow V$ such that $\langle T(x), y \rangle = \langle x, T^*(y) \rangle$.

$T^*$ is called the adjoint of $T$.

---

theorem 6.10:

let $\beta$ be an orthonormal basis for $V$. if $T$ is a linear endomorphism, then $[T^*]_{\beta}=[T]^*_{\beta}$.

---

theorem 6.11:

let $T$ and $U$ be linear endomorphisms whose adjoints for sure exist. then;

• $T+U$ has an adjoint. $(T+U)^*=T^* + U^*$
• $cT$ has an adjoint. $(cT)^* = \overline{c}T^*$
• $TU$ has an adjoint. $(TU)^* = U^*T^*$
• $T^*$ has an adjoint. $T^{**}=T$
• $I$ has an adjoint. $I^* = I$

same holds for matrices.

---

$T$ has an eigenvector $\implies$ $T^*$ has an eigenvector

---

theorem 6.14:

if the characteristic polynomial of $T$ splits, then there exists an orthonormal basis $\beta$ for $V$ such that $[T]_{\beta}$ is upper triangular.

---

$T$ is normal $\iff$ $TT^* = T^*T$

same for matrices

---

theorem 6.15:

given $T$ is normal:

• $||T(x)||=||T^*(x)||, \forall x \in V$
• $T-cI$ is normal $\forall c \in \mathbb{F}$
• $T(x)= \lambda x \implies T^*(x)=\overline{\lambda}x$
• $T(x_1)= \lambda_1x_1$ and $T(x_2) = \lambda_2x_2 \implies \langle x_1, x_2 \rangle = 0$ for distinct $\lambda_1$ and $\lambda_2$

---

theorem 6.16:

$T$ is normal $\iff$there exists an orthonormal basis for complex IPS $V$ consisting of eigenvectors of $T$

---

$T$ is called self adjoint if $T = T^*$

---

if $T$ is a self adjoint endomorphism on $V$, then:

• every eigenvalue of $T$ is real
  • if $V$ is a real IPS, then the characteristic polynomial of $T$ splits

---

theorem 6.17:

given $T$ is a linear endomorphism on a real IPS $V$, $T$ is self-adjoint $\iff$there exists an orthonormal basis for $V$ consisting of eigenvectors of $T$.

---

if $||T(x)|| = ||x||$, $\forall x \in V$, then $T$ is called an orthogonal operator over $\mathbb{R}$ or a unitary operator over $\mathbb{C}$.

---

note: since this type of operator preserves length, any kind of reflection/rotation in $\mathbb{R}^2$ is an orthogonal operator.

---

theorem 6.18:

these are equivalent:

• $T^*T=I$
• $TT^*=I$
• $\langle T(x), T(y) \rangle = \langle x, y \rangle, \forall x, y \in V$
• if $\mathcal{B}$ is an orthonormal basis for $V$, then $T(\mathcal{B})$ is an orthonormal basis for $V$
• there exists an orthonormal basis $\mathcal{B}$ for $V$ such that $T(\mathcal{B})$ is an orthonormal basis for $V$
• $||T(x)|| = ||x||, \forall x \in V$

---

to prove that $||T(x)|| = ||x|| \implies T^*T=I$, we need this lemma:

lemma: given a self-adjoint operator $U$ and $\langle x, U(x) \rangle = 0, \forall x \in V$, $U = T_0$, the zero transformation.

---

corollaries:

given $T$, $V$ has an orthonormal basis of eigenvectors of $T$ with corresponding eigenvalues $\pm 1$ $\iff$ $T$ is both self-adjoint and orthogonal for real IPS

OR

$T$ is unitary for complex IPS

---

theorem 6.23

given $T: \mathbb{R}^2 \rightarrow \mathbb{R}^2$ is an orthogonal linear operator, and $A = [T]_{\beta}$, where let’s assume $\beta$ is the standard ordered basis for $\mathbb{R}^2$. exactly one of these statements is true:

• $T$ is a rotation, and $\det(A) = 1$
• $T$ is a reflection about a line through the origin, and $\det(A) = -1$

---

let $T$ be a projection. we say that $T$ is an orthogonal projection if $\text{Im}(T)^\perp = \ker(T)$ and $\ker(T)^\perp=\text{Im}(T)$.

• $V = \ker(T) \oplus \text{Im}(T)$

---

note: orthogonal projections are different from orthogonal operators ‼️

---

theorem 6.24

$T$ is an orthogonal projection $\iff$ $T$ has an adjoint $T^*$, and $T^2 = T = T^*$.

---

theorem 6.25 (spectral theorem)

given $T$ on $V$ over $\mathbb{F}$ with the distinct eigenvalues $\lambda_1, \dots, \lambda_k$. let’s assume $T$ is normal if $\mathbb{F} = \mathbb{C}$ or $T$ is self-adjoint if $\mathbb{F} = \mathbb{R}$. let $E_{\lambda_i}$ be the eigenspace of $T$ corresponding to eigenvalue $\lambda_i$, and let $T_i$ be the orthogonal projection of $V$ on $E_{\lambda_i}$. the following hold:

• $V=E_{\lambda_1} \oplus \dots \oplus E_{\lambda_k}$
• let $E_{\lambda_i}'$ denote the sum of eigenspaces $E_{\lambda_j}$ for $i \neq j$, then $E_{\lambda_i}' = E_{\lambda_i}^\perp$.
• $T_iT_j=\delta_{ij}T_i, 1 \leq i,j \leq k$
• $I=T_1 + \dots + T_k$
• $T=\lambda_1T_1 + \dots + \lambda_k T_k$

---

corollaries:

• if $\mathbb{F} = \mathbb{C}$, then $T$ is unitary $\iff$ $T$ is normal and $|\lambda| = 1$ for every eigenvalue $\lambda$ of $T$.
• if $\mathbb{F} = \mathbb{C}$, then $T$ is self-adjoint $\iff$ $T$ is normal and every eigenvalue of $T$ is real.

---

---

reflections & rotations

bilinear forms

a function $H:V \times V \rightarrow \mathbb{F}$ is called a bilinear form on $V$ if $H$ is linear in each variable when the other variable is held fixed. aka satisfies the following:

• $H(ax_1+x_2,y)=aH(x_1,y) + H(x_2,y)$
• $H(y,ax_1+x_2)=aH(y,x_1) + H(y,x_2)$

---

theorem 6.31:

the sum of two bilinear forms and the product of a scalar and a bilinear form on $V$ are also bilinear forms on $V$. the set of all bilinear forms $\mathcal{B}(V)$ is a vector space with respect to these operations.

---

given an ordered basis $\beta = \{\vec{v_1}, \dots, \vec{v_n}\}$ for $V$, we can define a matrix $A$ corresponding to a bilinear form $H \in \mathcal{B}(V)$. let the $ij$ entry of $A$ be defined by:

$$A_{ij}=H(\vec{v_i}, \vec{v_j})$$

$A$ is called the matrix representation of $H$ with respect to the ordered basis $\beta$. it is denoted by $\psi_{\beta}(H):\mathcal{B}(V) \rightarrow \mathbb{F}^{n \times n}$.

---

theorem 6.32:

$\psi_{\beta}$ is an isomorphism as defined above.

---

corollary:

$\psi_{\beta}(H)=A \iff H(\vec{x}, \vec{y}) = [\phi_{\beta}(\vec{x})]^tA[\phi_{\beta}(\vec{y})]$

---

corollary:

there exists a unique matrix $A = \psi_{\beta}(H)$ such that:

$$H(\vec{x}, \vec{y})=\vec{x}^tA\vec{y}$$

---

let $A,B \in \mathbb{F}^{n \times n}$. $B$ is congruent to $A$ if there exists an invertible matrix $Q \in \mathbb{F}^{n \times n}$ such that $B=Q^tAQ$.

---

theorem 6.33:

given two bases $\beta$ and $\gamma$ for $V$, let $Q=[I]_{\gamma}^{\beta}$, the change of coordinates matrix from $\gamma$ coordinates to $\beta$ coordinates. for any $H \in \mathcal{B}(V)$, $\psi_{\gamma}(H)=Q^t\psi_{\beta}(H)Q$. thus, $\psi_{\gamma}$ is congruent to $\psi_{\beta}$.

---