\documentclass{article}%
\usepackage{amsfonts}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{graphicx}%
\setcounter{MaxMatrixCols}{30}
%TCIDATA{OutputFilter=latex2.dll}
%TCIDATA{Version=5.50.0.2960}
%TCIDATA{CSTFile=40 LaTeX article.cst}
%TCIDATA{Created=Friday, July 13, 2018 07:45:53}
%TCIDATA{LastRevised=Monday, July 23, 2018 08:16:18}
%TCIDATA{<META NAME="GraphicsSave" CONTENT="32">}
%TCIDATA{<META NAME="SaveForMode" CONTENT="1">}
%TCIDATA{BibliographyScheme=Manual}
%TCIDATA{<META NAME="DocumentShell" CONTENT="Standard LaTeX\Standard LaTeX Article">}
%TCIDATA{Language=American English}
%BeginMSIPreambleData
\providecommand{\U}[1]{\protect\rule{.1in}{.1in}}
%EndMSIPreambleData
\newtheorem{theorem}{Theorem}
\newtheorem{acknowledgement}[theorem]{Acknowledgement}
\newtheorem{algorithm}[theorem]{Algorithm}
\newtheorem{axiom}[theorem]{Axiom}
\newtheorem{case}[theorem]{Case}
\newtheorem{claim}[theorem]{Claim}
\newtheorem{conclusion}[theorem]{Conclusion}
\newtheorem{condition}[theorem]{Condition}
\newtheorem{conjecture}[theorem]{Conjecture}
\newtheorem{corollary}[theorem]{Corollary}
\newtheorem{criterion}[theorem]{Criterion}
\newtheorem{definition}[theorem]{Definition}
\newtheorem{example}[theorem]{Example}
\newtheorem{exercise}[theorem]{Exercise}
\newtheorem{lemma}[theorem]{Lemma}
\newtheorem{notation}[theorem]{Notation}
\newtheorem{problem}[theorem]{Problem}
\newtheorem{proposition}[theorem]{Proposition}
\newtheorem{remark}[theorem]{Remark}
\newtheorem{solution}[theorem]{Solution}
\newtheorem{summary}[theorem]{Summary}
\newenvironment{proof}[1][Proof]{\noindent\textbf{#1.} }{\ \rule{0.5em}{0.5em}}
\begin{document}

\title{LINEAR ALGEBRA OUTLINE\\LINEAR ALGEBRA OUTLINE}
\author{

}

\begin{center}
{\LARGE UNIVERSITY OF ARIZONA INTEGRATION WORKSHOP,}

{\LARGE AUGUST 2018}

{\small (Notes by Dan Madden based of previous notes by Bryden Cais, Nick
Rogers, Dinesh Thakur, and Doug Ulmer.)}
\end{center}

\section{Lecture 1}

\subsection{\bigskip Vector Spaces}

First we start with a field $F$. A field is a set with all the properties you
want numbers to have. There is a well defined addition; the addition is
associative and commutative. There is an additive identity we call $0$. Every
element has an additive inverse; we write the additive inverse of $a$ as $-a$.
There is a well defined multiplication; the multiplication is associative and
commutative. There is a multiplicative identity we call $1$. Every element
except $0$ has a multiplicative inverse; we write the multiplicative inverse
of $a$ as $a^{-1}$. Multiplication distributes over addition, and $0\neq1$.

For us, the field will almost always be the rational numbers $%
%TCIMACRO{\U{211a} }%
%BeginExpansion
\mathbb{Q}
%EndExpansion
$, the real numbers $%
%TCIMACRO{\U{211d} }%
%BeginExpansion
\mathbb{R}
%EndExpansion
$, the complex numbers $%
%TCIMACRO{\U{2102} }%
%BeginExpansion
\mathbb{C}
%EndExpansion
$. or $%
%TCIMACRO{\U{2124} }%
%BeginExpansion
\mathbb{Z}
%EndExpansion
_{p}$ the integers $%
%TCIMACRO{\U{2124} }%
%BeginExpansion
\mathbb{Z}
%EndExpansion
$ modulo a prime. Eventually we will consider the special properties of vector
spaces over $%
%TCIMACRO{\U{211a} }%
%BeginExpansion
\mathbb{Q}
%EndExpansion
$, $%
%TCIMACRO{\U{211d} }%
%BeginExpansion
\mathbb{R}
%EndExpansion
$, and $%
%TCIMACRO{\U{2102} }%
%BeginExpansion
\mathbb{C}
%EndExpansion
.$The issue is that these fields have orders that measure the size of the
numbers they contain and thus each has a topology. We will develop a linear
algebra for any field, and only add the topology when necessary. So keeping $%
%TCIMACRO{\U{2124} }%
%BeginExpansion
\mathbb{Z}
%EndExpansion
_{p}$ in mind as a field without a topology might be helpful.

\begin{definition}
A vector space $(V,+,\cdot)$ over a field $F$ is a non empty set $V$ with a
binary operation $+:V\times V\rightarrow V$ called addition and an operation
$\cdot:F\times V\rightarrow V$ called scalar multiplication. These operations
have the following properties:

\begin{enumerate}
\item Addition is associative: $\forall u,v,w\in V$, $(u+v)+w=u+(v+w);$

\item Addition is commutative: $\forall v,w\in V$, $v+w=w+v;$

\item There is an additive identity: $\exists0\in V$ such that $\forall v\in
V$, $v+0=0+v=v;$

\item Every element in $V$ has an additive identity: $\forall v\in V$,
$\exists w\in V$ such that $v+w=w+v=0;$

(We also say that $(V,+)$ forms an abelian group.);

\item For all $v\in V$, $1\cdot v=v$.

\item For all $a,b\in F$, and all $v\in V$, $a\cdot(b\cdot v)=(ab)\cdot v$.;

\item For all $a,b\in F$, and all $v\in V$, $(a+b)\cdot v=a\cdot v+b\cdot v;$

\item For all $a\in F$, and all $v,w\in V$, $a\cdot(v+w)=a\cdot v+a\cdot w.$
\end{enumerate}
\end{definition}

Usually it is a bad idea to use one symbol for two different things, but the
arithmetic that follows means that we can tell $0\in F$ from $0\in V$ by the
context of any statement without undo confusion.

There is a bit of arithmetic housekeeping to do before we get started.

\begin{remark}
The additive identity in a vector space is unique.

\begin{proof}
Suppose $0$ and $z$ are both identities, then $0=0+z=z$.
\end{proof}
\end{remark}

\begin{remark}
For all $v\in V$, the additive inverse of $v$ is unique

\begin{proof}
Suppose $u$ and $w$ are both additive inverses of $v$. Consider $u+v+w$.
\end{proof}
\end{remark}

\begin{remark}
For all $v\in V$, $0\cdot v=0$.

\begin{proof}
Consider $0\cdot v+0\cdot v$. Then $0\cdot v+0\cdot v=(0+0)\cdot v=0\cdot v$.
Thus adding the inverse of $\left(  0\cdot v\right)  $ to both sides gives
$0\cdot v=0$. (Notice that the first $0$ is clearly the scalar in the field
and the second $0$ is in the vector space.)
\end{proof}
\end{remark}

\begin{definition}
Let $V$ be a vector space and $W\subseteq V$. We say $W$ is a subspace of $V$
when $W$ is a vector space using the same addition and multiplication as $V$.
\end{definition}

To see if set is a vector space, we must check that all of the properties in
the definition hold. To see if a subset of a vector space is a subspace, we
only need to check a few conditions.

\begin{remark}
Let $V$ be a vector space over $F$ and $W\subseteq V$. If

\begin{enumerate}
\item $W\neq\emptyset$

\item $W$ is closed under addition: $\forall u,w\in W$, $u+w\in W;$

\item $W$ is closed under scalar multiplication: $\forall a\in F$ and $w\in
W$, $a\cdot w\in W,$
\end{enumerate}

Then $W$ is a subspace of $V$.
\end{remark}

\begin{definition}
Let $W_{i}\subseteq V$ for $i\in\mathcal{I}$ be a family of subspaces of the
vector space $V$. Then the sum of that family is%
\[
\sum_{i\in\mathcal{I}}W_{i}=\left\{  \sum_{\substack{i\in\mathcal{J}%
\\\mathcal{J\subseteq I}\text{ finite}}}w_{i}\;|\;w_{i}\in W_{i}\right\}  .
\]

\end{definition}

\bigskip

Note that even if there are infinitely many subspaces in the family, we can
only add finitely many vectors at a time. Thus the sum can only be taken over
finite subsets of the index set.

\begin{remark}
If $W_{i}\subseteq V$ for $i\in\mathcal{I}$ is a family of subspaces of the
vector space $V$, then $\sum_{i\in\mathcal{I}}W_{i}$ is a subspace of $V$.
\end{remark}

\begin{definition}
Let $W_{i}\subseteq V$ for $i\in\mathcal{I}$ be a family of subspaces of the
vector space $V$. Then the intersection of that family is%
\[%
%TCIMACRO{\tbigcap \limits_{i\in\mathcal{I}}}%
%BeginExpansion
{\textstyle\bigcap\limits_{i\in\mathcal{I}}}
%EndExpansion
W_{i}=\left\{  w\in V\;|\;\forall i\in\mathcal{I}\text{, }w\in W_{i}\right\}
.
\]

\end{definition}

\begin{remark}
If $W_{i}\subseteq V$ for $i\in\mathcal{I}$ is a family of subspaces of the
vector space $V$, then $\sum_{i\in\mathcal{I}}W_{i}$ is a subspace of $V$.
\end{remark}

\subsubsection{New Spaces from Old}

If we have any family of vector spaces, we can combine them to form a larger
vector space even if they are not subspaces of an even larger space and no
matter how many members the family may have.

\begin{definition}
Let $V_{i}$ for $i\in\mathcal{I}$ be a family the vector spaces. Then the
direct product of that family is%
\[%
%TCIMACRO{\tprod \limits_{i\in\mathcal{I}}}%
%BeginExpansion
{\textstyle\prod\limits_{i\in\mathcal{I}}}
%EndExpansion
V_{i}=\left\{  \varphi:\mathcal{I\rightarrow}%
%TCIMACRO{\tbigcup \limits_{i\in\mathcal{I}}}%
%BeginExpansion
{\textstyle\bigcup\limits_{i\in\mathcal{I}}}
%EndExpansion
V_{i}\;|\;\forall i\in\mathcal{I}\text{, }\varphi(i)\in V_{i}\right\}
\]
where%
\begin{align*}
\left(  \varphi+\psi\right)  (i)  &  =\varphi(i)+\psi(i)\\
\left(  a\cdot\varphi\right)  (i)  &  =a\cdot\varphi(i)
\end{align*}

\end{definition}

\begin{remark}
If $V_{i}$ for $i\in\mathcal{I}$ is a family the vector spaces, then the
direct product of that family is a vector space.
\end{remark}

To make sense of this, let $I=\{1,2\}$ then $%
%TCIMACRO{\tprod \limits_{i\in\mathcal{I}}}%
%BeginExpansion
{\textstyle\prod\limits_{i\in\mathcal{I}}}
%EndExpansion
V_{i}$ can be written as $\left\{  (v_{1},v_{2}\right\}  \;|\;v_{1}\in V_{1}$
and $v_{2}\in V_{2}\}$. \ Thus for $I=\{1,2\cdots n\},$ $%
%TCIMACRO{\tprod \limits_{i\in\mathcal{I}}}%
%BeginExpansion
{\textstyle\prod\limits_{i\in\mathcal{I}}}
%EndExpansion
V_{i}$ is a set of $n$-tuples of vectors. \ For $I=%
%TCIMACRO{\U{211d} }%
%BeginExpansion
\mathbb{R}
%EndExpansion
$, one must believe in $%
%TCIMACRO{\U{211d} }%
%BeginExpansion
\mathbb{R}
%EndExpansion
$-tuples to think of $%
%TCIMACRO{\tprod \limits_{i\in\mathcal{I}}}%
%BeginExpansion
{\textstyle\prod\limits_{i\in\mathcal{I}}}
%EndExpansion
V_{i}$. this way$.$



\begin{quotation}
"I can't believe that!" said Alice.

"Can't you?" the Queen said in a pitying tone. "Try again: draw a long breath,
and shut your eyes."

Alice laughed. "There's no use trying," she said: "one can't believe
impossible things."

"I daresay you haven't had much practice," said the Queen. "When I was your
age, I always did it for half-an-hour a day. Why, sometimes I've believed as
many as six impossible things before breakfast."

(Through the Looking-Glass, and What Alice Found There (1871) by Lewis Carroll )
\end{quotation}



\begin{definition}
Let $V_{i}$ for $i\in\mathcal{I}$ be a family the vector spaces. Then the
direct sum of that family is%
\[%
%TCIMACRO{\tbigoplus \limits_{i\in\mathcal{I}}}%
%BeginExpansion
{\textstyle\bigoplus\limits_{i\in\mathcal{I}}}
%EndExpansion
V_{i}=\left\{  \varphi:\mathcal{I\rightarrow}%
%TCIMACRO{\tbigcup \limits_{i\in\mathcal{I}}}%
%BeginExpansion
{\textstyle\bigcup\limits_{i\in\mathcal{I}}}
%EndExpansion
V_{i}\;|\;\forall i\in\mathcal{I}\text{, }\varphi(i)\in V_{i}\text{ and
}\varphi(i)=0\text{ for all but finitely many }i\in\mathcal{I}\right\}
\]
where%
\begin{align*}
\left(  \varphi+\psi\right)  (i)  &  =\varphi(i)+\psi(i)\\
\left(  a\cdot\varphi\right)  (i)  &  =a\cdot\varphi(i)
\end{align*}

\end{definition}

\begin{remark}
If $V_{i}$ for $i\in\mathcal{I}$ is a family the vector spaces, then the
direct sum of that family is a vector space and a subspace of the direct product.
\end{remark}

\begin{definition}
Let $V$ be a vector space and $W$ be a subspace of $V.$ \ If $v\in V$, then%
\[
v+W=\left\{  x\in V\;|\;\exists w\in W\text{ so that }x=v+w\right\}  .
\]
Now let
\[
V/W=\left\{  v+W\;|\;v\in V\right\}
\]

\end{definition}

The space $V/W$ is called "$V$ modulo $W$" and its element $v+W\;$is "$v$
modulo $W$."

\begin{remark}
If $v_{1}+W$ and $v_{2}+W\in V/W$, then $v_{1}+W$ $=v_{2}+W$ if and only if
$v_{1}-v_{2}\in W$.
\end{remark}

\begin{remark}
If $V$ is a vector space and $W$ is a subspace of $V$, then $V/W$ is a vector
space under%
\begin{align*}
\left(  v_{1}+W\right)  +\left(  v_{2}+W\right)   &  =\left(  v_{1}%
+v_{2}\right)  +W;\\
a\cdot\left(  v+W\right)   &  =\left(  a\cdot v\right)  +W
\end{align*}

\end{remark}

\subsubsection{Examples}

\begin{example}
$V=\{0\}$ is a vector space.
\end{example}

\begin{example}
If $V$ is a vector space, then $\{0\}$ and $V$ are both subspaces of $V$.
\end{example}

\begin{example}
If $F$ is any field, then for $n\in%
%TCIMACRO{\U{2115} }%
%BeginExpansion
\mathbb{N}
%EndExpansion
$,
\[
F^{n}=\{(a_{1},a_{2},\cdots a_{n})\,|\,a_{i}\in F\}
\]
is a vector space under coordinate-wise addition and scalar multiplication$.$
\end{example}

\begin{example}
If $F$ is any field, then the set of polynomials with coefficients in $F$ is a
vector space.
\[
F[x]=\left\{
%TCIMACRO{\tsum \limits_{k\in0}^{n}}%
%BeginExpansion
{\textstyle\sum\limits_{k\in0}^{n}}
%EndExpansion
a_{k}x^{k}\;|\;a_{k}\in F\text{.}\right\}
\]

\end{example}

\begin{example}
If $F$ is any field, then the set of power series in $x$ with coefficients in
$F$ is a vector space.
\[
F[[x]]=\left\{
%TCIMACRO{\tsum \limits_{k\in0}^{\infty}}%
%BeginExpansion
{\textstyle\sum\limits_{k\in0}^{\infty}}
%EndExpansion
a_{k}x^{k}\;|\;a_{k}\in F\right\}
\]

\end{example}

\begin{example}
If $F=%
%TCIMACRO{\U{211a} }%
%BeginExpansion
\mathbb{Q}
%EndExpansion
$, $%
%TCIMACRO{\U{211d} }%
%BeginExpansion
\mathbb{R}
%EndExpansion
$, or $%
%TCIMACRO{\U{2102} }%
%BeginExpansion
\mathbb{C}
%EndExpansion
$, then the set of power series in $x$ with coefficients in $F$ and positive
radius of convergence is a vector space.
\[
F[[x]]_{C}=\left\{
%TCIMACRO{\tsum \limits_{k\in0}^{\infty}}%
%BeginExpansion
{\textstyle\sum\limits_{k\in0}^{\infty}}
%EndExpansion
a_{k}x^{k}\;|\;a_{k}\in F\text{ and }%
%TCIMACRO{\tsum \limits_{k\in0}^{\infty}}%
%BeginExpansion
{\textstyle\sum\limits_{k\in0}^{\infty}}
%EndExpansion
a_{k}x^{k}\text{ converges for }|x|<r\text{ with }r>0\right\}  \text{.}%
\]

\end{example}

\begin{example}
Forgetting about the multiplication, $%
%TCIMACRO{\U{2102} }%
%BeginExpansion
\mathbb{C}
%EndExpansion
$ is a vector space over $%
%TCIMACRO{\U{211d} }%
%BeginExpansion
\mathbb{R}
%EndExpansion
$.
\end{example}

\begin{example}
Forgetting about the multiplication, $%
%TCIMACRO{\U{2102} }%
%BeginExpansion
\mathbb{C}
%EndExpansion
$ and $%
%TCIMACRO{\U{211d} }%
%BeginExpansion
\mathbb{R}
%EndExpansion
$ are both vector spaces over $%
%TCIMACRO{\U{211a} }%
%BeginExpansion
\mathbb{Q}
%EndExpansion
$.
\end{example}

\begin{example}
The set of functions $f:%
%TCIMACRO{\U{211d} }%
%BeginExpansion
\mathbb{R}
%EndExpansion
\rightarrow%
%TCIMACRO{\U{211d} }%
%BeginExpansion
\mathbb{R}
%EndExpansion
$ that are continuous on $[0,1]$ form a vector space over $%
%TCIMACRO{\U{211d} }%
%BeginExpansion
\mathbb{R}
%EndExpansion
$.
\end{example}



\subsubsection{Span and Independence}

Let $V$ be a vector space over $F$ and $S\subseteq V$. Then the span of $S$ in
$V$ is
\[
<S>\,=\left\{
%TCIMACRO{\tsum \limits_{i=1}^{k}}%
%BeginExpansion
{\textstyle\sum\limits_{i=1}^{k}}
%EndExpansion
a_{i}s_{i}\;|\;k\in%
%TCIMACRO{\U{2115} }%
%BeginExpansion
\mathbb{N}
%EndExpansion
,\,a_{i}\in F,\,\text{and }s_{i}\in S\right\}  \text{.}%
\]
Note that you can only take finitely many sums in a vector space.

\begin{remark}
If $V$ is a vector space over $F$ and $S\subseteq V$., then the span of $S$ is
a subspace$.$
\end{remark}

\begin{definition}
Let $V$ be a vector space over $F$ and $S\subseteq V$. We say $S$ spans $V$ or
$S$ generates $V$ when $<S>=V$.
\end{definition}

\begin{definition}
Let $V$ be a vector space over $F$. We say $V$ is finitely generated when
there is a finite $S\subseteq V$ where $<S>=V$.
\end{definition}

\begin{definition}
Let $V$ be a vector space over $F$ and $S\subseteq V$. We say $S$ is linearly
independent when
\[
\text{If }%
%TCIMACRO{\tsum \limits_{i=1}^{k}}%
%BeginExpansion
{\textstyle\sum\limits_{i=1}^{k}}
%EndExpansion
a_{i}s_{i}\text{ where }k\in%
%TCIMACRO{\U{2115} }%
%BeginExpansion
\mathbb{N}
%EndExpansion
,\,a_{i}\in F\,s_{i}\in S\text{, then }\forall i\text{, }a_{i}=0\text{.}%
\]

\end{definition}

\begin{remark}
If $V$ be a vector space over $F$ and $S\subseteq V$ is linearly independent
then
\begin{align*}
\text{If }%
%TCIMACRO{\tsum \limits_{i=1}^{k}}%
%BeginExpansion
{\textstyle\sum\limits_{i=1}^{k}}
%EndExpansion
a_{s_{i}}s_{i}\text{ }  &  =%
%TCIMACRO{\tsum \limits_{i=1}^{l}}%
%BeginExpansion
{\textstyle\sum\limits_{i=1}^{l}}
%EndExpansion
b_{s_{i}}t_{i}\text{ where }k,l\in%
%TCIMACRO{\U{2115} }%
%BeginExpansion
\mathbb{N}
%EndExpansion
,\,a_{i},b_{i}\in F\,s_{i},t_{i}\in S\text{, }\\
\text{then }k  &  =l,\{s_{1}\cdots s_{k}\}=\{t_{1}\cdots t_{l}\}\text{
\ }\forall i\text{, }a_{s_{i}}=b_{s_{i}}\text{.}%
\end{align*}

\end{remark}

When $S\subseteq V$ is linearly independent and $v=%
%TCIMACRO{\tsum \limits_{i=1}^{k}}%
%BeginExpansion
{\textstyle\sum\limits_{i=1}^{k}}
%EndExpansion
a_{s_{i}}s_{i},$ we say $a_{s_{i}}$ is the $s_{i}$ coordinate of $v$. \ The
last remark says that the coordinates of every vector in the span of a
independent set of vectors are unique.

\begin{definition}
Let $V$ be a vector space over $F$ and $S\subseteq V$. We say that $S$ is a
basis of $V$ if it generates $V$ and is linearly independent.
\end{definition}



\subsubsection{Bases}

The following theorem is fundamental. To prove it in its full generality, one
needs a complete collection of logical tricks including the Axiom of Choice
(actually Zorn's Lemma.) First we define two terms.

\begin{definition}
Let $U$ be a set. Let $\mathcal{S}$ be a set of subsets of $U$. We say that
$M\in\mathcal{S}$ $\ $is maximal in $\mathcal{S}$ \ when, if $M\subseteq
T\subseteq U$, then either $M=T$ or $T\notin\mathcal{S}$.
\end{definition}

\begin{definition}
Let $U$ be a set. Let $\mathcal{S}$ be a set of subsets of $U$. We say that
$M\in\mathcal{S}$ $\ $is minimal in $\mathcal{S}$ \ when, if $T\subseteq
M\subseteq U$, then either $M=T$ or $T\notin\mathcal{S}$.
\end{definition}

\begin{theorem}
Let $V$ be any vector space over $F$.

\begin{enumerate}
\item Then $V$ has a basis.

\item Any two bases of $V$ have the same cardinality$.$(If $S_{i}$ and $S_{2}$
are bases of $V$, then there is a bijection $b:S_{i}$ $\rightarrow S_{2}$.)

\item Any independent subset of $V$ that is maximal in the set of independent
subsets of $V$ is a basis.

\item Any subset that generates $V$ that is minimal in the set of generating
subsets of $V$ is a basis.
\end{enumerate}
\end{theorem}

It is possible to define an intrinsic property of any vector space of any size
called its dimension without regard to one of its bases, but doing so requires
that " complete collection of logical tricks" we mentioned. \ Suffice it to
say, however:

\begin{definition}
If $V$ is a vector space over $F$, then the dimension of $V$ over $F$ is
Dim$_{F}(V)=\,$Card$(S)$ where $S$ is a basis of $V$.
\end{definition}

Despite what you might expect, \ bases of large vector spaces are not always
very valuable. It may be that a basis is impossible to actually find. Even if
a basis is known, it may be of limited use. It may be that only bases with
extra properties in a vector space - itself with extra properties - provides
any useful information. But for finitely generated vector spaces, bases play
an important role.

\begin{remark}
If $V$ is a vector space that is finitely generated over $F$, then $F$ has a
finite bases and Dim$_{F}(V)$ is a nonnegative integer.
\end{remark}

\subsection{Linear Transformations}

\subsubsection{Basics}

\begin{definition}
Let $V$ and $W$ be vector spaces over $F$. A linear map, linear function or
linear transformation is a function $T:V\rightarrow W$ so that%
\begin{align*}
\forall v_{1}\text{, }v_{2}  &  \in V\text{, }T(v_{1}+v_{2})=T(v_{1}%
)+T(v_{2})\\
\forall a  &  \in F\text{, }v\in V\text{, }T(av)=aT(v)
\end{align*}

\end{definition}

\begin{definition}
Let $V$ and $W$ be vector spaces over $F$. Let $T:V\rightarrow W$ be a linear
transformation. The kernel of $T$ is
\[
\text{Ker}(T)=\left\{  v\in V\;|\;T(v)=0\right\}
\]

\end{definition}

\begin{definition}
Let $V$ and $W$ be vector spaces over $F$. Let $T:V\rightarrow W$ be a linear
transformation. We say $T$ is an isomorphism if $T$ is bijective. In that case
we write $V\cong W$.
\end{definition}

\begin{remark}
Let $V$ and $W$ be vector spaces over $F.$ Then $V$ and $W$ are isomorphic if
and only if Dim$_{F}(V)=$Dim$_{F}(W)$ While being isomprphic sort of means
"almost the same," for vector spaces it does not say all that more than
Dim$_{F}(V)=$Dim$_{F}(W)$. This is some information when the cardinalities are
natural numbers, but not so much when they are infinite.
\end{remark}

\begin{example}
Let $F$ be any field, consider the vector space.
\[
V=F[[x]]=\left\{
%TCIMACRO{\tsum \limits_{k\in0}^{\infty}}%
%BeginExpansion
{\textstyle\sum\limits_{k\in0}^{\infty}}
%EndExpansion
a_{k}x^{k}\;|\;a_{k}\in F\right\}
\]
and the subspace%
\[
W=\left\{
%TCIMACRO{\tsum \limits_{k\in0}^{\infty}}%
%BeginExpansion
{\textstyle\sum\limits_{k\in0}^{\infty}}
%EndExpansion
a_{k}x^{2k}\;|\;a_{k}\in F\right\}  .
\]
Here $W\subseteq V$, $W\neq V$, but nevertheless $W\cong V$.
\end{example}

\begin{notation}
Let $V$ be a vector space over $F$. Let $W_{1}$ and $W_{2}$ be subspaces of
$V$. Then
\[
\left(  W_{1}+W_{2}\right)  \cong\left(  W_{1}\oplus W_{2}\right)  \text{ \ if
and only if }W_{1}\cap W_{2}=\{0\}\text{.}%
\]
Because of this we use $\left(  W_{1}\oplus W_{2}\right)  $ is mean $\left(
W_{1}+W_{2}\right)  $ and $W_{1}\cap W_{2}=\{0\}$. Also for families of
subsets, we use the direst sum to mean the sum where the subspaces have
pairwise trivial intersection. That is
\[%
%TCIMACRO{\tbigoplus \limits_{i\in\mathcal{I}}}%
%BeginExpansion
{\textstyle\bigoplus\limits_{i\in\mathcal{I}}}
%EndExpansion
W_{i}=%
%TCIMACRO{\tsum \limits_{i\in\mathcal{I}}}%
%BeginExpansion
{\textstyle\sum\limits_{i\in\mathcal{I}}}
%EndExpansion
W_{i}\text{ \ \ where \ }\left(  \text{if }i\neq j\text{, then }W_{i}\cap
W_{j}=\{0\}\right)  \text{.}%
\]

\end{notation}

\subsubsection{Kernels and Images}

\begin{definition}
Let $V$ and $W$ be vector spaces over $F$. Let $T:V\rightarrow W$ be a linear
transformation. The image of $T$ is
\[
\text{Im}(T)=\left\{  w\in W\;|\;\exists v\in V\text{ such that }%
T(v)=w\right\}  \text{.}%
\]

\end{definition}

\begin{remark}
Let $V$ and $W$ be vector spaces over $F$. Let $T:V\rightarrow W$ be a linear
transformation. The kernel of $T$ is a subspace of $V$ and the image of $T$ is
a subspace of $W$.
\end{remark}

\begin{theorem}
Let $V$ and $W$ be vector spaces over $F$. Let $T:V\rightarrow W$ be a linear
transformation. Then there is a canonical isomorphism
\[
m:V/\text{Ker}(T)\rightarrow\text{Im}(T)
\]
given by $I(v)=v+$Ker$(T)$.
\end{theorem}

\begin{definition}
Let $V$ and $W$ be vector spaces over $F$. Let $T:V\rightarrow W$ be a linear
transformation. The rank of $T$ is%
\[
\text{Rank}(T)=\text{Dim}_{F}(\text{Im}(T))
\]

\end{definition}

\begin{definition}
Let $V$ and $W$ be vector spaces over $F$. Let $T:V\rightarrow W$ be a linear
transformation. The nullity of $T$ is%
\[
\text{Null}(T)=\text{Dim}_{F}(\text{Ker}(T))
\]

\end{definition}

\begin{remark}
If $V$ is finitely generated, and $T:V\rightarrow W$ is a linear
transformation, then
\[
\text{Rank}(T)+\text{Null}(T)=\text{Dim}_{F}(V)\text{.}%
\]


\begin{proof}
We start with a basis $R$ of Ker$(T)$ complete it to a basis of $S$ of $V$ and
then prove $\{v+W\in V/W\;|\;v\in S\backslash R\}$ is a basis of $V/W$.
\end{proof}
\end{remark}

We can also refer to a linear transformation between vectors paces as a
homomorphism of vector spaces.

\begin{definition}
Let $V$ and $W$ be vector spaces over $F$. Then
\[
\text{Hom}_{F}(V,W)=\left\{  T:V\rightarrow W|\;T\text{ is a linear
map}\right\}  \text{.}%
\]

\end{definition}

\begin{remark}
Let $V$ and $W$ be vector spaces over $F$. Then Hom$_{F}(V,W)$ is a vector
space over $F$ under he operations
\begin{align*}
\left(  T_{1}+T_{2}\right)  (v) &  =T_{1}(v)+T_{2}(v)\\
\left(  a\cdot T\right)  (v) &  =aT(v)
\end{align*}

\end{remark}

The identity function on a vector space $V$, $I:V-V$ is always an element of
Hom$_{F}(V,V).$If $a\in F$ then scalar multiplication by $a$ is also a linear
transformation from $V$ $\rightarrow$ $V$. To make it clear that we are using
it as such we might choose to write it as $aI:V\rightarrow V$.

\subsubsection{}

\subsection{Eigenvalues and Eigenvectors}

\begin{definition}
Let $V$ be any vector space over $F$. Let $T:V\rightarrow V$ be a linear
transformation$.$We say $v\in V$ is an eigenvector of $T$ when $v\neq0$ and
there is a $\lambda\in F$ so that%
\[
T(v)=\lambda\cdot v
\]
We call the scalar $\lambda$ an eigenvalue of $T$.
\end{definition}

\begin{definition}
If $\lambda$ is an eigenvalue of $T$ then the eigenspace of $\lambda$ is the
subspace given by%
\[
V_{\lambda}=\{v\in V\;|\;T(v)=\lambda\cdot v\}
\]

\end{definition}

By virtue of our definitions, no eigenspace is simply $\{0\}$.

For reasons to be seen later, we will generalize the ideas of eigenvector and
eigenspace. \ Notice that, for an eigenvalue $\lambda$, the eigenspace of
$\lambda$ can be written as%
\[
V_{\lambda}=\{v\in V\;|\;(T-\lambda I)(v)=0\}.
\]
This leads us to

\begin{definition}
If $\lambda$ is an eigenvalue of $T:V\rightarrow V$ then the generalized
eigenspace of $\lambda$ is the subspace given by%
\[
U_{\lambda}=\{v\in V\;|\;\exists k\in%
%TCIMACRO{\U{2115} }%
%BeginExpansion
\mathbb{N}
%EndExpansion
\text{ so that }(T-\lambda I)^{k}(v)=0
\]

\end{definition}

The idea behind this is that we might hope that we can use $T$ to decompose
$V$ into its $T$-eigenspaces. But if we cannot, maybe we can use the
generalized eigenspaces.



\section{Lecture 2}

\subsection{Linear Transforms of Finitely Generated Vector Spaces}

\begin{definition}
Let $V$ and $W$ \ be finitely generated vector spaces over $F$ and
$T:V\rightarrow W$ is a linear transformation.

Further let $S=\{s_{1},s_{2},\cdots s_{n}\}$ be a basis of $V$ and
$R=\{r_{1},r_{2},\cdots r_{m}\}$ be a basis of $W$. Since $T(s_{i})\in W$, we
can write%
\[
T(s_{j})=%
%TCIMACRO{\tsum \limits_{i=1}^{m}}%
%BeginExpansion
{\textstyle\sum\limits_{i=1}^{m}}
%EndExpansion
a_{i,i}r_{i}.
\]
The matrix of $T$ with respect to $S$ and $T$ is
\begin{align*}
_{R}\left\vert T\right\vert _{S}  &  =(a_{i,j})_{m\times n}\\
&  =%
\begin{pmatrix}
a_{1,1} & a_{1,3} & a_{1,3} & \cdots & a_{1,n}\\
a_{2,1} & a_{2,2} & a_{2,3} & \cdots & a_{2,n}\\
a_{3,1} & a_{3,1} & a_{3,1} & \cdots & a_{3,n}\\
\vdots & \vdots & \vdots & \vdots & \vdots\\
a_{m,1} & a_{m,2} & a_{m,3} & \cdots & a_{m,n}%
\end{pmatrix}
\end{align*}

\end{definition}

Take particular notice of the notation including the indices. There is no
absolute standard. Because we like functions acting on things to their right,
we set our notation to reflect this. For us $_{R}\left\vert T\right\vert _{S}$
represents $T:V\rightarrow W$ from a vector space with basis $S$ to a vector
space with basis $R$. So a transformation from a dimension $n$ space to a
dimension $m$ space has an $m\times n$ matrix. The coefficients in the matrix
are based on sum $T(s_{i})=%
%TCIMACRO{\tsum \limits_{j=1}^{m}}%
%BeginExpansion
{\textstyle\sum\limits_{j=1}^{m}}
%EndExpansion
a_{j,i}r_{j}$ in this way to keep our right/left idea.

\ Of course, this comes clear when we write out the coordinates of any $v$ in
the basis $S$.%
\[
v=%
%TCIMACRO{\tsum \limits_{k=1}^{n}}%
%BeginExpansion
{\textstyle\sum\limits_{k=1}^{n}}
%EndExpansion
b_{k}s_{k}\text{.}%
\]
Then%
\begin{align*}
T(v)  &  =T\left(
%TCIMACRO{\tsum \limits_{k=1}^{n}}%
%BeginExpansion
{\textstyle\sum\limits_{k=1}^{n}}
%EndExpansion
b_{k}s_{k}\right) \\
&  =%
%TCIMACRO{\tsum \limits_{k=1}^{n}}%
%BeginExpansion
{\textstyle\sum\limits_{k=1}^{n}}
%EndExpansion
b_{k}T(s_{k})\text{.}\\
&  =%
%TCIMACRO{\tsum \limits_{k=1}^{n}}%
%BeginExpansion
{\textstyle\sum\limits_{k=1}^{n}}
%EndExpansion
b_{k}\left(
%TCIMACRO{\tsum \limits_{i=1}^{m}}%
%BeginExpansion
{\textstyle\sum\limits_{i=1}^{m}}
%EndExpansion
a_{k,i}r_{k}\right) \\
&  =%
%TCIMACRO{\tsum \limits_{k=1}^{n}}%
%BeginExpansion
{\textstyle\sum\limits_{k=1}^{n}}
%EndExpansion
\left(
%TCIMACRO{\tsum \limits_{i=1}^{m}}%
%BeginExpansion
{\textstyle\sum\limits_{i=1}^{m}}
%EndExpansion
a_{k,i}b_{k}r_{k}\right) \\
&  =%
%TCIMACRO{\tsum \limits_{i=1}^{m}}%
%BeginExpansion
{\textstyle\sum\limits_{i=1}^{m}}
%EndExpansion
\left(
%TCIMACRO{\tsum \limits_{k=1}^{n}}%
%BeginExpansion
{\textstyle\sum\limits_{k=1}^{n}}
%EndExpansion
a_{k,i}b_{k}\right)  r_{i}\text{.}%
\end{align*}


But the entries in $_{R}\left\vert T\right\vert _{S}=(a_{i,j})_{m\times n}$
\ are given by $T(s_{j})=%
%TCIMACRO{\tsum \limits_{i=1}^{m}}%
%BeginExpansion
{\textstyle\sum\limits_{i=1}^{m}}
%EndExpansion
a_{i,i}r_{i}.$ so that
\[%
\begin{pmatrix}
a_{1,1} & a_{1,2} & a_{1,3} & \cdots & a_{1,n}\\
a_{2,1} & a_{2,2} & a_{2,3} & \cdots & a_{2,n}\\
a_{3,1} & a_{3,1} & a_{3,1} & \cdots & a_{3,n}\\
\vdots & \vdots & \vdots & \vdots & \vdots\\
a_{m,1} & a_{m,2} & a_{m,3} & \cdots & a_{m,n}%
\end{pmatrix}%
\begin{pmatrix}
b_{1}\\
b_{2}\\
b_{3}\\
\vdots\\
b_{n}%
\end{pmatrix}
=%
\begin{pmatrix}
c_{1}\\
c_{2}\\
c_{3}\\
\vdots\\
c_{m}%
\end{pmatrix}
\]
gives
\[
T(v)=%
%TCIMACRO{\tsum \limits_{k=1}^{m}}%
%BeginExpansion
{\textstyle\sum\limits_{k=1}^{m}}
%EndExpansion
c_{k}r_{k}\text{.}%
\]
Not only that, but the definition of matrix multiplication is designed so that
it respects composition of linear maps. If $V,W,$and $U$ are finitely
generated vector spaces over $F$, with bases $Q,R$ and $S$ respectively. If
$T_{1}:V\rightarrow W$ and $T_{2}:W\rightarrow U$, then%
\[
\left(  _{S}\left\vert T_{1}\right\vert _{R}\right)  \left(  _{R}\left\vert
T_{1}\right\vert _{Q}\right)  =\left(  _{S}\left\vert T_{2}\circ
T_{1}\right\vert _{Q}\right)
\]
where $T_{2}\circ T_{1}:V\rightarrow U$ is the composition of the linear maps.

\begin{remark}
With this connection made, if $V$and $W$ are finitely generated vector spaces
over $F$, with bases $R$ and $S$ \ then%
\[
\text{Hom}_{F}(V,W)\cong\text{Mat}_{m\times n}(F)
\]
where the isomorphism definitely depends on the bases.
\end{remark}

\begin{remark}
Let $V$ and $W$ be finitely generated vector spaces over $F$, with bases
$S=\{s_{1},s_{2},\cdots s_{n}\}$ and $R=\{r_{1},r_{2},\cdots r_{m}\}$
respectively \ Let $T:V\rightarrow W$ be a linear transformation. Let
\[
_{R}\left\vert T\right\vert _{S}=%
\begin{pmatrix}
a_{1,1} & a_{1,3} & a_{1,3} & \cdots & a_{1,n}\\
a_{2,1} & a_{2,2} & a_{2,3} & \cdots & a_{2,n}\\
a_{3,1} & a_{3,1} & a_{3,1} & \cdots & a_{3,n}\\
\vdots & \vdots & \vdots & \vdots & \vdots\\
a_{m,1} & a_{m,2} & a_{m,3} & \cdots & a_{m,n}%
\end{pmatrix}
\]


If we find all solutions to
\[%
\begin{pmatrix}
a_{1,1} & a_{1,3} & a_{1,3} & \cdots & a_{1,n}\\
a_{2,1} & a_{2,2} & a_{2,3} & \cdots & a_{2,n}\\
a_{3,1} & a_{3,1} & a_{3,1} & \cdots & a_{3,n}\\
\vdots & \vdots & \vdots & \vdots & \vdots\\
a_{m,1} & a_{m,2} & a_{m,3} & \cdots & a_{m,n}%
\end{pmatrix}%
\begin{pmatrix}
x_{1}\\
x_{2}\\
x_{3}\\
\vdots\\
x_{n}%
\end{pmatrix}
=%
\begin{pmatrix}
0\\
0\\
0\\
\vdots\\
0
\end{pmatrix}
\]
then
\[
\text{Ker}(T)=\left\{
%TCIMACRO{\tsum \limits_{k=1}^{n}}%
%BeginExpansion
{\textstyle\sum\limits_{k=1}^{n}}
%EndExpansion
x_{i}s_{i}\;|\;(x_{1},\cdots x_{n})\text{ is a solution to the matrix equation
above}\right\}  .
\]


Further, if we consider the columns of $_{R}\left\vert T\right\vert _{S}$ as
vectors in the space $F^{m}$ \ and let $\mathcal{I=}<$columns$>$, then%

\[
\text{Im}(T)=\left\{
%TCIMACRO{\tsum \limits_{k=1}^{m}}%
%BeginExpansion
{\textstyle\sum\limits_{k=1}^{m}}
%EndExpansion
b_{i}r_{i}\;|\;(b_{1},\cdots b_{m})\in I\right\}  .
\]

\end{remark}

\subsection{Change of Basis}

\begin{definition}
Let $V$ be a vector space over $F$. The automorphisms of $V$ are
\[
\text{Aut}_{F}(V)=\left\{  T:V\rightarrow V|\;T\text{ is a bijective linear
map}\right\}
\]

\end{definition}

Now Aut$_{F}(V)\ $and Aut$_{F}(W)\ \ $can be used as method for changing the
bases used to set the isomorphism Hom$_{F}(V,W)\cong\,$Mat$_{m\times n}(F)$.

\begin{theorem}
Let $V$ and $W$ be finitely generated vector spaces over $F$. Let
\begin{align*}
S_{1}  &  =\{s_{1,1},s_{1,2},s_{1,3}\cdots s_{1,n}\}\\
S_{2}  &  =\{s_{2,1},s_{2,2},s_{2,3}\cdots s_{2,n}\}
\end{align*}
be two bases of $V$, and let
\begin{align*}
R_{1}  &  =\{r_{1,1},r_{1,2},r_{1,3}\cdots r_{1,m}\}\\
R_{2}  &  =\{r_{2,1},r_{2,2},r_{2,3}\cdots r_{2,m}\}
\end{align*}
be two bases of $W$. (Note the order will definitely matter.)

We can create an element $C\in$ Aut$_{F}(V)$ (the automorphisms of $V$) by
extending the formula%
\[
C(s_{1,i})=s_{2,i}%
\]
linearly to all vectors of $V$. Similarly, we create $D\in$ Aut$_{F}(V)$ by
extending the formula%
\[
D(r_{1,i})=r_{2,i}%
\]
linearly to all vectors of $W$.

Let $T:V\rightarrow W$. Then
\begin{align*}
\left(  _{R_{2}}\left\vert T\right\vert _{S_{2}}\right)   &  =\left(  _{R_{2}%
}\left\vert D\right\vert _{R_{1}}\right)  \left(  _{R_{1}}\left\vert
T\right\vert _{S_{1}}\right)  \left(  _{S_{1}}\left\vert C\right\vert _{S_{2}%
}\right) \\
&  =\left(  _{R_{2}}\left\vert D\right\vert _{R_{1}}\right)  \left(  _{R_{1}%
}\left\vert T\right\vert _{S_{1}}\right)  \left(  _{S_{2}}\left\vert
C\right\vert _{S_{1}}\right)  ^{-1}%
\end{align*}

\end{theorem}

Thus if we think of $\left(  _{R_{2}}\left\vert D\right\vert _{R_{1}}\right)
$ as the matrix that changes basis $R_{2}$ to basis $R_{1}$ as the notation
suggests, we can change the $W$ basis of a matrix for a transformation
$T:V\rightarrow W$. by multiplying on the left by the change of basis matrix.
\ However, we change the $V$ basis of a matrix for a transformation
$T:V\rightarrow W$. by multiplying on the right by\textbf{ the inverse} of the
change of basis matrix $\left(  _{S_{2}}\left\vert C\right\vert _{S_{1}%
}\right)  $.

Now changing the bases used to represent a linear transformation
$T:V\rightarrow W$ does not change the transformation itself. Thus Ker$(T)$
and Im$(T)$ are not changed, only the coordinates in the respective bases are
changed. This tells us something about matrices.

First a definition that is useful.

\begin{definition}
The transpose of an $m\times n$ matrix $M$ is the $n\times m$ matrix \ $M^{T}$
where the rows of one are the columns of the other.
\end{definition}

\begin{remark}
For matrices $M$ and $N$, \ $(MN)^{T}=N^{T}M^{T}$.
\end{remark}

\begin{definition}
Let $M\in\,$Mat$_{m\times n}(F)$. We consider the columns of $_{R}\left\vert
T\right\vert _{S}$ as vectors in the space $F^{m}$ $.$ The column rank of $M$
is Dim$_{F}\left(  <\text{columns}>\right)  $.

We consider the rows of $_{R}\left\vert T\right\vert _{S}$ as vectors in the
space $F^{n}$ $.$ The row rank of $M$ is Dim$_{F}\left(  <\text{rows}>\right)
$.
\end{definition}

\begin{theorem}
Let $M\in\,$Mat$_{m\times n}(F)$; $A\in\,$Mat$_{m\times m}(F)$; $B\in
\,$Mat$_{n\times n}(F)$ where $A$ and $B$ have multiplicative matrix inverses.
Let $N=AMB^{-1}.$ \ Then%
\[
\text{ColumnRank}(M)=\text{ComumnRank}(N)\text{.}%
\]


Also%
\[
\text{RowRank}(M)=\text{RowRank}(N)\text{.}%
\]


Finally, if
\[
K=\left\{  (a_{i})\in F^{n}\,|\,M(a_{i})=0\right\}
\]
then $K$ is a subspace of $F^{n}$ and
\[
\text{Dim}_{F}(K)+\text{ColumnRank}(M)=n\text{.}%
\]


\begin{proof}
We think of $M$ as giving a transformation $T:F^{n}\rightarrow F^{m}$ with
respect to some bases. We think of $A$ and $B$ as changes of bases. Then
$N=AMB^{-1}$ is the matrix of the same $T:F^{n}\rightarrow F^{m}$ just with
respect to different bases. Thus
\[
\text{ColumnRank}(M)=\text{Dim}_{F}(\text{Im}(T))=\text{ComumnRank}(N)\text{.}%
\]
Then%
\[
N^{T}=\left(  B^{-1}\right)  ^{T}M^{T}A^{T}\text{.}%
\]
So RowRank$(M)=$ColumnRank$(M^{T})=$ComumnRank$(N^{T})=$RowRank$(M).$

And lastly,%
\begin{align*}
\text{Dim}_{F}(K)+\text{ColumnRank}(M)  &  =\text{Null}(T)+\text{ColumnRank}%
(M)\\
&  =\text{Null}(T)+\text{Dim}_{F}\left(  <\text{Columns of }M>\right) \\
&  =\text{Null}(T)+\text{Rank}(T)\\
&  =n\text{.}%
\end{align*}

\end{proof}
\end{theorem}

\subsection{Elementary Row and Column Operations.}

Keeping the field $F$, define the following $n\times n$ matrices$.$%
\[
_{n}\mathcal{S}_{ij}\mathcal{=}%
\begin{pmatrix}
1 & 0 & \cdots & 0 & \cdots & 0 & \cdots & 0\\
0 & 1 & \cdots & 0 & \cdots & 0 & \cdots & 0\\
0 & 0 & \cdots & 0 & \cdots & 1 & \cdots & 0\\
\vdots & \vdots & \cdots & \vdots & \cdots & \cdots & \vdots & \vdots\\
&  & \cdots & 1 & \cdots & 0 &  & \\
0 & 0 & \cdots &  & \cdots & 0 & \cdots & 1
\end{pmatrix}
\]
That is $\mathcal{S}_{ij}$ is the identity matrix with the $i$-th and $j$-th
rows switched$.$%
\[
_{n}\mathcal{P}_{i}(a)\mathcal{=}%
\begin{pmatrix}
1 & 0 & \cdots & 0 & 0 & 0\\
0 & 1 & \cdots & 0 & 0 & 0\\
\vdots & \vdots & \ddots & 0 & 0 & 0\\
\vdots & \vdots & \cdots & a & \cdots & \vdots\\
\vdots & \vdots & \cdots & \vdots & \ddots & \vdots\\
0 & 0 & \cdots & 0 & \cdots & 1
\end{pmatrix}
\]
That is $\mathcal{P}_{ij}$ is the identity matrix with an $a$ in the $i$-th
row instead of a $1.$%
\[
_{n}\mathcal{E}_{ij}(a)\mathcal{=}%
\begin{pmatrix}
1 & 0 & \cdots & 0 & \cdots & 0 & \cdots & 0\\
0 & 1 & \cdots & 0 & \cdots & 0 & \cdots & 0\\
\vdots & \vdots & \ddots & 0 & \cdots & 0 & \vdots & \vdots\\
0 & 0 & 0 & 1 & \cdots & 0 & \cdots & 0\\
\vdots & \vdots & \vdots & \vdots & \ddots & \vdots & \vdots & \vdots\\
\vdots & \vdots & \cdots & -a & \cdots & 1 & \cdots & 0\\
\vdots & \vdots & \cdots & \vdots & \cdots & \vdots & \ddots & \vdots\\
0 & 0 & \cdots & 0 & \cdots & 0 &  & 1
\end{pmatrix}
\]
That is $\mathcal{E}_{ij}$ is the identity matrix with a $-a$ in the $ij$-th
position where $i\neq j.$

\begin{remark}
Let $M$ be an $m\times n$ matrix$.$Then

\begin{enumerate}
\item $_{m}\mathcal{S}_{ij}$ $M$ is the matrix $M$ with the $i$-th and $j$-th
rows switched.

\item $_{m}\mathcal{P}_{ij}$ $M$ is the matrix $M$ but with the $i$-th row
multiplied by $a$.

\item $_{m}\mathcal{E}_{ij}$ $M$ is the matrix obtained from $M$ by
multiplying the $i$-th row by $a$ and subtracting it from the $j$-th row.
\end{enumerate}
\end{remark}

All these matrices have multiplicative inverses, although $_{n}\mathcal{P}%
_{i}(a)$ requires that $a\neq0$. \ These are called the elementary row
operations on $M$.

\begin{remark}
Let $M$ be an $m\times n$ matrix$.$Then

\begin{enumerate}
\item $M_{n}\mathcal{S}_{ij}$ is the matrix $M$ with the $i$-th and $j$-th
columns switched.

\item $M_{n}\mathcal{P}_{ij}$ is the matrix $M$ with the $i$-th column
multiplied by $a$.

\item $M_{n}\mathcal{E}_{ij}$ is the matrix obtained from $M$ by multiplying
the $i$-th column by $a$ and adding it to the $j$-th column.
\end{enumerate}
\end{remark}

These are called the elementary column operations on $M$.

\begin{remark}
The elementary row operations do not change the row rank of the starting matrix.

The elementary column operations do not change the column rank of the starting matrix.
\end{remark}

\begin{definition}
We say a matrix $N$ is in row echelon form when:

\begin{enumerate}
\item All zero rows are below all rows without a zero entry.

\item The first entry from the left in every non-zero row is a $1$.

\item The first entry from the left in every non-zero row is to the strict
right of any higher leading $1$.
\end{enumerate}
\end{definition}

\begin{example}%
\[%
\begin{pmatrix}
1 & a & b & d & f & i & l & p\\
0 & 1 & c & e & g & j & m & q\\
0 & 0 & 0 & 1 & h & k & n & r\\
0 & 0 & 0 & 0 & 0 & 0 & 0 & 1\\
0 & 0 & 0 & 0 & 0 & 0 & 0 & 0
\end{pmatrix}
\]

\end{example}

\begin{definition}
We say a matrix $N$ is in reduced row echelon form when:

\begin{enumerate}
\item It is in row echelon form

\item In any column with a leading $1$ all other entries are $0$.
\end{enumerate}
\end{definition}

\begin{example}%
\[%
\begin{pmatrix}
1 & a & b & 0 & f & i & l & 0\\
0 & 1 & c & 0 & g & j & m & 0\\
0 & 0 & 0 & 1 & h & k & n & 0\\
0 & 0 & 0 & 0 & 0 & 0 & 0 & 1\\
0 & 0 & 0 & 0 & 0 & 0 & 0 & 0
\end{pmatrix}
\]

\end{example}

\begin{remark}
Any matrix $M$ can be transformed into a matrix $N$ in either row echelon form
or reduced row echelon form using elementary row operations.
\end{remark}

\begin{remark}
For any matrix $M$ there is a matrix $A$ with a multiplicative inverse so that
$N=AM$ is either row echelon form or reduced row echelon form.

\begin{proof}
$A$ is the product of the required elementary row operation matrices.
\end{proof}
\end{remark}

\begin{remark}
The $A$ above can be obtained by augmenting the $m\times n$ matrix $M$ by
placing an $n\times n$ matrix on the left%
\[
M\mapsto\left(  M\,\vdots\,I_{n}\right)
\]
and using elementary row operations to change it to.%
\[
\left(  M\,\vdots\,I_{n}\right)  \approx\left(  N\,\vdots\,A\right)  .
\]
with $N$ in echelon form.

\begin{theorem}
Let $M$ be an $m\times n$ matrix. Then there exist an $m\times m$ matrix $A$
and an $n\times n$ matrix, both with inverses so that%
\[
D=(d_{ij})=AMB^{-1}%
\]
has all zero entries except on the "diagonal" where $d_{ii}=1$ for
$i=1,2\cdots k\leq n$. \ \ Thus%
\[
\text{RowRank}(M)=\text{ColumnRank}(M).
\]


\begin{proof}
After changing $M$ to reduced row echelon form with elementary row operations,
we change the result to "reduced column echelon form$.$" Then%
\begin{align*}
\text{RowRank}(M)  &  =\text{RowRank}(D)\\
&  =\text{Dim}_{F}(<\text{ rows of }D>)\\
&  =\#\text{ }1\text{'s in }D\\
&  =\text{Dim}_{F}(<\text{ columns of }D>)\\
&  =\text{ColumnRank}(D).\\
&  =\text{ColumnRank}(M).
\end{align*}

\end{proof}
\end{theorem}
\end{remark}

\subsection{Determinants}

\subsubsection{Definitions and Properties}

The determinant of an $n\times n$ matrix is an important theoretical tool for
studying linear transformations of a finitely generated vector space into
itself. In practice, for larger $n$, they can be quite a challenge to compute
and even to define. \ Consequently, there are several different ways of
defining the determinant of a matrix each requiring its own increasing level
of sophistication, and none of them are truly useful for computing the
determinant except under the right conditions. It is the properties that
follow from these definitions that make determinants useful.

\begin{definition}
(1) \ Let $n\in%
%TCIMACRO{\U{2115} }%
%BeginExpansion
\mathbb{N}
%EndExpansion
$. The determinant is a map Det: Mat$_{n\times n}(F)\rightarrow F$ so that

\begin{enumerate}
\item Det$(M)$ is multilinear on the rows of $M$. That is, if we hold all but
one of the rows of $M$ fixed, and view the remaining row as a variable vector
$v$ in $F^{n}$, then the determinant as a map on $F^{n}$ is linear in $v$.

\item Det$(M)=0$ if two rows of $M$ are equal.

\item Det$(I_{n})=1.$
\end{enumerate}
\end{definition}

A - slightly - more computable definition uses permutations of $\{1,2,3\cdots
n\}$ There are, of course, $n!$ of these$.$As it turns out, every such
permutation can be accomplished by a sequence of simply switching one pair of
these numbers. This divides the collection of all permutations $\mathcal{S}%
_{n}$ into those that require an even number of transpositions and those that
require an odd number. We define sgn$:\mathcal{S}\rightarrow\{\pm1\}$ by
sending the even permutations to $1$ and the odd to $-1$.

\begin{definition}
(2) \ Let $n\in%
%TCIMACRO{\U{2115} }%
%BeginExpansion
\mathbb{N}
%EndExpansion
$. The determinant is a map Det: Mat$_{n\times n}(F)\rightarrow F$ given by%
\[
\text{Det}(M)=\text{Det}((a_{i,j}))=%
%TCIMACRO{\tsum \limits_{\sigma\in\mathcal{S}_{n}}}%
%BeginExpansion
{\textstyle\sum\limits_{\sigma\in\mathcal{S}_{n}}}
%EndExpansion
\left(  \text{sgn}(\sigma)a_{1,\sigma(1)}a_{2,\sigma(2)}a_{3,\sigma(3)}\cdots
a_{n,\sigma(n)}\right)  \text{.}%
\]

\end{definition}

It is the properties of the determinant that make it useful.

\begin{theorem}
Let $N$, $M\in\,$Mat$_{n\times n}(F).$

\begin{enumerate}
\item Det$(NM)=\,$Det$(N)$ Det$(M)$

\item Det$(M^{T})=$\thinspace Det$(M)$

\item $M$ has a multiplicative inverse if and only if Det$(M)\neq0$

\item If $M$ has a multiplicative inverse, Det$(M^{-1})=$\thinspace$\left(
\text{Det}(M)\right)  ^{-1}$.
\end{enumerate}
\end{theorem}

Because of these properties, if $A\in$Mat$_{n\times n}(F)$ has an inverse,
then
\[
\text{Det}(AMA^{-1})=\,\text{Det}(M)\text{.}%
\]
So the determinant is independent of a change of basis. The following
definition is possible.

\begin{definition}
Let $T:V\rightarrow V$ be a linear transformation on a finitely generated
vector space. Then%
\[
\text{Det}(T)=\text{Det}(_{S}|T|_{S})
\]
for any basis $S$ of $V$.
\end{definition}

\begin{theorem}
Laplace's Formula. \ Let $M=(a_{i,j})\in\,$Mat$_{n\times n}(F)$. For any pair
$(i,j)$ (of usable size), the $(i,j)$-minor of $M$ is the matrix $M_{(i,j)}$
obtained by eliminating the $i$-th row and the $j$-th column. Consider
Det$(M_{(i,j)})$. (Some call this determinant the minor.)

\begin{enumerate}
\item For a fixed row, $i_{0}$,
\[
\text{Det}(M)=%
%TCIMACRO{\tsum \limits_{j=1}^{n}}%
%BeginExpansion
{\textstyle\sum\limits_{j=1}^{n}}
%EndExpansion
(-1)^{i+j}a_{i_{0},j}\text{Det}(M_{(i_{0},j)})
\]


\item For a fixed column, $j_{0}$,
\[
\text{Det}(M)=%
%TCIMACRO{\tsum \limits_{j=1}^{n}}%
%BeginExpansion
{\textstyle\sum\limits_{j=1}^{n}}
%EndExpansion
(-1)^{i+j}a_{i,j_{0}}\text{Det}(M_{(i,j_{0})})
\]

\end{enumerate}
\end{theorem}

Laplace's formula works best when a row of column containing mostly $0$'s.
\ Referring back to the elementary matrices, we see%
\begin{align*}
\text{Det}(_{n}\mathcal{S}_{i,j})  &  =1\\
\text{Det}(_{n}\mathcal{P}_{i,}(a))  &  =a\\
\text{Det}(_{n}\mathcal{E}_{i,j}(a))  &  =1
\end{align*}
So multiplying a row by a scalar multiplies the determinant by the same
scalar. The other elementary row operations do not change it. That means we
can use row (and column) operations to make Laplace's formula more useful. But
we need to account for any row (or column) that we multiply by $a\neq0$.

\subsection{Eigenvalues and Eigenvectors.}

All that we saw about change of bases in transformations between different
vector spaces applies just as well to $T\in\,$Hom$_{F}(V,V)$ or if $M\in
\,$Mat$_{n\times n}(F)$. However, it would seem preferable to use the same
basis on both sides of such a transformation. \ Once this is done, any change
of basis should occur on both sides to be consistent. Thus we want%
\[
\left(  _{S_{2}}\left\vert T\right\vert _{S_{2}}\right)  =\left(  _{S_{2}%
}\left\vert A\right\vert _{S_{1}}\right)  \left(  _{S_{1}}\left\vert
T\right\vert _{S_{1}}\right)  \left(  _{S_{2}}\left\vert A\right\vert _{S_{1}%
}\right)  ^{-1}.
\]
From a matrix point of view, for $M\in\,$Mat$_{n\times n}(F)$ we want $A\in
\,$Mat$_{n\times n}(F)$ with a multiplicative inverse so that
\[
N=AMA^{-1}\text{.}%
\]
One question that comes up is, how close to "diagonal" can we make $N$?

Since the right and left matrices must be the same, the diagonal $N$ cannot be
expected to be all $1$'s and $0$'s unless $M$ is pretty special. \ \ But
suppose that we can find a matrix $M$ and change it to $N=AMA^{-1}$ where the
first column has $e$ at the top and all other entries in the first row and
column are $0$ (sort of the first step in finding a diagonal $N$) .Then%
\[
NA=MA.
\]
The first column of $NA$ would be the first column of $A$ multiplied by $e$.
The first column of $MA$ would be the transformation $M:F^{n}\rightarrow
F^{n}$ acting on the first column of $A$ as a vector in $F^{n}.$ These would
be the same. \ That is to say, if $v$ is the first column of $A$, then
$Mv=ev$. This makes $e$ an eigenvalue of $M$ as a transformation and $v$ an
$e$-eigenvector$.$

To see how nice a matrix $N$ we can obtain from $M$ by $N=AMA^{-1}$. We must
start by finding the eigenvalues of $M$ as a transformation $M:F^{n}%
\rightarrow F^{n}$

Recall, if $\lambda$ is an eigenvalue of $T$ then the eigenspace of $\lambda$
is the subspace given by%
\begin{align*}
V_{\lambda}  &  =\{v\in V\;|\;M(v)=\lambda\cdot v\}\\
&  =\{v\in V\;|\;\left(  M-\lambda I\right)  (v)=0\}
\end{align*}
and no eigenspace is simply $\{0\}$. That would mean that there is
$v_{\lambda}\neq0$ with $\left(  M-\lambda I\right)  (v_{\lambda})=0=\left(
M-\lambda I\right)  (0)$. So $\left(  M-\lambda I\right)  $ is not a bijective
transformation; it cannot have an inverse matrix. This is
\[
\text{Det}(\left(  M-\lambda I\right)  )=0\text{.}%
\]
This leads us to:

\begin{definition}
Let $M$ be an $n\times n$ matrix over $F$. The characteristic polynomial of
$M$ is%
\[
\varkappa_{M}(x)=\text{Det}(\left(  M-xI\right)  ).
\]

\end{definition}

\begin{remark}
Let $M$ be an $n\times n$ matrix over $F$. Then

\begin{enumerate}
\item $\varkappa_{M}(0)=$Det$(M)$

\item $\varkappa_{M}(0)$ had degree $n$ and the leading coefficient is
$(-1)^{n}$.
\end{enumerate}
\end{remark}

\begin{remark}
If $M$ is an $n\times n$ matrix over $F$ , then $\lambda\in F$ is an
eigenvalue of $M$ if and only if $\lambda$ a root of $\varkappa_{M}(x).$ Any
solution $v$ to
\[
\left(  M-\lambda I\right)  \left(  v\right)  =0
\]
is an element of the eigenspace of $\lambda$.
\end{remark}

That is to say, if $\lambda$ is an eigenvalue of $M$, then the eigenspace of
$\lambda$ is
\[
V_{\lambda}=\text{Ker}(\left(  M-\lambda I\right)  )\text{.}%
\]


\bigskip

As it turns out every polynomial with coefficients in $F$ is the
characteristic polynomial of some matrix over $n$.

\begin{theorem}
Let $F$ be a field$.$Let
\[
p(x)=x^{n}+a_{n-1}x^{n-1}+a_{n-2}x^{n-2}+\cdots a_{1}x+a_{0}\text{.}%
\]
The companion matrix of $p(x)$ is the $n\times n$ matrix
\[
M=%
\begin{pmatrix}
0 & 0 & 0 & 0 & -a_{0}\\
1 & 0 & 0 & 0 & -a_{1}\\
0 & 1 & 0 & \vdots & \vdots\\
0 & 0 & \ddots & 0 & -a_{n-1}\\
0 & 0 & 0 & 1 & -a_{n-1}%
\end{pmatrix}
\]
Then%
\[
\varkappa_{M}(x)=p(x).
\]

\end{theorem}

So the moral of the story is that a matrix may not have eigenvalues in the
field of its coefficients.

\subsection{Diagonalizable Matrices}

At this point, we must pay attention to our field $F$. \ If $F=%
%TCIMACRO{\U{2102} }%
%BeginExpansion
\mathbb{C}
%EndExpansion
$, the complex numbers, the fundamental theorem of algebra says every
polynomial over $F$ with degree $n$ has $n$ roots counting multiplicities. If
$F=%
%TCIMACRO{\U{211d} }%
%BeginExpansion
\mathbb{R}
%EndExpansion
$, there are quadratic polynomials with real coefficients that do not have
real roots like $x^{2}+1$ and $x^{2}+x+1$. If $F=%
%TCIMACRO{\U{211a} }%
%BeginExpansion
\mathbb{Q}
%EndExpansion
$ or $F=%
%TCIMACRO{\U{2124} }%
%BeginExpansion
\mathbb{Z}
%EndExpansion
_{p}$ there a polynomials of every degree that do not have roots in $F$.

The object still remains to start with a matrix $M$ over a field $F$ and find
the nicest possible a matrix $N=AMA^{-1}$ can have. As we have just noticed,
$N$ is not going to be as dependably simple as a diagonal matrix. The first
step in finding eigenvalues of a matrix $M$ over $F$ is factoring
$\varkappa_{M}(x)$ into polynomials irreducible over $F$. \ There are
algebraic calculators that will actually do this for $F=%
%TCIMACRO{\U{211a} }%
%BeginExpansion
\mathbb{Q}
%EndExpansion
$ and $F=%
%TCIMACRO{\U{2124} }%
%BeginExpansion
\mathbb{Z}
%EndExpansion
_{p}$. They will also do it for $F=%
%TCIMACRO{\U{2102} }%
%BeginExpansion
\mathbb{C}
%EndExpansion
$ and $F=%
%TCIMACRO{\U{211d} }%
%BeginExpansion
\mathbb{R}
%EndExpansion
$, although perhaps only approximately.

Since the first step in reducing a matrix $M$ is to find all the roots of the
characteristic polynomial $\varkappa_{M}(x)$. Even if the degree of this
polynomial is as small as $4$, finding exact expressions for the roots is a
complicated process. For degree $5$ or higher, finding exact roots might
involve such complicated complex functions that one might consider it
impossible. The only recourse for $F=%
%TCIMACRO{\U{211d} }%
%BeginExpansion
\mathbb{R}
%EndExpansion
$ or $%
%TCIMACRO{\U{2102} }%
%BeginExpansion
\mathbb{C}
%EndExpansion
$ is to approximate the roots using decimals. The most likely outcome of any
specific approximate calculation starting with a matrix $M$ will result in it
appearing to be diagonalizable. Speaking roughly, without regard to complete
truth, any non-diagonalizable matrix is totally surrounded by diagonalizable
ones. So we move away from exact results by approximating the eigenvalues.
Approximating the eigenvectors of approximate eigenvalues means moving even
further away from the exact results. It may even be necessary to increase the
precision of the approximations substantially simply to avoid dividing by zero
while finding the approximate eigenvectors. In the course of this, the result
will show the results for a diagonalizable matrix that is "close" to the
original $M$. As a result, the best you will find is an approximate matrix $A$
with an inverse matrix and a diagonal matrix $D$ with approximations of the
eigenvalues on the diagonal so that%
\[
D\approx AMA^{-1}.
\]
Thus the $D$ you find is a diagonal approximation of what might not be a
diagonal matrix.

\subsubsection{Guaranteed Diagonalizable Matrices}

When can we expect an exact diagonal result, at least theoretically? \ Our
first step is to see how a matrix can be reduced to a diagonal form in the
right circumstances.

\begin{theorem}
Let $M$ be an $n\times n$ matrix over the field $F$. If its characteristic
polynomial $\varkappa_{M}(x)$ factors completely into distinct linear factors
over $F$, \ then there exists an $n\times n$ matrix $E$ over the field $F$
with an inverse so that $D=E^{-1}ME$ is made of all zeros except on the diagonal.

\begin{proof}
Assume $M$ is an $n\times n$ matrix over the field $F$. Assume its
characteristic polynomial $\varkappa_{M}(x)$ is written%
\[
\varkappa_{M}(x)=(-1)^{n}(x-\lambda_{1})(x-\lambda_{2})\cdots(x-\lambda_{n})
\]
where $\lambda_{i}=\lambda_{j}$ implies $i=j$. \ Thus each $\lambda_{i}$ is an
eigenvalue for $M$, and so there is a $v_{\lambda_{i}}\neq0\in F^{n}$ so that
\[
Mv_{\lambda_{i}}=\lambda_{i}v_{\lambda_{i}}.
\]


\begin{claim}
The set $S=\left\{  v_{\lambda_{1}},v_{\lambda_{2}},\cdots v_{\lambda_{n}%
}\right\}  $ is a basis of $F^{n}$.

\begin{proof}
Since all $v_{\lambda_{i}}\neq0,$ any one element subset of $S$ is linearly
independent. So there is a subset $T$ of $S$ so that $T$ is a maximal linearly
independent subset of $S$. If $S=T$, $\ $then $S$ is a linearly independent
set whose size is the dimension of the vector space. Thus $S$ is a basis.
Suppose BWOC $T\subseteq S$, but $T\neq T$. Then there is a
\[
v_{\lambda_{k+1}}\in S\backslash T\text{.}%
\]
Now $T\cup\{v_{\lambda_{k+1}}\}$ is not linearly independent, so I can write%
\[
v_{\lambda_{k+1}}=a_{1}v_{\lambda_{1}}+a_{2}v_{\lambda_{2}}\cdots
+a_{k}v_{\lambda_{k}}%
\]
with $v_{\lambda_{i}}\in T$ and $a_{i}\neq0$ for $i=1,2\cdots k$.

Then%
\begin{align*}
Mv_{\lambda_{k+1}}  &  =a_{1}Mv_{\lambda_{1}}+a_{2}Mv_{\lambda_{2}}%
\cdots+a_{k}Mv_{\lambda_{k}}\\
\lambda_{k+1}v_{\lambda_{k+1}}  &  =a_{1}\lambda_{1}v_{\lambda_{1}}%
+a_{2}\lambda_{2}v_{\lambda_{2}}\cdots+a_{k}\lambda_{k}v_{\lambda_{k}}%
\end{align*}
But we also have
\[
\lambda_{k+1}v_{\lambda_{k+1}}=a_{1}\lambda_{k+1}v_{\lambda_{1}}+a_{2}%
\lambda_{k+1}v_{\lambda_{2}}\cdots+\lambda_{k+1}a_{k}v_{\lambda_{k}}%
\]
So
\begin{align*}
&  a_{1}\lambda_{1}v_{\lambda_{1}}+a_{2}\lambda_{2}v_{\lambda_{2}}\cdots
+a_{k}\lambda_{k}v_{\lambda_{k}}\\
&  =a_{1}\lambda_{k+1}v_{\lambda_{1}}+a_{2}\lambda_{k+1}v_{\lambda_{2}}%
\cdots+\lambda_{k+1}a_{k}v_{\lambda_{k}}%
\end{align*}
So
\[
a_{1}(\lambda_{1}-\lambda_{k+1})v_{\lambda_{1}}+a_{2}(\lambda_{2}%
-\lambda_{k+1})v_{\lambda_{2}}\cdots+a_{k}(\lambda_{k}-\lambda_{k+1}%
)v_{\lambda_{k}}=0
\]
Since these vectors in this sum come from $T$, they are linearly independent.
Thus%
\[
a_{i}(\lambda_{i}-\lambda_{k+1})=0\text{ for }i=1,2\cdots k.\text{ }%
\]
But $a_{i}\neq0$ so $\lambda_{i}=\lambda_{k+1}$, and this contradicts our
assumption that the $\lambda$ are distinct.
\end{proof}
\end{claim}

Now consider the matrix $E$ whose columns are the vectors in the basis $S$:%
\[
E=\left(  v_{\lambda_{1}}\vdots\,v_{\lambda_{2}}\vdots\,v_{\lambda n}%
\cdots\vdots\,v_{\lambda_{n}}\right)  .
\]
Since its column rank is $n$, $E$ has an inverse. Let $D$ be the diagonal
matrix%
\[
D=%
\begin{pmatrix}
\lambda_{1} & 0 & 0 & 0 & 0\\
0 & \lambda_{2} & 0 & 0 & 0\\
0 & 0 & \lambda_{3} & 0 & 0\\
0 & 0 & 0 & \ddots & -0\\
0 & 0 & 0 & 0 & \lambda_{n}%
\end{pmatrix}
\]
Then one sees that%
\[
ED=ME
\]
So indeed%
\[
D=E^{-1}ME.
\]

\end{proof}
\end{theorem}

In general, this is the way a matrix can be diagonalized. If $\Lambda
=\{\lambda_{1},\lambda_{2}\cdots\lambda_{k}\}$ are the distinct eigenvalues of
a matrix $M$, it may be that any one $\lambda_{i}$ has several linearly
independent eigenvectors. If $S=\{v_{1},v_{2}\cdots v_{n}\}$ is a linearly
independent set where each $v_{i}$ is an eigenvector for some eigenvalue, then
the matrix whose columns are the elements in $S$ diagonalizes $M$. In this
case, the characteristic polynomial of $M$ factors as%
\[
\varkappa_{M}(x)=(-1)^{n}(x-\lambda_{1})^{e_{i}}(x-\lambda_{2})^{e_{2}}%
\cdots(x-\lambda_{k})^{e_{k}}%
\]
and%
\[
e_{i}=\text{Dim}_{F}\left(  \left\{  v\in F^{n}\;|\;Mv=\lambda_{i}v\right\}
\right)
\]
and $\lambda_{i}$ appears $e_{i}$ times on the diagonal of $D$.

\subsubsection{The Cayley-Hamilton Theorem}

This is enough to prove a famous theorem about square matrices, at least for
the diagonalizable ones. We will state it in full generality, but forgo the proof.

\begin{theorem}
The Cayley-Hamilton Theorem. \ If $T:V\rightarrow V$ is a linear
transformation on a finitely generated $F$ vector space, then%
\[
\varkappa_{T}(T)=0\text{.}%
\]

\end{theorem}

It may not be that this is the polynomial of smallest degree that $T$
satisfies. So we have:

\begin{definition}
Let $T:V\rightarrow V$ be a linear transformation on a finitely generated $F$
vector space, then the minimum polynomial of $T$
\[
\text{min}_{T}(x)
\]
is the monic polynomial (over $F$) of minimum degree so that%
\[
\text{min}_{T}(T)=0
\]

\end{definition}

If $M$ is a diagonalizable matrix with
\[
\varkappa_{M}(x)=(-1)^{n}(x-\lambda_{1})^{e_{i}}(x-\lambda_{2})^{e_{2}}%
\cdots(x-\lambda_{k})^{e_{k}}%
\]
Then%
\[
\text{Min}_{M}(x)=(x-\lambda_{1})(x-\lambda_{2})\cdots(x-\lambda_{k})
\]
This is only true because $M$ is diagonalizable. However, if one knows that
the minimum polynomial factors into distinct linear factors, the
transformation is diagonalizable.\ 

\begin{remark}
In general Min$_{T}(x)$ divides $\varkappa_{T}(x)$ and any factor of
$\varkappa_{M}(x)$ could be the minimum polynomial. \ 


\end{remark}

\subsection{Canonical Forms}

There are definitely square matrices that cannot be diagonalized this way. The
best one can hope for is that $D=AMA^{-1}$ be "block diagonal" with elementary
blocks. Block diagonal means that the $n\times n$ is made up of various sized
square $r_{i}\times r_{i}$ matrices arranged down the "diagonal" with all
other entries $0$. Thus a block diagonal matrix has the form%
\[
D=%
\begin{pmatrix}
B_{r_{_{1}}} & 0 & 0 & 0 & 0\\
0 & B_{r_{2}} & 0 & 0 & 0\\
0 & 0 & B_{r_{_{3}}} & 0 & 0\\
0 & 0 & 0 & \ddots & -0\\
0 & 0 & 0 & 0 & B_{r_{k}}%
\end{pmatrix}
\]
where
\[
r_{1}+r_{2}+\cdots r_{k}=n\text{.}%
\]
Without saying exactly how to find them, we will list the possible elementary
blocks that we can place in the best block diagonal form of a matrix $M$.

The first elementary block is technically the one by one matrix given by a
eigenvalue. However, several of these can be place together to form an
$e\times e$ matrix.
\[
E_{e}(\lambda)=%
\begin{pmatrix}
\lambda & 0 & 0 & 0 & 0\\
0 & \lambda & 0 & 0 & 0\\
0 & 0 & \lambda & 0 & 0\\
0 & 0 & 0 & \ddots & -0\\
0 & 0 & 0 & 0 & \lambda
\end{pmatrix}
_{e\times e}%
\]


The characteristic polynomial of this block is%
\[
\varkappa_{E_{r}(\lambda)}(x)=(x-\lambda)^{e}%
\]


Its minimum polynomial is
\[
\text{Min}_{E_{r}(\lambda)}(x)=(x-\lambda)
\]
And if this block appears in a matrix obtained from $M$,%
\[
e=\text{Dim}_{F}\left(  \left\{  v\in F^{n}\;|\;Mv=\lambda v\right\}  \right)
\]


Notice that ,if $M$ is diagonalizable with%
\[
\varkappa_{M}(x)=(-1)^{n}(x-\lambda_{1})^{e_{i}}(x-\lambda_{2})^{e_{2}}%
\cdots(x-\lambda_{k})^{e_{k}}%
\]
then there is an $E$ so that%
\[
E^{-1}ME=%
\begin{pmatrix}
E_{e_{1}}(\lambda_{1}) & 0 & 0 & 0 & 0\\
0 & E_{r_{2}}(\lambda_{2}) & 0 & 0 & 0\\
0 & 0 & E_{r_{3}}(\lambda_{3}) & 0 & 0\\
0 & 0 & 0 & \ddots & -0\\
0 & 0 & 0 & 0 & E_{r_{k}}(\lambda_{k})
\end{pmatrix}
\]


The next elementary block given by a eigenvalue and its generalized
eigenvalues. We define an $r\times r$ matrix
\[
GE_{r}(\lambda)=%
\begin{pmatrix}
\lambda & 1 & 0 & 0 & 0\\
0 & \lambda & 1 & 0 & 0\\
0 & 0 & \lambda & 1 & 0\\
0 & 0 & 0 & \ddots & 1\\
0 & 0 & 0 & 0 & \lambda
\end{pmatrix}
_{r\times r}%
\]
The characteristic polynomial of this block is%
\[
\varkappa_{GE_{r}(\lambda)}(x)=(x-\lambda)^{r}%
\]


Its minimum polynomial is
\[
\text{Min}_{GE_{r}(\lambda)}(x)=(x-\lambda)^{r}%
\]
And if this block appears in a matrix obtained from $M$, we consider the
subspace of generalized eigenvectors%
\[
U_{\lambda}=\left\{  v\in F^{n}\;|\;\exists i\in%
%TCIMACRO{\U{2115} }%
%BeginExpansion
\mathbb{N}
%EndExpansion
\text{ such that }(M-\lambda)^{i}(v)=0\right\}  .
\]
This subspace will have a basis
\[
S_{\lambda}=\{w_{1},w_{2},\cdots w_{r}\}
\]
where%
\begin{align*}
Mw_{1}  &  =\lambda w_{1}\\
Mw_{2}  &  =\lambda w_{2}+w_{1}\\
Mw_{3}  &  =\lambda w_{3}+w_{2}\\
&  \vdots\\
Mw_{r}  &  =\lambda w_{r}+w_{r-1}%
\end{align*}
These vectors can be used as columns in the basis transformation that leads to
the block diagonal form.

We now state a theorem that takes advantage of the Fundamental Theorem of
Algebra, and set our field to the complex numbers. That way any characteristic
polynomial will factor into linear factors.

\begin{theorem}
Jordan Canonical Form$.$ Let $T:V\rightarrow V$ be a linear transformation on
a finitely generated vector space over $%
%TCIMACRO{\U{2102} }%
%BeginExpansion
\mathbb{C}
%EndExpansion
$. Then there is a basis $S$ of $V$ so that the matrix $_{S}|T|_{S}$ is in
Jordan Canonical Form. That is,%
\[
_{S}|T|_{S}=%
\begin{pmatrix}
B_{r_{_{1}}} & 0 & 0 & 0 & 0\\
0 & B_{r_{2}} & 0 & 0 & 0\\
0 & 0 & B_{r_{_{3}}} & 0 & 0\\
0 & 0 & 0 & \ddots & 0\\
0 & 0 & 0 & 0 & B_{r_{k}}%
\end{pmatrix}
\]
where $B_{r_{i}}$ is either $E_{e}(\lambda)$ or $GE_{r}(\lambda)$ and the
$\lambda$ that appear may or may not be distinct.

The characteristic polynomial of $T$, $\varkappa_{T}(x)$ is the product of the
characteristic polynomial of the blocks. The minimum polynomial of T,
min$_{T}(x)$ is the least common multiple of the minimum polynomials of the blocks.
\end{theorem}

Next we move to other fields where there are irreducible polynomials that are
not linear. We do, however, add a technical condition that the field $F$ be
perfect. (That is, we require that every irreducible polynomial with
coefficients in $F$ factors into distinct factors in any larger field where it
has all its factors. All the fields we have been thinking about - $%
%TCIMACRO{\U{211a} }%
%BeginExpansion
\mathbb{Q}
%EndExpansion
,$ $%
%TCIMACRO{\U{211d} }%
%BeginExpansion
\mathbb{R}
%EndExpansion
$, $%
%TCIMACRO{\U{2102} }%
%BeginExpansion
\mathbb{C}
%EndExpansion
$, and $%
%TCIMACRO{\U{2124} }%
%BeginExpansion
\mathbb{Z}
%EndExpansion
_{p}$ - are perfect.)

If
\[
p(x)=x^{r}+a_{r-1}x^{r-1}+a_{r-2}x^{r-2}+\cdots a_{1}x+a_{0}\text{.}%
\]
is an irreducible polynomial with coefficients in $F$, we create an elementary
block from its companion matrix%
\[
C(p(x))=%
\begin{pmatrix}
0 & 0 & 0 & 0 & -a_{0}\\
1 & 0 & 0 & 0 & -a_{1}\\
0 & 1 & 0 & \vdots & \vdots\\
0 & 0 & \ddots & 0 & -a_{n-1}\\
0 & 0 & 0 & 1 & -a_{n-1}%
\end{pmatrix}
_{r\times r}%
\]
The characteristic polynomial of this block is%
\[
\varkappa_{C(p(x)}(x)=p(x)
\]


Its minimum polynomial is
\[
\text{Min}_{C(p(x)}(x)=p(x)
\]
And if this block appears in a matrix obtained from $M$,%
\[
r=\text{Dim}_{F}\left(  \left\{  v\in F^{n}\;|\;P(M)(v)=0\right\}  \right)
\]


There is one more elementary block type that is similar to the generalized
eigenvectors we used before.

If
\[
p(x)=x^{r}+a_{r-1}x^{r-1}+a_{r-2}x^{r-2}+\cdots a_{1}x+a_{0}\text{.}%
\]
is an irreducible polynomial with coefficients in $F$, we create an elementary
block using $s$ copies of its companion matrix%
\[
GC_{s}(p(x))=%
\begin{pmatrix}
C(p(x) & I_{r_{1}} & 0 & 0 & 0\\
0 & C(p(x) & I_{r_{2}} & 0 & 0\\
0 & 0 & C(p(x) & I_{r_{3}} & 0\\
0 & 0 & 0 & \ddots & I_{r_{s}}\\
0 & 0 & 0 & 0 & C(p(x)
\end{pmatrix}
_{rs\times rs}%
\]
The characteristic polynomial of this block is%
\[
\varkappa_{GC(p(x)}(x)=\left(  p(x)\right)  ^{s}%
\]


Its minimum polynomial is
\[
\text{Min}_{GC(p(x)}(x)=p(x)
\]
And if this block appears in a matrix obtained from $M$,%
\[
r=\text{Dim}_{F}\left(  \left\{  v\in F^{n}\;|\;\exists k\in%
%TCIMACRO{\U{2115} }%
%BeginExpansion
\mathbb{N}
%EndExpansion
\text{ so that }P(M)^{k}(v)=0\right\}  \right)  \text{.}%
\]


\begin{theorem}
$.$ Let $T:V\rightarrow V$ be a linear transformation on a finitely generated
vector space over a perfect field $F$. Then there is a basis $S$ of $V$ so
that the matrix $_{S}|T|_{S}$ is in Canonical Form. That is,%
\[
_{S}|T|_{S}=%
\begin{pmatrix}
B_{r_{_{1}}} & 0 & 0 & 0 & 0\\
0 & B_{r_{2}} & 0 & 0 & 0\\
0 & 0 & B_{r_{_{3}}} & 0 & 0\\
0 & 0 & 0 & \ddots & 0\\
0 & 0 & 0 & 0 & B_{r_{k}}%
\end{pmatrix}
\]
where $B_{r_{i}}$ is either $E_{e}(\lambda)$ ; $GE_{r}(\lambda);$ $C(p(x))$ or
G$C(p(x))$ and parameters that appear may or may not be distinct.

The characteristic polynomial of $T$, $\varkappa_{T}(x)$ is the product of the
characteristic polynomial of the blocks. The minimum polynomial of T,
min$_{T}(x)$ is the least common multiple of the minimum polynomials of the blocks.
\end{theorem}

\section{Lecture 3}

\subsection{Bilinear Forms}

\subsubsection{Preliminaries}

In multivariable calculus we are told that vectors in $%
%TCIMACRO{\U{211d} }%
%BeginExpansion
\mathbb{R}
%EndExpansion
^{n}$ have magnitude and direction. In retrospect, we might realize that these
concepts are based on the standard basis; $S=\{(1,0,0\cdots0),(0,1,0\cdots
0),(0,0,1\cdots0)\cdots(0,0,0\cdots1)\}$. \ And defined using the dot product
of vectors. We next try to reintroduce generalized versions of these concepts
in vector spaces over general fields. Basically our goal is to allow from
different operations like the dot product. For each of these operations, we
find bases for which the new operation works well. And in the end, we find
something almost exactly like the dot product.

We have been trying to allow for any type of field $F$, but we gave up a bit
to get to Jordan Canonical form. We want to keep that generality going in the
next discussion, but we do need to consider the characteristic of the field we
are using in our next subject. \ The characteristic of the field $F$ is the
smallest $n$ where%
\[%
%TCIMACRO{\tsum \limits_{k=1}^{n}}%
%BeginExpansion
{\textstyle\sum\limits_{k=1}^{n}}
%EndExpansion
1=0
\]
If that sum is never $0$ - like in $%
%TCIMACRO{\U{211a} }%
%BeginExpansion
\mathbb{Q}
%EndExpansion
,%
%TCIMACRO{\U{211d} }%
%BeginExpansion
\mathbb{R}
%EndExpansion
.$and $%
%TCIMACRO{\U{2102} }%
%BeginExpansion
\mathbb{C}
%EndExpansion
$- we say the field has characteristic $0$. Except in that case, the
characteristic is always a prime natural number. The characteristic of $%
%TCIMACRO{\U{2124} }%
%BeginExpansion
\mathbb{Z}
%EndExpansion
_{p}$ is automatically $p$ $\ $Every field of characteristic $0$ is perfect.
In time, though, we will get back to cases $F=%
%TCIMACRO{\U{211d} }%
%BeginExpansion
\mathbb{R}
%EndExpansion
$ and $F=%
%TCIMACRO{\U{2102} }%
%BeginExpansion
\mathbb{C}
%EndExpansion
$, where vectors can be considered to have magnitude and direction once again.

In what follows, fields of characteristic $2$ are weird; so we will leave that
case out of our discussion without always pointing that out. Remember only
some of what follows is true for fields of characteristic $2$.

\subsubsection{Definitions}

\begin{definition}
Let $V$ be a vector space over a field $F$. A bilinear form of $V$ is a map
$B:V\times V\rightarrow F$ where

\begin{enumerate}
\item $B(av_{1}+bv_{2},w)=aB(v_{1},w)+bB(v_{2},w)$

\item $B(v,aw_{1}+bw_{2})=aB(v,w_{1})+bB(v,w_{2})$.
\end{enumerate}
\end{definition}

\begin{definition}
Let $V$ be a vector space over a field $F$. Then%
\[
\text{Bilin}_{F}(V)=\left\{  B:V\times V\rightarrow F\;|\;B\text{ is a
bilinear form}\right\}  \text{.}%
\]

\end{definition}

\begin{remark}
If $V$ is a vector space over a field $F$, then Bilin$_{F}(V)$ is also a
vector space over $F$ under%
\begin{align*}
\left(  B_{1}+B_{2}\right)  (v,w)  &  =B_{1}(v,w)+B_{2}(v,w)\\
\left(  aB\right)  (v,w)  &  =aB(v,w).
\end{align*}

\end{remark}

\begin{remark}
If $V$ is a vector space over a field $F$, then there are two canonical linear
transformations of Bilin$_{F}(V)$ into Hom$_{F}(V,F)$
\begin{align*}
L_{B}  &  :\text{Bilin}_{F}(V)\rightarrow\text{Hom}_{F}(V,F)\text{ where
}L_{B}(v)\left(  w\right)  =B(v,w)\\
R_{B}  &  :\text{Bilin}_{F}(V)\rightarrow\text{Hom}_{F}(V,F)\text{ where
}R_{B}(w)\left(  v\right)  =B(v,w)
\end{align*}

\end{remark}

\bigskip

\begin{definition}
Let $B\ $be a bilinear form on the vector space $V$.

\begin{enumerate}
\item We say $B$ is left-nondegenerate when $L_{B}:$Bilin$_{F}(V)\rightarrow
$Hom$_{F}(V,F)$ is an injection.

\item We say $B$ is right-nondegenerate when $R_{B}:$Bilin$_{F}(V)\rightarrow
$Hom$_{F}(V,F)$ is an injection.

\item We say $B$ is nondegenerate when it is both right and left nondegenerate.
\end{enumerate}
\end{definition}

\begin{remark}
For a bilinear $B$, $L_{B}:$Bilin$_{F}(V)\rightarrow$Hom$_{F}(V,F)$ is an
injection means that Ker$(L_{B})=\{0\}$. So being left-nondegenerate means
that, if $v\neq0$, then $B(v,w)$ is not $0$ for all $w\in V$. \ So
\[
\forall v\in V\text{ with }v\neq0\text{, }\exists w\in V\text{ so that
}B(v,w)\neq0\text{.}.
\]
Also being right-nondegenerate means that,
\[
\forall w\in V\text{ with }w\neq0\text{, }\exists v\in V\text{ so that
}B(v,w)\neq0\text{.}.
\]

\end{remark}

\begin{definition}
Let $B\ $be a bilinear form on the vector space $V$.

\begin{enumerate}
\item We say $B$ is symmetric when $\forall v,w\in V$, $B(v,w)=B(w,v)$.

\item We say $B$ is skew symmetric when $\forall v,w\in V$, $B(v,w)=-B(w,v)$.

\item We say $B$ is alternating when $\forall v\in V$, $B(v,v)=0$.

\item We say $B$ is reflexive when $B(v,w)=0$ if and only if $B(w,v)=0.$
\end{enumerate}
\end{definition}

Obviously, some of these are related.

\begin{theorem}
Let $B\ $be a bilinear form on the vector space $V$. \ \ Then there exists
unique bilinear forms $B^{+}$ and $B^{-}$ on $V$ so that

\begin{enumerate}
\item $B^{+}$ is symmetric

\item $B^{-}$ is skew symmetric (equivalently alternating)

\item $B=B^{+}+B^{-}$.
\end{enumerate}
\end{theorem}

Just remember our assumption about the characteristic of the field.

\begin{theorem}
Let $B\ $be a bilinear form on the vector space $V$. \ Then $B$ is reflexive
if and only if it is either alternating or symmetric.
\end{theorem}

\bigskip

\begin{definition}
Let $B$ be a fixed reflexive bilinear form on $V$.

\begin{enumerate}
\item We say $v$ and $w$ $\in V$ are orthogonal when $B(v,w)=0$
\end{enumerate}
\end{definition}

\begin{enumerate}
\item We say $v$ $\in V$ is isotropic when $B(v,v)=0$

\item We say $v$ $\in V$ is anisotropic when $B(v,v)\neq0$

\item If $W\subseteq V$ is a subspace, then the orthogonal complement of $W$
is%
\[
W^{\bot}=\left\{  v\in V\;|\;\forall w\in W\text{, }B(v,w)=0\right\}
\]


\item If $W\subseteq V$ is a subspace, the radical of $W$ is
\[
\text{Rad}_{B}(W)=W\cap W^{\bot}\text{.}%
\]

\end{enumerate}

\begin{definition}
Let $B$ be a fixed reflexive bilinear form on $V$. If $W\subseteq V$ is a subspace,
\end{definition}

\begin{enumerate}
\item We say $W$ is an isotropic subspace when Rad$_{B}(W)=W$.

\item We say $W$ is a coisotropic subspace when Rad$_{B}(W)=W^{\bot}$.

\item We say $W$ is a Lagrangian subspace when $W=W^{\bot}$ which is to say it
is both isotropic and coisotropic.

\item We say $W$ is nondegenerate when .Rad$_{B}(W)=\{0\}$.
\end{enumerate}

All of these definitions are "relative to $B$." Notice that we defined
"orthogonal." relative to a fixed reflexive bilinear form. This is the first
step in introducing direction into a vector space by creating a notion of
vectors being perpendicular. But realize that we use in even when
"perpendicular" does not make any sense; for example, if $v$ is isotropic
under $B$, then it is orthogonal to itself.

\subsubsection{Matrices for Bilinear Forms}

\begin{theorem}
Let $B$ be a fixed reflexive bilinear form on $V$. \ If $W$ is a nondegenerate
subspace of $V$, then
\[
V=W\oplus W^{\bot}\text{.}%
\]

\end{theorem}

\subsection{}

\begin{definition}
If $B$ is a symmetric bilinear form on the finitely generated vector space $V$
with basis $S=\{e_{1}$, $e_{2},\cdots e_{n}\}$. Then the matrix of $B$
relative to $S$ is%
\[
|B|_{S}=(B(e_{i}\text{, }e_{j}))_{n\times n}\text{.}%
\]


\begin{remark}
If $B$ is a symmetric bilinear form on the finitely generated vector space $V$
with basis $S=\{e_{1}$, $e_{2},\cdots e_{n}\}$ and
\begin{align*}
v  &  =a_{1}e_{1}+a_{2}e_{2}+\cdots a_{n}e_{n}\text{.}\\
w  &  =b_{1}e_{1}+b_{2}e_{2}+\cdots b_{n}e_{n}\text{.}%
\end{align*}
We write out two (row) vectors in $F^{n}$ as $\overline{v}=(a_{1},a_{2},\cdots
a_{n})$ and $\overline{w}=(b_{1},b_{2},\cdots b_{n}).$ Then
\[
B(v,w)=\overline{v}|B|_{S}\overline{w}\text{.}%
\]

\end{remark}
\end{definition}

\begin{remark}
Let $V$ be a finitely generated vector space of dimension $n$ over $F$, and we
consider Bilin$_{F}(V)$ as a vector space over $F$, then we have an
isomorphism%
\[
\text{Bilin}_{F}(V)\leftrightarrow\text{Mat}_{n\times n}(F)
\]
relative to the basis $S=\{e_{1}$, $e_{2},\cdots e_{n}\}$.
\end{remark}

Our first question is, how do we change bases in the representation of the
matrix of a bilinear form?

Let $V$ be a vector space over $F$. We change basis using the automorphism of
$V$:
\[
\text{Aut}_{F}(V)=\left\{  T:V\rightarrow V|\;T\text{ is a bijective linear
map}\right\}
\]


\begin{theorem}
Let $V$ be a finitely generated vector space over $F$. Let
\begin{align*}
S_{1}  &  =\{s_{1,1},s_{1,2},s_{1,3}\cdots s_{1,n}\}\\
S_{2}  &  =\{s_{2,1},s_{2,2},s_{2,3}\cdots s_{2,n}\}
\end{align*}
be two bases of $V.$ (Note the order will definitely matter.)

We can create an element $C\in$ Aut$_{F}(V)$ by extending the formula%
\[
C(s_{1,i})=s_{2,i}%
\]
linearly to all vectors of $V$.

Let $B:V\times V\rightarrow F$ be a bilinear form on $V$. Then
\[
\left(  \left\vert B\right\vert _{S_{2}}\right)  =\left(  _{S_{1}}\left\vert
C\right\vert _{S_{2}}\right)  ^{T}\left(  \left\vert B\right\vert _{S_{1}%
}\right)  \left(  _{S_{1}}\left\vert C\right\vert _{S_{2}}\right)
\]

\end{theorem}

Next we show that in finitely generated vector spaces, where we have finite
bases, many of the terms we have defined mean the same thing about the
Bilinear form

\begin{theorem}
Let $B$ be a bilinear form on the vector space $V$ over $F$ with dimension
$n$. \ The following are equivalent statements about $B$.

\begin{enumerate}
\item $B$ is left-nondegenerate.

\item $L_{B}:$Bilin$_{F}(V)\rightarrow$Hom$_{F}(V,F)$ is an injection.

\item $L_{B}:$Bilin$_{F}(V)\rightarrow$Hom$_{F}(V,F)$ is a bijection.

\item $B$ is right-nondegenerate.

\item $R_{B}:$Bilin$_{F}(V)\rightarrow$Hom$_{F}(V,F)$ is an injection.

\item $R_{B}:$Bilin$_{F}(V)\rightarrow$Hom$_{F}(V,F)$ is a bijection.

\item $B$ is nondegenerate

\item For any basis $S=\{e_{1}$, $e_{2},\cdots e_{n}\}$, Rank$(\left(
\left\vert B\right\vert _{S}\right)  )=n$

\item For any basis $S=\{e_{1}$, $e_{2},\cdots e_{n}\}$, Det$(\left(
\left\vert B\right\vert _{S}\right)  )\neq0$.
\end{enumerate}
\end{theorem}

This allows us to define the rank of a bilinear form.

\begin{definition}
Let $B$ be a bilinear form on a finitely generated vector space $V$ over $F$
with dimension $n$. \ Then
\[
\text{Rank}(B)=\text{Rank}(\left(  \left\vert B\right\vert _{S}\right)  )
\]
for any basis $S$.
\end{definition}

\begin{remark}
Let $B$ be a bilinear form on a finitely generated vector space $V$ over $F$
with dimension $n$. Then $B$ is degenerate if and only if Rank$(B)<n$.
\end{remark}

\subsection{The Cases $F=%
%TCIMACRO{\U{211d} }%
%BeginExpansion
\mathbb{R}
%EndExpansion
$ and $F=%
%TCIMACRO{\U{2102} }%
%BeginExpansion
\mathbb{C}
%EndExpansion
$}

Our goal is still to add the concepts of magnitude and direction back into
general vector spaces.$.$A nondegenerate bilinear form $B$:$V\times
V\rightarrow F$ seems to add direction, but perhaps only because we decided to
say $B(v,w)$ means $v$ and $w$ are orthogonal$.$ To get magnitude, though, we
need a measure of size on the field before we can begin to measure vectors. So
first we restrict ourselves to fields of characteristic $0$, and the further
to only $%
%TCIMACRO{\U{211d} }%
%BeginExpansion
\mathbb{R}
%EndExpansion
$ and $%
%TCIMACRO{\U{2102} }%
%BeginExpansion
\mathbb{C}
%EndExpansion
$.\ Using the first bilinear form we ever saw, the dot product from
multivariable calculus, as a guide, we hope to define the magnitude of a
vector $v$ to be the square root of  $B(v,v)$.

\subsubsection{The case $F=%
%TCIMACRO{\U{211d} }%
%BeginExpansion
\mathbb{R}
%EndExpansion
$}

If $F$ is any field of characteristic $0$, we can identify the best kind of
basis for working with a bilinear form $B$. one where all the elements are
orthogonal.\ In the case where the vector space is finitely generated, such a
basis gives the best form of the matrix $\left\vert B\right\vert _{S}$ we can have.

\begin{theorem}
If $F$ is any field of characteristic $0$. Let $B$ be a symmetric bilinear
form on a finitely generated vector space $V$ over $F$ with dimension $n$. Let
Rank$(B)=r$. Then there exists a basis $S$ of $V$ so that $\left\vert
B\right\vert _{S}$ is diagonal with exactly $r$ nonzero entries on the diagonal.
\end{theorem}

Thus over any field of characteristic $0$, every vector space with a fixed
bilinear form has a basis whose elements are orthogonal.

This theorem certainly applies to $F=%
%TCIMACRO{\U{211d} }%
%BeginExpansion
\mathbb{R}
%EndExpansion
$, but in that case we can do just a little bit more. Since every positive
number in $%
%TCIMACRO{\U{211d} }%
%BeginExpansion
\mathbb{R}
%EndExpansion
$ has a square root, we can take and nonzero entry on the diagonal of our
matrix and divide the appropriate basis element by the square root of its
absolute value. That will reduce the entry in the diagonal matrix to $\pm1$ or
$0.$

\begin{theorem}
Let $B$ be a symmetric bilinear form on a finitely generated vector space $V$
over $%
%TCIMACRO{\U{211d} }%
%BeginExpansion
\mathbb{R}
%EndExpansion
$ with dimension $n$. Let Rank$(B)=r$. Then there exists a basis
\[
S=\{e_{1},e_{2},\cdots e_{n}\}
\]
of $V$ so that $\left\vert B\right\vert _{S}$ is diagonal with exactly $r$
nonzero entries on the diagonal where the nonzero diagonal entries are
\[
B(e_{i},e_{i})=\pm1\text{.}%
\]
Further, the number of times the diagonal entry is $-1$ does not depend on the
particular basis that gives this form.
\end{theorem}

\begin{definition}
If $B$ is a symmetric bilinear form on a finitely generated vector space $V$
over $%
%TCIMACRO{\U{211d} }%
%BeginExpansion
\mathbb{R}
%EndExpansion
$ with dimension $n$. the signature of $B$ is $[r,s,t]$ \ where $r$ is the
number of $+1$'s in the diagonal of the matrix in the theorem; $s$ is the
number of $-1$'s in the diagonal of the matrix, and $t$ is the number of
$0$'s$.$
\end{definition}

We are getting close to defining the magnitude of a vector. But still, not all
symmetric bilinear forms on a vector space over $%
%TCIMACRO{\U{211d} }%
%BeginExpansion
\mathbb{R}
%EndExpansion
$ are going to give a good measure of magnitude of vectors from $B(v,v)$. We
do not want a nonzero vector to have $0$ size or $-1$ either. To get a good
magnitude for a vector space of dimension $n$ over $%
%TCIMACRO{\U{211d} }%
%BeginExpansion
\mathbb{R}
%EndExpansion
$, we need a bilinear form with signature $[n,0,0]$. We can not state an
important corollary to our theorem.

\begin{corollary}
Let $B$ be a symmetric bilinear form on a finitely generated vector space $V$
over $%
%TCIMACRO{\U{211d} }%
%BeginExpansion
\mathbb{R}
%EndExpansion
$ with dimension $n$.

If the signature of $B$ is $[n,0,0]$, then there exists a basis
\[
S=\{e_{1},e_{2},\cdots e_{n}\}
\]
of $V$ so that $\left\vert B\right\vert _{S}$ $=I$, the identity matrix. We
call $S$ an orthonormal basis of $V$.
\end{corollary}

\subsubsection{The case $F=%
%TCIMACRO{\U{2102} }%
%BeginExpansion
\mathbb{C}
%EndExpansion
$}

We next move to the complex numbers. Since $%
%TCIMACRO{\U{2102} }%
%BeginExpansion
\mathbb{C}
%EndExpansion
$ has characteristic $0$, any symmetric bilinear form on a finitely generated
vector space $V$ over $%
%TCIMACRO{\U{2102} }%
%BeginExpansion
\mathbb{C}
%EndExpansion
$ can be diagonalized as a bilinear form. Since every nonzero complex number
has a complex square root, we can make all the diagonal entries $1$ or $0$. So
if the bilinear form is nondegenerate, we would seem to have an orthonormal
basis, but we do not. \ The problem is that $B(v,v)\in%
%TCIMACRO{\U{2102} }%
%BeginExpansion
\mathbb{C}
%EndExpansion
$ and $%
%TCIMACRO{\U{2102} }%
%BeginExpansion
\mathbb{C}
%EndExpansion
$ does not have a total order like $%
%TCIMACRO{\U{211d} }%
%BeginExpansion
\mathbb{R}
%EndExpansion
.$ So $B(v,v)$ is not a good measure of size. \ To deal with this, we define
an "bilinear-like" function that uses complex conjugation.

First recall that if $\alpha=a+bi\in%
%TCIMACRO{\U{2102} }%
%BeginExpansion
\mathbb{C}
%EndExpansion
$ then the conjugate of $\alpha$ is $\overline{\alpha}=a-bi$. Then
$\overline{(\alpha+\beta)}=\overline{\alpha}+\overline{\beta}$ and
$\overline{(\alpha\cdot\beta)}=\overline{\alpha}\cdot\overline{\beta}$ \ And
$\alpha=\overline{\alpha}$ if any only if $\alpha\in%
%TCIMACRO{\U{211d} }%
%BeginExpansion
\mathbb{R}
%EndExpansion
$.

\begin{definition}
Let $V$ be a vector space over the complex numbers $%
%TCIMACRO{\U{2102} }%
%BeginExpansion
\mathbb{C}
%EndExpansion
$. A Hermitian form on $V$ is a map $B:V\times V\rightarrow%
%TCIMACRO{\U{2102} }%
%BeginExpansion
\mathbb{C}
%EndExpansion
$ where

\begin{enumerate}
\item $B(av_{1}+bv_{2},w)=aB(v_{1},w)+bB(v_{2},w)$

\item $B(v,w)=\overline{B(w,v)}$.
\end{enumerate}
\end{definition}

\begin{remark}
If $B$ is a Hermitian form on $V$, then

\begin{enumerate}
\item $B(v,aw_{1}+bw_{2})=\overline{a}\;B(v,w_{1})+\overline{b}\;B(v,w_{2})$

\item $B(v,v)\in%
%TCIMACRO{\U{211d} }%
%BeginExpansion
\mathbb{R}
%EndExpansion
$.
\end{enumerate}
\end{remark}

Again we hope to find an orthonormal basis for any nondegenerate Hermitian
form. In the case of a finitely generated vector space over $%
%TCIMACRO{\U{2102} }%
%BeginExpansion
\mathbb{C}
%EndExpansion
$, this is guaranteed.

\begin{theorem}
Let $B$ be a Hermitian bilinear form on a finitely generated complex vector
space $V$ with dimension $n$. Then there exists a basis
\[
S=\{e_{1},e_{2},\cdots e_{n}\}
\]
of $V$ so that $\left\vert B\right\vert _{S}$ is diagonal with
\[
B(e_{i},e_{i})=0\text{ or \ }1
\]
The number of times the diagonal entry is $1$ is called the rank of $B$ and
does not depend on the basis.
\end{theorem}

\begin{corollary}
Let $B$ be a nondegenerate Hermitian bilinear form on a finitely generated
complex vector space $V$ with dimension $n$. Then there exists a basis
\[
S=\{e_{1},e_{2},\cdots e_{n}\}
\]
of $V$ so that $\left\vert B\right\vert _{S}$ is diagonal with
\[
B(e_{i},e_{i})=1
\]
This is called an orthonormal basis of $V$.
\end{corollary}

Notice that, because complex conjugation does not change real numbers, one
could say any symmetric bilinear form is "Hermitian." We do not though. Still
it does mean that the proofs of the theorems above are only slightly
different, and that the results are virtually the same. Because of this, we
combine the two types of vector spaces with the most desirable type of
bilinear form

\begin{definition}
An inner product space is a vector space $V$ over $%
%TCIMACRO{\U{2102} }%
%BeginExpansion
\mathbb{C}
%EndExpansion
$ or $%
%TCIMACRO{\U{211d} }%
%BeginExpansion
\mathbb{R}
%EndExpansion
$ so that

\begin{enumerate}
\item For $%
%TCIMACRO{\U{211d} }%
%BeginExpansion
\mathbb{R}
%EndExpansion
$, there is a nondegenerate bilinear form $B:V\times V\rightarrow%
%TCIMACRO{\U{211d} }%
%BeginExpansion
\mathbb{R}
%EndExpansion
$ with $B(v,v)\geq0$ for all $v\in V$.

\item For $%
%TCIMACRO{\U{2102} }%
%BeginExpansion
\mathbb{C}
%EndExpansion
$, there is a nondegenerate Hermitian form $B:V\times V\rightarrow%
%TCIMACRO{\U{2102} }%
%BeginExpansion
\mathbb{C}
%EndExpansion
$.
\end{enumerate}

\begin{notation}
If $V$ is an inner product space, we write
\[
B(v,w)=<v,w>\text{.}%
\]

\end{notation}
\end{definition}

This allows us to combine the two theorems about finitely generated vector
spaces into one.

\begin{corollary}
If $V$ is a finitely generated inner product space, then there is an
orthonormal basis of $V$.
\end{corollary}

The proofs of these theorems are actually constructive, at least for the basis
of the nondegererate part. There is a recursive process using explicit formula
that add one basis element at a time starting with any nonzero vector. This
process, known as the Gram-Schmidt algorithm, is important and you should
learn it. \ It has surprising uses in other mathematical areas including
optimization, geometry and number theory$.$

\bigskip

\subsubsection{Quadratic Forms}

In general, a quadratic form in $n$ variables is a function of the form%
\[
Q(x_{1,}x_{2,}x_{3,}\cdots x_{n,})=%
%TCIMACRO{\tsum \limits_{i=1}^{n}}%
%BeginExpansion
{\textstyle\sum\limits_{i=1}^{n}}
%EndExpansion
a_{i}x_{i,}^{2}+%
%TCIMACRO{\tsum \limits_{1\leq i<j\leq n}}%
%BeginExpansion
{\textstyle\sum\limits_{1\leq i<j\leq n}}
%EndExpansion
b_{ij}x_{i}x_{i}%
\]
That is, a quadratic form in $n$ is a homogeneous function of degree $2$. We
say $Q(x_{1,}x_{2,}x_{3,}\cdots x_{n,})$ is definite when
\[
Q(x_{1,}x_{2,}x_{3,}\cdots x_{n,})=0\text{ implies }x_{1,}=x_{2,}=x_{3,}%
\cdots=x_{n,=0}.
\]
When the quadratic form has coefficients in $%
%TCIMACRO{\U{211d} }%
%BeginExpansion
\mathbb{R}
%EndExpansion
$, we say $Q(x_{1,}x_{2,}x_{3,}\cdots x_{n,})$ is positive when
\[
\forall x_{i}\text{, }Q(x_{1,}x_{2,}x_{3,}\cdots x_{n,})\geq0\text{.}%
\]
When $Q(x_{1,}x_{2,}x_{3,}\cdots x_{n,})$ is positive definite, we can use the
square roots of $Q(x_{1,}x_{2,}x_{3,}\cdots x_{n,})$ to give the vector
$(x_{1,}x_{2,}x_{3,}\cdots x_{n,})\in%
%TCIMACRO{\U{211d} }%
%BeginExpansion
\mathbb{R}
%EndExpansion
^{n}$ a size.

Our next task is to generalize this. We can use any symmetric bilinear form to
generalize the idea of a quadratic form. In a finitely generated vector space
over $%
%TCIMACRO{\U{211d} }%
%BeginExpansion
\mathbb{R}
%EndExpansion
$ this turns out to be no generalization at all.

\begin{definition}
Let $V$ be a vector space over $F$. Let $B$ be a symmetric bilinear form on
$V$ or a Hermitian form over a complex vector space$.$ The quadratic form on
$V$ associated with $B$ is the function $Q_{B}:V\rightarrow F$ given by%
\[
Q_{B}(v)=B(v,v)\text{.}%
\]


\begin{remark}
If $Q$ is a quadratic form on $V$, the bilinear form from which it was
obtained is given by%
\[
B(v,w)=\frac{1}{2}\left(  Q(v+w)-Q(v)-Q(w)\right)  \text{.}%
\]

\end{remark}
\end{definition}

This works over any field $F$ with characteristic not $2$. \ But we have moved
to $F=%
%TCIMACRO{\U{211d} }%
%BeginExpansion
\mathbb{R}
%EndExpansion
$ and $F=%
%TCIMACRO{\U{2102} }%
%BeginExpansion
\mathbb{C}
%EndExpansion
$. \ We would like to have a quadratic form of a real or complex vector space
so that

\begin{enumerate}
\item $\forall v\in V$, $Q_{B}(v)\in%
%TCIMACRO{\U{211d} }%
%BeginExpansion
\mathbb{R}
%EndExpansion
$

\item $\forall v\in V$, $\ Q_{B}(v)\geq0$ (We say $Q$ is positive.)

\item $\forall v\in V$, $\ Q_{B}(v)=0$ if and only if $v=0.$(We say $Q$ is definite.)
\end{enumerate}

This leads to a rather obvious theorem and an expected definition.

\begin{theorem}
If $V$ is an inner product space, then the quadratic form $Q_{B}(v)=<v,v>$. is
positive definite.
\end{theorem}

\begin{definition}
If $V$ is an inner product space, the magnitude of a vector $v\in V$ is
\[
||v||=\sqrt{Q(v)}=\sqrt{<v,v>}\text{.}%
\]

\end{definition}

I

\subsubsection{The Triangle Inequality}

A magnitude can be used to measure distance by saying that the distance
between $v$ and $w$ is $||v-w||$. This only works if the result satisfies "the
triangle inequality." which basically says that the shortest distance between
two things is a straight line. More formally,

\begin{criterion}
The triangle inequality. Let $|s|$ be a measure on a set $U$. We say the
measure satisfies the triangle inequality when%
\[
\forall s,t,u\in U\text{, \ }|s-t|\,\leq|s-u|\,+|u-s|
\]
or equivalently%
\[
\forall s,t\in U\text{, \ }|s+t|\,\leq|s|\,+|u|
\]

\end{criterion}

We will now prove

\begin{theorem}
If $V$ is an inner product space, the magnitude of a vector $v\in V$ satisfies
the Triangle Inequality$.$
\end{theorem}

Before we prove this, we need

\begin{theorem}
The Cauchy--Schwarz inequality. \ In $V$ an inner product space%
\[
<u,v>\,\leq\,||u||\,||v||\text{.}%
\]


\begin{proof}
For any complex number $\lambda$ and any $u,v\in V$,
\begin{align*}
0  &  \leq||u-\lambda v||\\
&  \leq<u-\lambda v,u-\lambda v>\\
&  \leq<u,u-\lambda v>-\lambda<v,u-\lambda v>\\
&  \leq<u,u>-\overline{\lambda}<u,v>-\lambda<v,u>+\lambda\overline{\lambda
}<v,v>\\
&  \leq||u||^{2}+|\lambda|^{2}||v||^{2}-\overline{\lambda}<u,v>-\lambda<v,u>
\end{align*}


So for any $\lambda\in%
%TCIMACRO{\U{2102} }%
%BeginExpansion
\mathbb{C}
%EndExpansion
$%
\[
\overline{\lambda}<u,v>+\lambda<v,u>-|\lambda|^{2}||v||^{2}\leq||u||^{2}%
\]


In particular for $\lambda=\frac{<v,u>}{||v||^{2}},$%
\[
\overline{\lambda}<u,v>\,=\frac{|<u,v>|^{2}}{||v||^{2}}%
\]


and\thinspace%
\[
\lambda\overline{<v,u>}=\frac{|<v,u>|^{2}}{||v||^{2}}=\frac{|<u,v>|^{2}%
}{||v||^{2}}%
\]


and%

\[
|\lambda|^{2}||v||^{2}=\lambda\overline{\lambda}||v||^{2}=\frac{|<v,u>|^{2}%
}{||v||^{4}}||v||^{2}=\frac{|<u,v>|^{2}}{||v||^{2}}%
\]
%

\[
||u||^{2}\leq\frac{|<u,v>|^{2}}{||v||^{2}}+\frac{|<u,v>|^{2}}{||v||^{2}}%
-\frac{|<u,v>|^{2}}{||v||^{2}}%
\]


So%
\[
\frac{|<u,v>|^{2}}{||v||^{2}}\leq||u||^{2}%
\]


And finally\thinspace%
\[
<u,v>\,\leq\,||u||\,||v||\text{.}%
\]

\end{proof}
\end{theorem}

Now we can prove the triangle inequality holds in inner product spaces..

\begin{proof}
Assume $v$ and $w$ are elements of an inner product space.
\begin{align*}
||v+w||^{2}  &  =<v+w,v+w>\\
&  =<v,v>+<v,w>+<w,v>+<w,w>\\
&  =||v||^{2}+<v,w>+\overline{<v,w>}+||w||^{2}\\
&  =||v||^{2}+||w||^{2}+2\text{Re}(<v,w>)
\end{align*}


But the real part of any complex number is at most its absolute value. So%
\[
||v+w||^{2}\leq||v||^{2}+||w||^{2}+2\,|<v,w>|
\]


Then using the Cauchy-Schwarz Inequality%
\begin{align*}
||v+w||^{2}  &  \leq||v||^{2}+||w||^{2}+2\,||v||\,||w||\\
&  =(||v||\,+||w||)^{2}%
\end{align*}

\end{proof}

\subsubsection{Transformations of Inner Product Spaces}

Let $V$ be an inner product space$.$ Let $T:V\rightarrow V$ be a linear
transformation. Now that we have a metric on $V$, we would like this linear
transformation to preserve it. Without going into detail, this happens when
$T$ is self adjoint. That is,%
\[
<Tv,w>\,=\,<v,Tw>
\]
In the finite dimensional case, a matrix $M$ is self adjoint when it is equal
to the conjugate of its transpose $\overline{M^{T}}=M^{H}$. \ If
$T:V\rightarrow V$ is a self adjoint linear transformation, it preserves the
inner product, and there is a basis of eigenvectors of $V$ that is orthogonal.
A bit of thought and we realize that orthonormal is out of the question. But
all the eigenvalues that appear must be real.

\begin{theorem}
The Spectral Theorem$.$ If $M$ is a self adjoint matrix and it represents a
linear transformation of inner product spaces $M:V\rightarrow V$. Then there
is an orthogonal basis of eigenvalues. Thus $M$ is automatically
diagonalizable and all its eigenvalue are real.
\end{theorem}


\end{document}