Linear Algebra | Mathematics - Wyatt's Notes

sources:

• text: Spivak - Calculus

1. Vectors and Vector Spaces
2. Linear Independence, Span, Basis, and Dimension
3. Matrices
4. Systems of Linear Equations
5. Eigenvalues and Eigenvectors
6. Linear Transformations
7. Inner Product Spaces
8. Singular Value Decomposition
9. Problem Set

University-level linear algebra notes covering vector spaces, matrices, and decompositions.

• Vector Spaces and Subspaces: Definitions, examples, bases, dimension
• Matrices and Systems: Matrix operations, determinants, linear systems
• Eigenvalues and Eigenvectors: Diagonalisation, characteristic polynomials
• Linear Transformations: Kernel, image, rank-nullity theorem

• Basic algebra and arithmetic
• Mathematical proofs and logic
• Familiarity with mathematical notation

Start with vector spaces to build foundational knowledge, then progress to matrices and decompositions. Each section includes worked examples and practice problems.

Use the sidebar to browse topics, or start with the introductory pages linked from the sidebar.

Each section includes:

• Detailed explanations of key concepts
• Worked examples with step-by-step solutions
• Practice problems with answers
• Common pitfalls and how to avoid them
• Connections to other areas of mathematics

1. Master the definitions: Linear algebra requires precise understanding of vector spaces and linear maps
2. Practise proofs: Learn to write clear, rigorous proofs
3. Draw diagrams: Visualise vector spaces, transformations, and decompositions
4. Learn standard examples: Know the properties of common matrices (diagonal, symmetric, orthogonal)
5. Connect to applications: Relate linear algebra to data science, physics, and engineering

Linear algebra is the study of straight-line transformations. Vectors represent quantities with magnitude and direction; linear transformations stretch, rotate, and shear these vectors while preserving the operations of addition and scalar multiplication. Matrices are the numerical embodiment of these transformations. Eigenvalues reveal the natural stretching directions of a transformation, and diagonalisation exposes its simplest form. The singular value decomposition decomposes any matrix into rotation, scaling, and rotation, providing the best low-rank approximation to data. Linear algebra underpins everything from solving systems of equations to principal component analysis in statistics and quantum mechanics in physics.

Mistake 1: Assuming $AB = BA$ for matrix multiplication Matrix multiplication is not commutative. Even when both $AB$ and $BA$ are defined and have the same dimensions, they are generally not equal. For example, with $A = \begin{pmatrix} 0 & 1 \\ 0 & 0 \end{pmatrix}$ and $B = \begin{pmatrix} 1 & 0 \\ 0 & 0 \end{pmatrix}$, we get $AB \neq BA$. Always check the order of multiplication carefully.

Mistake 2: Assuming eigenvectors of a symmetric matrix are orthogonal only for distinct eigenvalues Eigenvectors corresponding to distinct eigenvalues of a symmetric (Hermitian) matrix are always orthogonal, but eigenvectors within the same eigenspace are not automatically orthogonal. When an eigenvalue has multiplicity greater than one, you must apply the Gram-Schmidt process to obtain an orthonormal basis for that eigenspace.

Mistake 3: Confusing the column space with the null space The column space of $A$ consists of all vectors $A\mathbf{x}$, while the null space consists of all $\mathbf{x}$ such that $A\mathbf{x} = \mathbf{0}$. These are subspaces of different spaces as a rule (column space in the codomain, null space in the domain). The rank-nullity theorem relates their dimensions, but they are not the same object.

• Abstract Algebra: Groups, rings, and modules.

• Real Analysis: Metric spaces and convergence.

• Multivariable Calculus: Applications of linear algebra in calculus.

• Quantum Mechanics

• Graph Theory