Linear Algebra by Jim Hefferon PDF Download (4th Edition) – BS Math/BSCS

Students taking a first Linear Algebra course can download the complete textbook “Linear Algebra” by Jim Hefferon, free as a PDF from the book’s official site. It is a developmental, proof-based text built for readers moving from calculation-focused early courses toward mathematical maturity — it proves everything, but motivates every proof with plenty of worked examples before asking students to write their own.

The book covers Gauss’s Method, vector spaces, linear maps and matrices, determinants, and eigenvalues and eigenvectors — the standard first-course syllabus for BS Mathematics and BSCS students. Each chapter closes with optional Topics sections on real applications (input-output economic analysis, Markov chains, computer graphics, page-ranking algorithms) and every exercise in the book has a fully worked answer in a companion answer book, which makes it equally usable for classroom study or self-study.

Book Overview

Chapter List

Chapter One, Part I: Solving Linear Systems

Difficulty: Easy · Key topics: linear equations and systems, Gauss’s Method, echelon form, free variables, homogeneous systems, General = Particular + Homogeneous

Opens with two motivating examples — a statics balance problem and a chemistry reaction-balancing problem — that both reduce to systems of linear equations, then develops Gauss’s Method as the general algorithm for solving any such system, proving that its three row operations never change a system’s solution set. It defines echelon form and free variables, shows how to describe an infinite solution set as a particular solution plus an unrestricted linear combination of vectors (the “General = Particular + Homogeneous” theorem), and closes with an optional check for when two different-looking descriptions actually describe the same set.

Key Points:

• Gauss’s Method allows exactly three row operations: swap two equations, multiply a row by a nonzero constant, add a multiple of one row to another
• A system is in echelon form when each row’s leading variable sits strictly right of the row above’s, and any all-zero rows are at the bottom
• Variables that never lead a row in echelon form are the system’s free variables
• Theorem: every solution set has the form particular solution + unrestricted linear combination of vectors, one vector per free variable
• A homogeneous equation has constant term 0; every homogeneous system has the zero vector as a solution
• A system is nonsingular exactly when it has one and only one solution

Practice Tip: After row-reducing a system, count the free variables first — that count alone tells you whether the solution set is a single point, empty, or infinite, before you do any back-substitution.

Common Mistake: Treating a row swap with itself, or scaling a row by zero, as legal Gauss’s Method moves. Scaling by 0 destroys an equation’s information and is explicitly disallowed.

Important Questions:

• What are the three row operations allowed by Gauss’s Method, and why is multiplying a row by zero not one of them? Swap two rows, multiply a row by a nonzero constant, and add a multiple of one row to another; scaling by 0 would erase an equation’s information and could change the solution set, so it is disallowed.
• How does the “General = Particular + Homogeneous” theorem describe an infinite solution set? As one particular solution vector plus an unrestricted linear combination of vectors, where the number of those vectors equals the number of free variables left after Gaussian reduction.

Chapter One, Part II: Linear Geometry

Difficulty: Easy · Key topics: vectors in R^n, vector length, dot product, Triangle Inequality, Cauchy-Schwarz Inequality, angle, orthogonality

An optional-but-load-bearing review that builds the geometric picture behind linear systems: two equations in two unknowns correspond to two lines in the plane that meet once, are parallel, or coincide — matching the three possible solution-set sizes proved algebraically in Part I. It defines vector length and the dot product, proves the Triangle Inequality, derives the Cauchy-Schwarz Inequality from it, and uses both to define the angle between two vectors in any R^n along with the special case of orthogonal (perpendicular) vectors.

Key Points:

• Two linear equations in two unknowns correspond geometrically to two lines: unique intersection, parallel, or identical
• Vector length: |u| = sqrt(u·u), built from the dot product
• The dot product u·v is a real number, not a vector, and is only defined for vectors with the same number of components
• Triangle Inequality: |u+v| ≤ |u|+|v|, with equality only when one vector is a nonnegative scalar multiple of the other
• Cauchy-Schwarz Inequality, derived from the Triangle Inequality: |u·v| ≤ |u||v|
• Two vectors are orthogonal exactly when their dot product is zero

Memory Tip: Picture the dot product’s sign as a compass: positive means the vectors point in a broadly similar direction, zero means perpendicular, negative means broadly opposite.

Common Mistake: Forgetting that the dot product requires two vectors with the same number of components — it is undefined between, say, a vector in R^2 and one in R^3.

Important Questions:

• When are two vectors in R^n orthogonal? Exactly when their dot product equals zero.
• What does the Cauchy-Schwarz Inequality state, and what is it derived from? |u·v| ≤ |u||v| for any vectors u, v in R^n; it is derived as a direct consequence of the Triangle Inequality.

Chapter One, Part III: Reduced Echelon Form

Difficulty: Medium · Key topics: Gauss-Jordan reduction, reduced echelon form, row equivalence, Linear Combination Lemma, uniqueness of reduced echelon form

Extends Gauss’s Method to Gauss-Jordan reduction, which keeps reducing past plain echelon form until every leading entry is a 1 that is alone in its column — reduced echelon form. It defines row equivalence (two matrices reachable from one another by row operations), proves the Linear Combination Lemma (a row-reduced matrix’s rows are linear combinations of the original’s), and closes the chapter with its capstone result: every matrix is row-equivalent to one and only one reduced echelon form matrix, which is why comparing two matrices’ reduced echelon forms can answer whether they came from the same original system.

Key Points:

• Gauss-Jordan reduction continues past echelon form until every leading entry is a 1, alone in its column
• Row equivalence: two matrices related by a sequence of row operations
• Linear Combination Lemma: if A row-reduces to B, every row of B is a linear combination of A’s rows
• Theorem: every matrix is row-equivalent to exactly one reduced echelon form matrix
• Plain echelon form is not unique — many different echelon forms come from the same matrix — only the fully reduced echelon form is

Practice Tip: When two people get different-looking echelon forms for the same system, push both all the way to reduced echelon form (Gauss-Jordan) — if the systems are truly equivalent, the reduced forms must match exactly.

Common Mistake: Assuming plain echelon form is unique and stopping there when comparing two answers. It isn’t — only reduced echelon form is guaranteed unique.

Important Questions:

• What extra condition turns an echelon form matrix into a reduced echelon form matrix? Every leading entry must be a 1, and that 1 must be the only nonzero entry in its entire column.
• Why does it matter that reduced echelon form is unique, beyond being convenient to compute? It guarantees that anyone who correctly row-reduces the same matrix arrives at an identical answer, so it can be used to test whether two matrices are row-equivalent.

Chapter Two, Part I: Definition of Vector Space

Difficulty: Medium · Key topics: vector space axioms, subspaces, span, spanning sets

Generalizes the pattern seen in R^n into an abstract vector space: a set with an addition and a scalar multiplication obeying ten specific conditions, including closure, commutativity, associativity, a zero vector, additive inverses, and how the two kinds of multiplication interact. Beyond R^n itself, the worked examples include spaces of polynomials, of matrices, and of functions, showing that “vector space” is genuinely broader than “a list of numbers.” The chapter then defines a subspace — a subset that is itself a vector space under the same operations — and proves that checking closure under addition and scalar multiplication alone is enough, before introducing span, the set of all linear combinations of a given set of vectors, and proving that any span is automatically a subspace.

Key Points:

• A vector space needs 10 conditions: closure and commutativity/associativity of addition, a zero vector, additive inverses, closure of scalar multiplication, two distributive laws, one associative law for scalars, and the identity 1·v = v
• Vector spaces include far more than R^n — polynomial spaces, matrix spaces, and function spaces all qualify
• A subspace is a subset that is itself a vector space using the same operations
• To verify a nonempty subset is a subspace, it suffices to check closure under addition and scalar multiplication
• The span of a set is the set of all linear combinations of its vectors, and any span is automatically a subspace

Memory Tip: When checking whether something is a vector space, test the two closure conditions (addition, scalar multiplication) first — most trick examples fail exactly there.

Common Mistake: Assuming any subset that “looks like” a vector space (for instance, one that contains the zero vector) is automatically a subspace, without actually checking closure.

Important Questions:

• Which two conditions are enough to verify that a nonempty subset of a vector space is a subspace? Closure under vector addition and closure under scalar multiplication.
• What is the span of a set of vectors, and is it always a subspace? The set of all linear combinations of the given vectors; yes, the span of any set is always a subspace.

Chapter Two, Part II: Linear Independence

Difficulty: Medium · Key topics: linear independence/dependence, minimal spanning sets, linear dependence relation

Defines what makes a spanning set “minimal” in the local sense — that removing no element shrinks its span — and names that property linear independence: no nontrivial combination of the vectors equals the zero vector. It links this directly back to Gauss’s Method from Chapter One, where a “0 = 0” row appearing in echelon form signals a linear dependence among the original rows, and proves several facts about how adding or removing a vector from a set changes both its span and its independence.

Key Points:

• A set is linearly independent if only the trivial combination (every coefficient zero) of its vectors equals the zero vector
• Linear dependence is the opposite: some nontrivial combination equals zero
• A “0 = 0” row appearing during Gauss’s Method signals a linear dependence among the system’s original rows
• Adding a vector to an independent set keeps it independent exactly when the new vector is not already in the set’s span
• Removing a vector from a set leaves the span unchanged exactly when that vector was already a linear combination of the others

Practice Tip: To test whether a set of vectors is independent, set up c1·v1 + c2·v2 + … = 0 and row-reduce — the set is independent exactly when the only solution is every c equal to zero.

Common Mistake: Confusing “spans the space” with “is independent” — a set can do either, neither, or both; independence is about not having redundant vectors, spanning is about covering the whole space.

Important Questions:

• How does linear independence relate to what happens during Gauss’s Method? A “0 = 0” row appearing in echelon form means the original rows had a linear dependence among them; no such row appears exactly when the rows were independent.
• When does removing a vector from a spanning set leave the span unchanged? Exactly when that vector is a linear combination of the other vectors in the set — that is, it was redundant.

Chapter Two, Part III: Basis and Dimension

Difficulty: Hard · Key topics: basis, standard basis, dimension, Exchange Lemma, row space, column space, rank, nonsingular equivalences

Names the idea that closes the prior section — a set that is both a minimal spanning set and a maximal independent set — as a basis, and proves via the Exchange Lemma that every basis of a given vector space has the same number of vectors, which becomes the space’s dimension. The second half reconnects vector spaces directly to linear systems: it defines a matrix’s row space, column space, and rank (the shared dimension of both), proves row rank always equals column rank, and shows that for an n-unknown system with a rank-r coefficient matrix, the homogeneous solution space has dimension n − r. It closes with a list of equivalent statements about when a square matrix is nonsingular.

Key Points:

• A basis is a linearly independent sequence of vectors that also spans the space
• The Exchange Lemma proves every basis of the same space has the same number of elements — the space’s dimension
• Row space = span of a matrix’s rows; column space = span of its columns; row rank always equals column rank, simply called the matrix’s rank
• For an n-unknown system with a coefficient matrix of rank r, the homogeneous solution space has dimension n − r
• An n×n matrix is nonsingular if and only if its rank is n, equivalently its rows (or columns) are independent, equivalently every system with that coefficient matrix has exactly one solution

Memory Tip: Dimension is literally “how many free choices” — the count of free variables left after reducing a homogeneous system, which is exactly n − rank.

Common Mistake: Thinking row rank and column rank could differ for a non-square matrix. They are always equal — that shared number is simply the matrix’s rank, whatever its shape.

Important Questions:

• What is the Exchange Lemma used to prove? That any two bases of the same vector space have the same number of vectors, so “dimension” can be defined unambiguously as that shared count.
• If a 5-unknown homogeneous system has a coefficient matrix of rank 3, what is the dimension of its solution space? n − r = 5 − 3 = 2.

Chapter Three, Part I: Isomorphisms

Difficulty: Medium · Key topics: isomorphism, structure-preserving correspondence, same dimension ⇔ isomorphic

Formalizes the intuition that some vector spaces are “essentially the same” despite having different-looking elements — for example, 2×2 matrices and 4-tall column vectors. An isomorphism is a correspondence between two spaces that is both a bijection and preserves the vector space operations. The chapter proves “isomorphic to” is an equivalence relation, and its main theorem shows that two finite-dimensional vector spaces are isomorphic if and only if they have the same dimension — so every n-dimensional real vector space is, in disguise, just R^n.

Key Points:

• An isomorphism is a correspondence between vector spaces that is a bijection and preserves both addition and scalar multiplication
• “Isomorphic to” is an equivalence relation: reflexive, symmetric, and transitive
• Main theorem: two finite-dimensional vector spaces are isomorphic exactly when they have the same dimension
• Consequence: every n-dimensional real vector space is isomorphic to R^n
• An automorphism is an isomorphism from a space to itself

Practice Tip: To show two spaces are isomorphic without building the map explicitly, just compare their dimensions — matching dimensions guarantee an isomorphism exists.

Common Mistake: Treating isomorphic spaces as literally equal. Their elements can look completely different (matrices vs. column vectors) — isomorphism only guarantees the operations behave identically.

Important Questions:

• What two conditions must a correspondence between vector spaces satisfy to be an isomorphism? It must be a bijection (one-to-one and onto) and it must preserve vector addition and scalar multiplication.
• Under what single condition are two finite-dimensional vector spaces isomorphic? Exactly when they have the same dimension.

Chapter Three, Part II: Homomorphisms

Difficulty: Medium · Key topics: homomorphism/linear map, range space, null space, rank-nullity theorem

Drops the bijection requirement from the isomorphism definition, keeping only structure preservation, to define the more general and more useful homomorphism (linear map). It proves a linear map is completely determined by its action on a basis of its domain, defines the range space (the map’s image) and null space (vectors sent to zero) as subspaces, and proves the rank-nullity theorem: the dimension of the domain equals the dimension of the range space plus the dimension of the null space.

Key Points:

• A homomorphism (linear map) preserves addition and scalar multiplication but need not be a bijection
• A linear map is fully determined once its action on a basis of the domain is known
• The range space is the map’s image; the null space is the set of vectors mapped to zero — both are subspaces
• Rank-nullity theorem: dim(domain) = dim(range space) + dim(null space)
• A linear map is one-to-one exactly when its null space contains only the zero vector

Memory Tip: Treat “rank + nullity = dimension of domain” as a budget — every dimension of the domain either survives into the range or gets “spent” collapsing into the null space.

Common Mistake: Confusing the null space (a subspace of the domain) with the range space (a subspace of the codomain) — they live in different spaces and are only connected through the rank-nullity equation.

Important Questions:

• What does the rank-nullity theorem state? For a linear map, the dimension of its domain equals the dimension of its range space plus the dimension of its null space.
• How can you tell from the null space alone whether a linear map is one-to-one? It is one-to-one exactly when its null space contains only the zero vector.

Chapter Three, Part III: Computing Linear Maps

Difficulty: Medium · Key topics: matrix representation of a linear map, representing vectors w.r.t. a basis, rank of a map vs. rank of its matrix

Turns the fact that a linear map is determined by a basis’s images into a concrete computational tool: representing vectors and maps as matrices with respect to chosen bases for the domain and codomain. It shows how to build the representing matrix column by column from the images of the domain basis vectors, proves every matrix represents some linear map once bases are fixed, and connects a map’s rank to the rank of its representing matrix and the idea of a nonsingular map.

Key Points:

• A linear map’s representing matrix is built column by column from the representations of the images of a domain basis
• Every matrix represents some linear map, once bases are fixed for domain and codomain
• The rank of a linear map equals the rank of any matrix that represents it
• A map is nonsingular exactly when its representing matrix is nonsingular
• Matrix-vector multiplication is defined precisely so that it computes h(v) from the representation of v

Practice Tip: When representing a linear map as a matrix, double-check which basis you used for the domain (its columns) and which for the codomain (how each column’s entries are read) — swapping either produces a different, wrong matrix.

Common Mistake: Forgetting that a matrix representation depends on the chosen bases — the same linear map looks like a different matrix under a different basis choice, so “the” matrix of a map is meaningless without stating the bases.

Important Questions:

• How is the matrix representing a linear map h built, given domain basis B and codomain basis D? Its j-th column is the representation, with respect to D, of h applied to the j-th vector of B.
• What does it mean for a linear map to be nonsingular, in terms of its representing matrix? The map is nonsingular exactly when its representing matrix, with respect to any bases, is a nonsingular matrix.

Chapter Three, Part IV: Matrix Operations

Difficulty: Hard · Key topics: matrix sum/scalar product, matrix multiplication, identity/diagonal/permutation matrices, elementary reduction matrices, matrix inverse, 2×2 inverse formula

Shows that matrix addition and scalar multiplication represent the sum of two linear maps and a scalar times a map, then defines matrix multiplication as the dot product of a row of the first matrix with a column of the second, and proves it represents composition of linear maps. It works through identity, diagonal, and permutation matrices, shows that every elementary row operation corresponds to left-multiplication by a specific elementary matrix, and finally defines matrix inverse, proving a matrix is invertible exactly when nonsingular and giving the explicit 2×2 inverse formula.

Key Points:

• Matrix sum and scalar product represent the sum of two linear maps and a scalar times a map
• Matrix multiplication is the dot product of a row of the first matrix with a column of the second, and it represents composition of linear maps
• Every row operation corresponds to left-multiplying by a specific elementary matrix
• A matrix is invertible if and only if it is nonsingular
• The product of two invertible matrices is invertible: (GH)^-1 = H^-1 G^-1 — note the reversed order
• The 2×2 matrix [[a,b],[c,d]] is invertible exactly when ad − bc ≠ 0, with an explicit formula for its inverse

Practice Tip: Before multiplying two matrices, check that the number of columns in the first equals the number of rows in the second — a product that doesn’t line up is simply undefined.

Common Mistake: Assuming matrix multiplication is commutative. In general GH ≠ HG, even when both products are defined.

Important Questions:

• What condition on ad − bc tells you a 2×2 matrix is invertible? The matrix is invertible exactly when ad − bc ≠ 0 (it is nonsingular).
• If G and H are both invertible, what is (GH)^-1? H^-1 G^-1 — the inverse of a product is the product of the inverses in reverse order.

Chapter Three, Part V: Change of Basis

Difficulty: Medium · Key topics: change-of-basis matrix, representation of vectors/maps in different bases, matrix equivalence, canonical form for matrix equivalence

Addresses the fact that a vector’s or a map’s matrix representation depends entirely on which basis is used, by building the change-of-basis matrix that translates a representation with respect to one basis into its representation with respect to another, then extends this to linear maps. It introduces matrix equivalence — two matrices representing the same map under possibly different basis pairs — and proves matrices are equivalent exactly when they share the same rank, with the canonical form being a partial-identity matrix.

Key Points:

• A change-of-basis matrix converts a vector’s representation in one basis directly into its representation in another
• The same identity map, represented with respect to two different bases, is exactly the change-of-basis matrix
• Changing a linear map’s domain and/or codomain basis multiplies its representing matrix by change-of-basis matrices on the appropriate side
• Two matrices are matrix equivalent when they represent the same underlying map under some pair of bases
• Theorem: matrices are equivalent if and only if they have the same rank; the canonical form for equivalence is a partial-identity matrix

Memory Tip: Think of a change-of-basis matrix as a translator, not a transformer — it doesn’t change the underlying vector or map, only the “language” used to describe it.

Common Mistake: Confusing matrix equivalence (same rank, any basis pair) with matrix similarity (same map, but one shared basis for both domain and codomain) — Chapter Five is built entirely around the stricter, similarity notion.

Important Questions:

• What single number determines whether two matrices are matrix equivalent? Their rank; two matrices are equivalent exactly when they have the same rank.
• What is the canonical form for matrix equivalence? A partial-identity matrix, with 1’s down as much of the diagonal as the rank allows and 0’s elsewhere.

Chapter Three, Part VI: Projection

Difficulty: Medium · Optional — a prerequisite only for the final parts of Chapter Five · Key topics: orthogonal projection into a line, orthogonal/orthonormal bases, Gram-Schmidt process, projection into a subspace

Generalizes the geometric idea of a “shadow” — projecting a vector orthogonally onto a line — into a formula, then uses repeated orthogonal projection to build the Gram-Schmidt process, which converts any basis into an orthogonal (or, after normalizing, orthonormal) basis for the same space. The final part generalizes further to projection into an arbitrary subspace along an arbitrary complementary subspace, not necessarily an orthogonal one.

Key Points:

• Orthogonal projection of v onto the line spanned by s is (v·s / s·s)·s
• An orthogonal basis has every pair of vectors mutually perpendicular; orthonormal adds that every vector has length 1
• The Gram-Schmidt process builds an orthogonal basis one vector at a time by subtracting off projections onto the previously built vectors
• Any orthogonal basis for a subspace can be scaled into an orthonormal one
• Projection into a general subspace M along a complementary subspace N generalizes orthogonal projection, the special case where N is M’s orthogonal complement

Practice Tip: Run Gram-Schmidt on a simple 3-vector basis of R^3 by hand once, projecting the second vector off the first and the third off both, to see directly why each leftover component is always orthogonal to what was subtracted.

Common Mistake: Using the orthogonal-projection formula with a non-normalized direction vector but forgetting the denominator — it is (v·s/s·s)·s, not (v·s)·s.

Important Questions:

• What does the Gram-Schmidt process produce from an arbitrary basis? An orthogonal (and, after scaling, orthonormal) basis that spans the same subspace as the original.
• What is the formula for the orthogonal projection of vector v onto the line spanned by s? (v·s / s·s)·s.

Chapter Four, Part I: Definition

Difficulty: Hard · Key topics: determinant as a nonsingularity test, permutation expansion, multilinearity, existence and uniqueness of the determinant function

Starts from the 1×1, 2×2, and 3×3 nonsingularity formulas (a ≠ 0; ad − bc ≠ 0; the six-term 3×3 formula) and asks whether one family of formulas generalizes the pattern. It defines the determinant axiomatically — multilinear in the rows, alternating on a row swap, 1 on the identity — then proves such a function exists and is unique by giving the permutation expansion: a sum, over every permutation of {1,…,n}, of signed products with one entry from each row and column.

Key Points:

• 1×1 determinant is a; 2×2 is ad−bc; 3×3 is aei+bfg+cdh−hfa−idb−gec — a matrix is nonsingular exactly when its determinant is nonzero
• The determinant is defined by conditions (multilinear in the rows, alternating sign on a row swap, det of the identity is 1) before being shown to exist
• The permutation expansion writes det(T) as a sum over all n! permutations of {1,…,n}, each term a signed product with one factor from each row and column
• A permutation’s sign (signum) is +1 or −1 depending on whether it takes an even or odd number of swaps to reach
• det(T) = det(T-transpose) — a determinant can equally be computed by rows or by columns

Memory Tip: For a 3×3 matrix, the diagonal mnemonic (“Sarrus’s Rule”) — three down-right products minus three down-left products — reproduces the six-term permutation expansion without memorizing signs one by one.

Common Mistake: Trying to extend the 2×2/3×3 diagonal shortcut directly to 4×4 matrices — Sarrus’s Rule only works for 3×3; larger matrices need the full permutation expansion or Laplace expansion instead.

Important Questions:

• What condition on a matrix’s determinant tells you the matrix is nonsingular? The matrix is nonsingular exactly when its determinant is nonzero.
• What is the permutation expansion of a determinant? A sum over every permutation φ of {1,…,n} of the signed product t(1,φ(1))·t(2,φ(2))…t(n,φ(n)), with sign given by the permutation’s parity.

Chapter Four, Part II: Geometry of Determinants

Difficulty: Medium · Key topics: box (parallelepiped), determinant as a size/volume function, sign of the determinant, determinant of an inverse

Complements the algebraic definition with a geometric one: the determinant of n vectors in R^n, up to sign, equals the volume of the “box” (parallelepiped) those vectors form. It proves a linear map multiplies volumes by exactly the absolute value of its determinant, and that the determinant’s sign tracks whether the map preserves or reverses orientation, with a quick corollary that the determinant of an invertible matrix’s inverse is the reciprocal of the original.

Key Points:

• The box formed by vectors v1,…,vn is the set of all points t1v1+…+tnvn with each ti in [0,1]
• Up to sign, a matrix’s determinant equals the volume of the box formed by its rows (or columns)
• A linear map with matrix T scales the volume of any region by the factor |det(T)|
• The sign of the determinant indicates whether the map preserves (positive) or reverses (negative) orientation
• det(T^-1) = 1/det(T) for any invertible matrix T

Memory Tip: Picture the determinant as a “volume multiplier with a direction flag” — magnitude scales volume, sign says whether the map flips the space inside out.

Common Mistake: Reading a zero determinant as “shrinks things a little” rather than what it means — the map collapses the box to zero volume, sending the space into a lower dimension.

Important Questions:

• Geometrically, what does the absolute value of a matrix’s determinant represent? The factor by which the linear map scales the volume of any region.
• If det(T) = 5, what is det(T^-1)? 1/5, since det(T^-1) = 1/det(T).

Chapter Four, Part III: Laplace’s Formula

Difficulty: Medium · Optional — a prerequisite only for Chapter Five’s Nilpotence section · Key topics: minors and cofactors, Laplace expansion, adjoint matrix, cofactor formula for the inverse

Derives, from the permutation expansion, a practical by-hand method: Laplace’s expansion, which computes a determinant by expanding along any single row or column into a signed sum of smaller determinants (minors) of the matrix with that row and column deleted. It defines the adjoint matrix (the transpose of the matrix of cofactors) and proves that a matrix times its adjoint gives a diagonal matrix of determinants, yielding a cofactor-based formula for the inverse of a nonsingular matrix.

Key Points:

• A minor is the determinant of the matrix left after deleting one row and one column
• Laplace’s expansion computes a determinant as a signed sum of entries times their minors, along any chosen row or column
• The sign attached to each term alternates in a checkerboard pattern based on position: (−1)^(i+j)
• The adjoint of a matrix is the transpose of its matrix of cofactors
• T times its adjoint equals det(T) times the identity, giving a formula for T^-1 in terms of cofactors

Practice Tip: When computing a determinant by hand, expand along whichever row or column has the most zeros — each zero entry eliminates one minor computation entirely.

Common Mistake: Forgetting the alternating sign pattern when expanding — the cofactor of entry (i,j) carries a factor of (−1)^(i+j), not just the minor itself.

Important Questions:

• What is Laplace’s expansion used for, and along what can you expand? Computing a determinant as a signed sum of entries times their minors, along any single row or any single column.
• What is the adjoint of a matrix, and how does it relate to the inverse? The transpose of the matrix of cofactors; for a nonsingular matrix T, T^-1 equals the adjoint of T divided by det(T).

Chapter Five, Part I: Complex Vector Spaces

Difficulty: Medium · Key topics: complex numbers as scalars, polynomial factoring, Division Theorem for Polynomials, Fundamental Theorem of Algebra

Motivates a shift from real to complex scalars by noting that many real polynomials — including the characteristic polynomials central to the rest of the chapter — don’t factor over the real numbers (x²+1 has no real roots) but always factor completely over the complex numbers. It reviews polynomial division, the link between a polynomial’s roots and its linear factors, and states the Fundamental Theorem of Algebra: every complex polynomial of degree n factors into exactly n linear factors, counting multiplicity.

Key Points:

• The rest of the chapter requires complex scalars because characteristic polynomials don’t always factor over the reals
• Division Theorem for Polynomials: dividing one polynomial by another gives a unique quotient and remainder, just like integer division
• A number is a root of a polynomial exactly when (x − root) is a linear factor of it
• Every real polynomial factors into linear and irreducible quadratic real factors
• Fundamental Theorem of Algebra: every degree-n complex polynomial factors into exactly n linear factors, counting multiplicity

Memory Tip: Remember why complex numbers appear here — not for their own sake, but because “every polynomial factors completely” is only true over the complex numbers, and factoring characteristic polynomials is the whole point of this chapter.

Common Mistake: Assuming a real matrix always has real eigenvalues. A real matrix’s characteristic polynomial can have complex roots — exactly why this chapter switches to complex scalars.

Important Questions:

• Why does this chapter switch from real to complex scalars? Because characteristic polynomials, needed to find eigenvalues, don’t always factor into real linear factors, but by the Fundamental Theorem of Algebra they always factor completely over the complex numbers.
• What does the Fundamental Theorem of Algebra state? Every degree-n polynomial with complex coefficients factors into exactly n linear factors over the complex numbers, counting multiplicity.

Chapter Five, Part II: Similarity

Difficulty: Hard · Key topics: similar matrices, diagonalizability, eigenvalues, eigenvectors, characteristic polynomial, eigenspace, distinct-eigenvalues theorem

Specializes matrix equivalence to the case where domain equals codomain and the same basis is used on both sides, defining two matrices as similar when one represents the same transformation as the other under a change of basis. It defines a diagonalizable transformation as one similar to a diagonal matrix, then introduces eigenvalues and eigenvectors (t(ζ) = λζ for nonzero ζ), the characteristic polynomial whose roots are the eigenvalues, and each eigenvalue’s eigenspace. The key theorem proves eigenvectors for distinct eigenvalues are automatically independent, so n distinct eigenvalues on an n-dimensional space guarantee diagonalizability.

Key Points:

• Similar matrices represent the same linear transformation under a single, shared change of basis
• A transformation is diagonalizable exactly when it has a basis made entirely of eigenvectors
• Eigenvalue/eigenvector: t(ζ) = λζ for some nonzero vector ζ
• The characteristic polynomial’s roots are exactly the transformation’s eigenvalues
• The eigenspace for a given eigenvalue is itself a subspace
• Eigenvectors for distinct eigenvalues are always linearly independent; n distinct eigenvalues on an n-dimensional space guarantees diagonalizability

Practice Tip: To find eigenvalues by hand, solve det(T − λI) = 0 for λ, then for each eigenvalue solve (T − λI)ζ = 0 to find its eigenvectors — always check by verifying Tζ actually equals λζ.

Common Mistake: Forgetting that the zero vector can never be an eigenvector — the definition explicitly requires ζ to be nonzero, or “eigenvalue” would be meaningless.

Important Questions:

• What guarantees that an n×n matrix with n distinct eigenvalues is diagonalizable? Eigenvectors for distinct eigenvalues are automatically linearly independent, so n distinct eigenvalues on an n-dimensional space produce a full basis of eigenvectors.
• How are a transformation’s eigenvalues found from its characteristic polynomial? They are exactly the roots of the characteristic polynomial.

Chapter Five, Part III: Nilpotence

Difficulty: Hard · Optional prerequisite: Laplace’s Formula · Key topics: self-composition, nilpotent transformation, index of nilpotency, string basis, canonical form for nilpotent matrices

Handles the case diagonalization can’t reach: a nilpotent transformation, one where some power t^j equals the zero map, sharply illustrating why not every matrix is diagonalizable. It studies self-composition of a transformation, defines the index of nilpotency as the smallest such j, and builds a “string basis” from chains of vectors under repeated application of t, proving every nilpotent transformation has a canonical matrix form of only 0’s and 1’s, with 1’s only immediately below the diagonal in blocks.

Key Points:

• A transformation t is nilpotent if some power t^j is the zero map; the smallest such j is its index of nilpotency
• Nilpotent transformations are never diagonalizable except the zero map itself — their only eigenvalue is 0
• A “string” is a chain of vectors v, t(v), t²(v), … ending the first time it hits zero
• Every nilpotent transformation has a basis made of such strings, giving a canonical form of 0’s and 1’s, with 1’s only directly below the diagonal, arranged in blocks
• This canonical form is the building block used for Jordan form in the next section

Memory Tip: Think of nilpotency as “everything eventually dies” — repeatedly applying the map to any vector eventually lands on zero, in at most (index) steps.

Common Mistake: Assuming a nilpotent matrix must be the zero matrix. A nonzero matrix can be nilpotent — for instance, a matrix with 1 just below the diagonal and 0 elsewhere squares to the zero matrix without being zero itself.

Important Questions:

• What does it mean for a linear transformation t to be nilpotent? Some power t^j of the transformation, composed with itself j times, equals the zero map.
• What shape does the canonical matrix form for a nilpotent transformation take? A matrix of only 0’s and 1’s, with 1’s appearing only immediately below the diagonal, arranged in blocks corresponding to the “strings.”

Chapter Five, Part IV: Jordan Form

Difficulty: Hard · Key topics: polynomials of maps/matrices, Cayley-Hamilton theorem, minimal polynomial, Jordan block, Jordan canonical form

Completes the chapter’s program of finding a canonical form for matrix similarity by combining the diagonalizable case and the nilpotent case: it proves the Cayley-Hamilton theorem (every matrix satisfies its own characteristic polynomial), defines the minimal polynomial, and shows any transformation on a finite-dimensional space can be represented, in a suitable basis, as a matrix that is the sum of a diagonal part and a nilpotent part — a Jordan block structure. Jordan canonical form is the book’s final, most general answer to “what is the simplest matrix similar to a given one?”

Key Points:

• Cayley-Hamilton theorem: every square matrix satisfies its own characteristic polynomial
• The minimal polynomial of a matrix is the lowest-degree polynomial it satisfies, and it always divides the characteristic polynomial
• A Jordan block is a square matrix with a single eigenvalue on the diagonal, 1’s directly below, and 0’s elsewhere
• Jordan canonical form: every square matrix is similar to a matrix built from Jordan blocks along the diagonal
• Jordan form generalizes diagonal form: a diagonalizable matrix’s Jordan form is simply diagonal (every block is 1×1)

Memory Tip: Jordan form is “diagonal form’s fallback” — when repeated eigenvalues don’t leave enough independent eigenvectors to diagonalize, Jordan blocks patch in just enough off-diagonal 1’s for a clean canonical answer.

Common Mistake: Assuming every square matrix is diagonalizable. Many are not — Jordan form exists precisely because diagonal form isn’t always achievable, and it is the true general canonical form for similarity.

Important Questions:

• What does the Cayley-Hamilton theorem state? That every square matrix satisfies its own characteristic polynomial — substituting the matrix itself into its characteristic polynomial gives the zero matrix.
• How does Jordan canonical form relate to diagonal form? Diagonal form is the special case where every Jordan block is 1×1; a matrix is diagonalizable exactly when its Jordan form has no blocks larger than that.

Appendix: Statements, Quantifiers, Proof Technique, and Sets, Functions, and Relations

Appendix · Difficulty: Easy · Key topics: logical connectives, quantifiers, direct/contrapositive/contradiction proof, mutual inclusion, functions and relations

A reference appendix, not part of the semester’s syllabus, that sketches (without proofs of its own) the background logic and proof vocabulary the rest of the book assumes: how “not,” “and,” “or,” and “if-then” statements are evaluated for truth, how “for all” and “there exists” quantifiers work and how their order changes meaning, the standard proof techniques (direct, contrapositive, contradiction), and a review of set, function, and relation vocabulary — subset, mutual inclusion, one-to-one, onto, equivalence relation — all used throughout the main chapters.

Key Points:

• An “if P then Q” statement is true unless P is true and Q is false — including vacuously true whenever P is false
• Two main ways to prove an implication: direct proof, or proving the logically equivalent contrapositive
• Swapping the order of “for all” and “there exists” quantifiers can change a statement’s meaning entirely
• To show two sets are equal, prove mutual inclusion: each is a subset of the other
• An equivalence relation must be reflexive, symmetric, and transitive — row equivalence and matrix similarity, both defined earlier in the book, are examples

Common Mistake: Treating “if P then Q” as claiming P causes Q, or that P must come before Q in time — in mathematics it is a purely truth-functional statement, true whenever P is false or Q is true, with no causal or temporal meaning implied.

Important Questions:

• Under what circumstance is an “if P then Q” statement automatically true, regardless of Q? Whenever P itself is false — this is called vacuous truth.
• What must you show to prove that two sets S1 and S2 are equal? Mutual inclusion: every member of S1 is in S2, and every member of S2 is in S1.

Download Linear Algebra (Hefferon) PDF (Free)

This book is free from its official source, author Jim Hefferon’s own book page. Click below to go to the official download page — set the price slider to $0 for a completely free copy of the complete PDF. A free companion answer book with every exercise fully worked, beamer slides, and a Sage lab manual are also linked from the same official page.

How to Study This Book

Hefferon’s own preface gives a week-by-week teaching schedule built around exactly the Roman-numeral sections used above, which makes this book unusually easy to plan a semester around. A typical 14-week course covers One.I in the first two weeks, One.III (skipping straight past the optional One.II if your class has already had vectors) in week two, all of Chapter Two by week five or six, most of Chapter Three’s six sections by week eleven, Chapter Four’s first and third sections by week twelve, and Chapter Five’s Similarity section in the last two weeks — leaving Nilpotence and Jordan Form as natural material for a second course or an honors extension, since the author’s own schedule does not reach them in one semester.

Read One.I and One.III first no matter what — they are not optional, and every later chapter’s proofs lean on Gauss’s Method and reduced echelon form. One.II (Linear Geometry) is explicitly marked optional review by the book itself: skip it if you have already seen dot products, length, and the Triangle Inequality; read it if you have not, since Two.III’s rank theorems quietly assume the geometric vocabulary.

Chapter Two is the hinge of the whole book. Two.I’s vector space axioms and Two.II’s linear independence feel abstract on a first pass, but Two.III’s basis-and-dimension material, and its second half connecting row space, column space, and rank back to Gauss’s Method, is what every later chapter builds on. Budget real time here rather than rushing to Chapter Three.

In Chapter Three, Three.VI (Projection) is explicitly optional — the book itself says it is only a prerequisite for the last two sections of Chapter Five, which most one-semester courses never reach. Skip it on a first pass unless your syllabus specifically continues into Jordan form, or unless you want Gram-Schmidt for its own sake.

In Chapter Four, Four.III (Laplace’s Formula) is also explicitly optional, needed only for Five.III (Nilpotence). If your course stops at Similarity and eigenvalues, as the author’s own schedule does, you can treat Four.I and Four.II as the complete determinants unit and skip Four.III entirely.

Chapter Five is where the payoff arrives: eigenvalues and eigenvectors in Five.II are the single most-tested topic in almost every linear algebra course and the material students search for most. Read Five.I (Complex Vector Spaces) immediately before Five.II even if your course is light on complex numbers elsewhere — the characteristic polynomial genuinely needs it, and skipping it makes Five.II’s proofs feel like they come from nowhere.

Because this book proves everything, the single best way to use it for self-study is to work every “Recommended” exercise (marked in the book’s own margins) and check each one immediately against the free companion answer book — do not read the answer before attempting the problem, since the point of a proof-based course is the attempt, not the answer. For independent learners without a classroom, the author’s own advice is to watch the matching video lecture first, then read the section, then do the exercises, in that order — a full playlist covering the entire book is linked from the author’s official page alongside the PDF.

Used In These Programs

This book is used for the Linear Algebra course in: BS Mathematics · BS Computer Science. Browse all Linear Algebra books or all Mathematics books.

Who Should Read This

This book is written for sophomores or juniors with at least one semester of calculus behind them — the author says so explicitly in his own preface — and it suits BS Mathematics students taking their first proof-based course, as well as BSCS students who want the theory behind the linear algebra used in graphics, machine learning, and algorithms, not just the computational recipes. It is not the right choice for a reader who wants only computational shortcuts with no proofs; every result here is proved, and the book deliberately builds mathematical maturity along the way, so a reader looking purely for worked numerical examples without justification will find it slower going than a computation-first text. Students who have not yet studied any calculus, or who want a gentler on-ramp to proof-writing first, may prefer to work through a first course in Discrete Mathematics: An Open Introduction before starting here, since it develops direct, contrapositive, and contradiction proofs from scratch.

Applicable Universities

This book is useful for students at Pakistani universities offering BS Mathematics and BS Computer Science including Punjab University, Virtual University, COMSATS, FAST, UET, NUST, and other HEC-recognized institutions.

FAQs

Related Books

• Discrete Mathematics: An Open Introduction – Oscar Levin
• Calculus Volume 1 – Strang & Herman (OpenStax)
• Calculus Volume 2 – Strang & Herman (OpenStax)

Linear Algebra by Jim Hefferon is a free, developmental first course in linear algebra covering everything from Gauss’s Method through Jordan canonical form, built for BS Mathematics and BSCS students who want proofs, not just recipes. Browse more Mathematics books for the rest of your semester.

Linear Algebra, Fourth edition (second printing, 2021-Oct-12), by Jim Hefferon, Saint Michael’s College. Free under the GNU Free Documentation License or the Creative Commons Attribution-ShareAlike 3.0 United States License, at the reader’s discretion. Official source and free download: https://hefferon.net/linearalgebra/