BS Math and BSCS students can use this guide to study Understanding Linear Algebra by David Austin of Grand Valley State University, a free first-course text under a Creative Commons Attribution 4.0 licence. The 2026 Update covers systems of equations, vectors and matrices, invertibility and bases, eigenvalues, numerical methods, orthogonality with least squares, and singular value decompositions, and it does not assume any calculus.
The book teaches by doing. Every section opens with a preview activity and then works through activities where you discover an idea before it gets a name, with embedded Sage cells so you can run the computation yourself. Applications such as computer animation, JPEG compression, Google’s PageRank and principal component analysis show why the theory matters, which helps in BS Math exams and in BSCS courses on graphics and data science. It is not an introduction to proofs, so pair it with a proof-based text if your course asks you to write them.
Book Overview
| Course | Linear Algebra |
| Degree Programs | BS Math, BSCS |
| Level | University — first course in linear algebra; written without assuming calculus |
| Edition | 2026 Update (August 20, 2026) |
| Author | David Austin, Grand Valley State University |
| Structure | 7 chapters plus Appendix A (Sage Reference), 519 pages in the PDF |
| Activities and Exercises | A preview activity in every section, over 100 further activities, and about 300 numbered exercises |
| Language | English |
| License | Creative Commons Attribution 4.0 International (CC BY 4.0) |
| Format | Free PDF, free online HTML edition with Sage cells, Runestone interactive version, separate activities workbook PDF; low-cost print edition from 619 Wreath Publishing |
Chapter List
Chapter 1: Systems of equations
Difficulty: Easy · Key topics: linear systems, Gaussian elimination, augmented matrices, reduced row echelon form, Sage rref, pivot positions, basic and free variables
The chapter starts by graphing small systems of two equations and noticing that a linear system has no solution, exactly one solution, or infinitely many. Section 1.2 turns that observation into an algorithm: scaling, interchange and replacement operations on an augmented matrix lead to a unique reduced row echelon form, a process called Gaussian elimination. Section 1.3 hands the arithmetic to Sage and notes that the work grows in proportion to n^3 for an n x n matrix. Section 1.4 then uses pivot positions to decide when solutions exist and when they are unique, and names the basic and free variables.
Key Points:
- A linear system has either no solution, exactly one solution, or infinitely many solutions.
- Scaling, interchange and replacement operations never change the solution space of a system.
- Every matrix is row equivalent to exactly one reduced row echelon matrix.
- A system is inconsistent exactly when a pivot position appears in the rightmost column of its augmented matrix.
- If a system is consistent, the solution is unique when every column of the coefficient matrix has a pivot; otherwise there are infinitely many solutions.
- Columns with pivots give basic variables and columns without pivots give free variables.
Memory Tip: Check the last column first: a pivot there means no solution at all. Then check the coefficient columns: a column without a pivot is a free variable, and a free variable means infinitely many solutions.
Common Mistake: Thinking a free variable makes a system unsolvable. A free variable only means you may choose its value, so a consistent system with a free variable has infinitely many solutions.
Important Questions:
- When is a linear system inconsistent? When a pivot position appears in the rightmost column of the augmented matrix. In that case a row says that zero equals a nonzero number, so no solution exists.
- How do pivot positions decide between one solution and infinitely many? For a consistent system, one solution means every column of the coefficient matrix has a pivot. A column without a pivot is a free variable, which gives infinitely many solutions.
Chapter 2: Vectors, matrices, and linear combinations
Difficulty: Medium · Key topics: vectors, linear combinations, matrix-vector product, span, linear independence, matrix transformations, composition, rotations, homogeneous coordinates
Chapter 2 reads a linear system as a question about vectors. Section 2.1 introduces vectors and linear combinations and shows that solving a system means finding the weights in a combination. Section 2.2 packages this as the product Ax and extends it to matrix products AB. The next two sections study the span of a set of vectors and linear independence, both decided by pivot positions. Sections 2.5 and 2.6 treat a matrix as a function, show that composing transformations is matrix multiplication, study rotations, reflections and scalings, and end with how computer animation uses homogeneous coordinates.
Key Points:
- Ax is the linear combination of the columns of A that uses the entries of x as weights.
- If A is m x n and B is n x p, then AB is m x p and its columns are A times the columns of B.
- A vector b is in Span{v1, …, vn} exactly when the system with augmented matrix [v1 … vn | b] is consistent.
- If the matrix of the vectors has a pivot in every row, their span is all of R^m.
- The vectors are linearly independent exactly when the matrix has a pivot in every column, so R^m has at most m independent vectors.
- For T(x) = Ax, the columns of A are T(e1), …, T(en), and composing transformations corresponds to multiplying matrices.
- A function is a matrix transformation exactly when T(cv) = cT(v) and T(v + w) = T(v) + T(w).
Memory Tip: Rows decide span, columns decide independence: a pivot in every row means the columns span the whole space, and a pivot in every column means they are independent.
Common Mistake: Treating AB and BA as the same thing. Matrix multiplication is not commutative, and AB only exists when the number of columns of A equals the number of rows of B.
Important Questions:
- How can you tell whether a vector b lies in the span of v1, …, vn? Row reduce the augmented matrix [v1 … vn | b]. The vector b is in the span exactly when that system is consistent.
- How do you find the matrix of a transformation T? Apply T to each standard basis vector e1, …, en. The resulting vectors T(e1), …, T(en) are the columns of the matrix.
Chapter 3: Invertibility, bases, and coordinate systems
Difficulty: Medium · Key topics: inverse matrices, elementary matrices, bases, coordinates, image compression, determinants, cofactor expansion, subspaces, column space, null space
Chapter 3 studies what happens when existence and uniqueness of solutions hold at the same time. Section 3.1 defines the inverse, finds it by row reducing [A | I], and links row operations to elementary matrices. Section 3.2 defines a basis and the coordinates of a vector in that basis. Section 3.3 applies a change of basis to image compression, covering colour models, the Discrete Fourier Transform and the JPEG algorithm on 8 x 8 blocks. Section 3.4 introduces determinants as signed area and as a test for invertibility, and Section 3.5 closes with subspaces, the column space and the null space.
Key Points:
- A square matrix A is invertible exactly when it is row equivalent to the identity matrix.
- Row reducing [A | I] to [I | A^(-1)] finds the inverse, and then the solution of Ax = b is x = A^(-1) b.
- A basis is a linearly independent set that spans, and a basis for R^m always has exactly m vectors.
- Every vector has exactly one expression in a basis, and the weights in it are its coordinates.
- det(AB) = det(A) det(B), the determinant of a triangular matrix is the product of its diagonal entries, and det(A) is nonzero exactly when A is invertible.
- dim Col(A) = rank(A) and dim Nul(A) = n – rank(A) for an m x n matrix A.
Memory Tip: Treat the determinant as a yes or no test for invertibility. For a 2 x 2 matrix it is the signed area of the parallelogram made by the columns, so zero means the columns are squashed flat and no inverse exists.
Common Mistake: Taking a basis for Col(A) from the pivot columns of the reduced row echelon form. Use the columns of the original matrix A that have pivot positions, because row operations change the column space.
Important Questions:
- How do you find the inverse of a matrix? Row reduce the augmented matrix [A | I]. If A is invertible it becomes [I | A^(-1)], and the right-hand block is the inverse.
- What does a nonzero determinant tell you? That the square matrix is invertible. A zero determinant means the matrix is not invertible, and for a 2 x 2 matrix it means the columns span zero area.
Chapter 4: Eigenvalues and eigenvectors
Difficulty: Hard · Key topics: eigenvalues, eigenvectors, characteristic polynomial, eigenspaces, diagonalization, similarity, complex eigenvalues, dynamical systems, Markov chains, PageRank
Chapter 4 begins with the idea that in the right basis a complicated matrix can act like a diagonal one. Section 4.1 defines eigenvectors through Av = λv and shows why they simplify repeated multiplication. Section 4.2 finds eigenvalues as roots of the characteristic polynomial det(A – λI) and eigenvectors as null spaces. Section 4.3 writes A = PDP^(-1), computes matrix powers, and uses similarity and complex eigenvalues to explain rotation combined with scaling. Sections 4.4 and 4.5 apply the theory to dynamical systems and to Markov chains, ending with Google’s PageRank algorithm.
Key Points:
- An eigenvector of A is a nonzero vector v with Av = λv, and λ is its eigenvalue.
- The eigenvalues are the roots of the characteristic polynomial det(A – λI), and the eigenspace is E = Nul(A – λI).
- If λ has multiplicity m in the characteristic polynomial, then 1 ≤ dim E ≤ m.
- A is diagonalizable exactly when R^n has a basis of eigenvectors, and then A = PDP^(-1) and A^k = P D^k P^(-1).
- A 2 x 2 matrix with complex eigenvalue a + bi is similar to a matrix that rotates by an angle and scales by r = √(a^2 + b^2).
- For a 2 x 2 system, both eigenvalues of absolute value below 1 give an attractor, one above and one below 1 gives a saddle, and both above 1 give a repellor.
- For a positive stochastic matrix the Perron-Frobenius theorem gives a unique, positive steady-state vector, which is the idea behind PageRank.
Memory Tip: To classify a 2 x 2 dynamical system, compare each eigenvalue’s absolute value with 1. Both below 1 pulls trajectories in, both above 1 pushes them out, and one of each is a saddle.
Common Mistake: Counting the zero vector as an eigenvector. An eigenvector must be nonzero, although the eigenvalue λ itself is allowed to be zero.
Important Questions:
- How do you find the eigenvalues and eigenvectors of a square matrix? Solve det(A – λI) = 0 for the eigenvalues. For each one, find the eigenvectors as the nonzero vectors in Nul(A – λI).
- When is a matrix diagonalizable? When there is a basis of R^n made of eigenvectors of A. Then A = PDP^(-1), with the eigenvectors as the columns of P and the eigenvalues on the diagonal of D.
Chapter 5: Linear algebra and computing
Difficulty: Medium · Key topics: round-off error, partial pivoting, permutation matrix, LU factorization, computational effort, power method, inverse power method, shifts
Chapter 5 shows how the methods of the earlier chapters are really carried out on a computer. Section 5.1 points out that computers only approximate arithmetic, so a straightforward Gaussian elimination can return answers far from the true solution. Partial pivoting, which swaps rows so each pivot is as large as possible, repairs much of this. The section then records elimination as PA = LU and shows how to solve Ax = b cheaply for many right-hand sides. Section 5.2 finds eigenvectors numerically with the power method and extends it with inverses and shifts.
Key Points:
- Computers store numbers approximately, so small round-off errors can lead to solutions far from the exact ones.
- Partial pivoting interchanges rows so the entry in each pivot position has the largest possible magnitude.
- Elimination can be recorded as PA = LU, with P a permutation matrix, L lower triangular and U upper triangular.
- Once PA = LU is known, Ax = b is solved by first solving Lc = Pb and then Ux = c.
- The power method repeats x(k+1) = A x(k) and rescales, so the vectors approach an eigenvector for the eigenvalue of largest absolute value.
- Applying the power method to A^(-1) finds the smallest eigenvalue in absolute value, and using (A – sI)^(-1) finds the eigenvalue closest to s.
Memory Tip: Think of LU as pay once, reuse many times. Factor A one time, and every new right-hand side then costs only two quick triangular solves.
Common Mistake: Expecting the power method to find any eigenvalue you ask for. It converges to the eigenvalue of largest absolute value, so you must change the matrix with an inverse or a shift to reach the others.
Important Questions:
- Why do computers use partial pivoting? Because floating-point arithmetic is only approximate. Choosing the largest available entry as the pivot improves the accuracy of the computed solution.
- How does an LU factorization help when the same matrix appears in many systems? The factorization is computed once. Each new vector b is then handled by two triangular solves, Lc = Pb and Ux = c, which is much cheaper than eliminating again.
Chapter 6: Orthogonality and Least Squares
Difficulty: Medium · Key topics: dot product, orthogonality, orthogonal complements, transpose, orthogonal projection, orthonormal bases, Gram-Schmidt, QR factorization, least squares, regression
Chapter 6 gives vectors length and angle. Section 6.1 introduces the dot product and orthogonality, with uses such as vector similarity, correlation of time series and k-means clustering. Section 6.2 brings in the transpose and orthogonal complements, and Section 6.3 builds orthogonal projection onto a subspace and orthogonal matrices. Section 6.4 gives the Gram-Schmidt algorithm and the QR factorization. Section 6.5 then uses these tools to approximate solutions of inconsistent systems by least squares, including fitting lines and polynomials to data and measuring the fit with R^2.
Key Points:
- v . v = |v|^2 and v . w = |v||w| cos(θ), so vectors are orthogonal exactly when their dot product is 0.
- Col(A)^⊥ = Nul(A^T), and for a subspace W of R^p, dim W + dim W^⊥ = p.
- For an orthogonal basis w1, …, wn of W, the projection of b is the sum of (b . wi)/(wi . wi) times wi.
- If the columns of Q are an orthonormal basis, then Q^T Q = I and the matrix QQ^T projects onto that subspace.
- Gram-Schmidt turns any basis into an orthogonal one and leads to the factorization A = QR.
- The least-squares solution of Ax = b satisfies the normal equations A^T A x = A^T b, or equivalently Rx = Q^T b when A = QR.
- The coefficient of determination R^2 measures the fraction of variance a model explains.
Memory Tip: Least squares is just projection. The best fit Ax-hat is the shadow of b on the column space of A, and the leftover error is perpendicular to that space.
Common Mistake: Using the projection formula on vectors that are only linearly independent. The simple sum of (b . wi)/(wi . wi) terms needs an orthogonal set, so run Gram-Schmidt first.
Important Questions:
- What is a least-squares solution? When Ax = b is inconsistent, it is the vector x-hat for which Ax-hat is as close to b as possible, meaning Ax-hat is the orthogonal projection of b onto Col(A).
- What does the Gram-Schmidt algorithm do? It starts from a basis of a subspace and subtracts off projections, one vector at a time, to produce an orthogonal basis. Scaling gives an orthonormal basis and the QR factorization.
Chapter 7: Singular value decompositions
Difficulty: Hard · Key topics: symmetric matrices, Spectral Theorem, covariance, quadratic forms, positive definite matrices, principal component analysis, singular value decomposition, low-rank approximation
The final chapter extends the eigenvalue ideas to every matrix. Section 7.1 shows that symmetric matrices are exactly the orthogonally diagonalizable ones (the Spectral Theorem) and uses the covariance matrix to measure the variance of data. Section 7.2 studies quadratic forms, their largest and smallest values on unit vectors, definite matrices and the Hessian test for critical points. Section 7.3 develops principal component analysis, using the iris data set as an example. Section 7.4 defines the singular value decomposition A = UΣV^T, and Section 7.5 applies it to least squares, low-rank approximation, image compression and denoising, and a study of Supreme Court voting records.
Key Points:
- The Spectral Theorem: A is orthogonally diagonalizable, A = QDQ^T, exactly when A is symmetric.
- The covariance matrix is C = (1/N) A A^T for demeaned data A, and the variance along a unit vector u is u . Cu.
- A quadratic form q(x) = x . (Ax) has maximum value λ1 and minimum value λm on unit vectors, the largest and smallest eigenvalues of A.
- A symmetric matrix is positive definite when all eigenvalues are positive, and the Hessian test uses this to spot local minima.
- The principal components are eigenvectors of C ordered by decreasing eigenvalue, and no other n-dimensional subspace keeps more variance.
- Every matrix has a singular value decomposition A = UΣV^T, and its singular values are the square roots of the eigenvalues of A^T A.
- The number of nonzero singular values is the rank, and truncating the SVD gives the closest matrix of rank k.
Memory Tip: An SVD is a ranked list of a matrix’s most important directions. The first singular value and vectors carry the most information, the second the next most, so cutting the list short compresses data while keeping the main features.
Common Mistake: Mixing up singular values and eigenvalues. Singular values are the square roots of the eigenvalues of A^T A, and they exist for every matrix, even a rectangular one that has no eigenvalues at all.
Important Questions:
- What does the Spectral Theorem say? A matrix is orthogonally diagonalizable, meaning it has an orthonormal basis of eigenvectors, exactly when it is symmetric. Then A = QDQ^T with Q orthogonal.
- How do you find a singular value decomposition? Form the symmetric Gram matrix G = A^T A. Its eigenvalues give the squared singular values, its eigenvectors are the right singular vectors, and the left singular vectors come from Avj = σj uj.
Appendix A: Sage Reference
Appendix · Difficulty: Easy · Key topics: defining matrices and vectors, integer and floating-point entries, rref, transpose, inverse, determinant, null space
Appendix A gathers the Sage commands used through the book into one short reference so you do not have to hunt for them. It shows how to define matrices and vectors, explains that integer entries and floating-point (RDF) entries are treated differently, and lists special matrices such as the identity and diagonal matrices. It then covers rref, vector and matrix arithmetic, and operations such as the transpose, inverse, determinant, null space basis and pulling out a single column. It is brief, but it is the page to keep open while you work through the activities.
Key Points:
- matrix([[1,2],[2,1]]) defines a matrix with integer entries, while adding RDF makes the entries floating-point numbers.
- A.rref() returns the reduced row echelon form of A.
- The * operator does scalar multiplication and also matrix-vector and matrix-matrix products.
- A.transpose() or A.T gives the transpose, A.inverse() or A^-1 gives the inverse, and A.det() gives the determinant.
- A.right_kernel() finds a basis for the null space Nul(A).
- v.norm() gives the length of a vector, and v*w is the dot product.
Practice Tip: Type A = matrix([[1,2],[2,1]]) in a Sage cell and run A.rref(), then redefine the same matrix with RDF to see the entries print as decimals.
Common Mistake: Entering numbers with decimal points by accident. The appendix warns that Sage treats mathematically equal matrices differently depending on how they are entered, so decimals bring floating-point approximations.
Important Questions:
- How do you compute the reduced row echelon form in Sage? Define the matrix, for example A = matrix([[1,2],[2,1]]), and call A.rref().
- How do you get a basis for the null space of A in Sage? Use A.right_kernel(), which returns the null space of A.
How to Study This Book
Read the chapters in order, because each one answers a question the previous one raised. Chapter 1 asks when a system has solutions, Chapter 2 rewrites that question with vectors and spans, and Chapter 3 asks what happens when solutions exist and are unique at the same time. The author’s site says the book can support a two-course introductory sequence, so a one-semester course often stops somewhere in Chapters 4 to 6, depending on the syllabus. Check your own course outline for where it ends.
Do the preview activity at the start of each section before you read the section. The book is built so that a definition appears after you have met the need for it, and the preview activity is where that happens. Then work the in-section activities with pencil and paper, and only afterwards read the explanation. Chapters 1 and 2 are where you build fluency with pivot positions; every later chapter assumes you can read existence and uniqueness of solutions straight from a reduced row echelon form.
Use Sage from Section 1.3 onward. The online edition has Sage cells built in, and the book’s own note says print readers can use the Sage cell server or a page of cells provided by the author. Keep Appendix A open as a command reference. In Section 4.2 pay attention to the warning about floating-point arithmetic, and in Chapter 5 notice how real software differs from the hand method of Chapter 1.
Chapters 3 and 4 are the heart of the book and the hardest part for most students. Section 3.2 (bases and coordinates) is the idea that makes eigenvectors make sense in Section 4.3, so do not rush it. Sections 3.3, 4.4 and 4.5 are applications (image compression, dynamical systems, Markov chains and PageRank). They are worth reading for the ideas, but a course focused on theory can treat them as optional reading. Chapter 5 is also separate from the main line and can be left for later if your course is not numerical.
Chapters 6 and 7 form a second stage that suits BSCS and data-oriented students especially. Section 6.5 (least squares) and Section 7.3 (principal component analysis) are the two places where the book connects directly to regression and machine learning. Section 7.1 needs the diagonalization from Section 4.3 and the orthogonal bases of Section 6.3, and Section 7.4 (singular value decompositions) builds on all of Chapter 6. The Hessian discussion in Section 7.2 is a link to multivariable calculus; the book itself does not assume calculus, so you can skim that part if you have not met it.
The book is not a proof course. The author says plainly that it is not meant to develop formal proof-writing, and it never defines abstract vector spaces: everything happens in R^n and its subspaces. If your course asks for rigorous proofs, or covers abstract vector spaces and linear transformations between them, read it alongside A First Course in Linear Algebra by Robert Beezer or Linear Algebra by Jim Hefferon.
For practice, each section ends with numbered exercises, about 300 across the book, from routine review to more conceptual questions. The PDF does not print solutions, but the author’s site says solutions are available on request, and the site also offers a separate activities workbook and an interactive Runestone version with online homework sets. Work the exercises, not just the activities, before an exam.
Used In These Programs
This book is used for the Linear Algebra course in: BS Mathematics · BS Computer Science. Browse all Linear Algebra books or all Mathematics books.
Who Should Read This
This book suits a BS Math or BSCS student in a first linear algebra course who learns best by trying examples before meeting definitions. It needs only high-school algebra, since calculus is not a prerequisite, and it moves from systems of equations all the way to singular value decompositions and PCA, which makes it a good fit for students heading toward data science, graphics or machine learning. It is a weaker choice for a course built around rigorous proofs or abstract vector spaces, because the book skips both on purpose; a student in that position should read it for intuition and computation and use Beezer’s A First Course in Linear Algebra for the proofs. Students taking a numerical methods or data science course will also find Chapters 5 to 7 useful.
Applicable Universities
This book is useful for students at Pakistani universities offering BS Math and BSCS including Punjab University, Virtual University, COMSATS, FAST, UET, NUST, and other HEC-recognized institutions.
Official Copy of Understanding Linear Algebra
This book is free from its official source. Click below to open the official Understanding Linear Algebra website, where you can read the book online and download the free PDF.
→ Visit Official PageFAQs
Is Understanding Linear Algebra free?
Yes. The title page states that the work is licensed under the Creative Commons Attribution 4.0 International License, with copyright held by David Austin (2026). You can read it online or download the PDF from the official site at no cost, and the licence lets you share and adapt it as long as you give credit. A low-cost print edition is also sold by 619 Wreath Publishing, but the print copy is optional.
Which edition is this, and is it current in 2026?
This page covers the 2026 Update, dated August 20, 2026 on the title page. The author’s site says the August 2026 changes are mostly maintenance of the source, redrawn interactive diagrams and added figure descriptions, with the content largely unchanged. Linear algebra itself does not date, so the book is current for a 2026 course.
How many chapters and exercises does it have, and are solutions included?
There are 7 chapters plus an appendix on Sage commands, 32 sections in total. Every section starts with a preview activity and has more activities and a set of numbered exercises, about 300 exercises across the book by our count. The PDF does not print solutions; the author’s site says solutions are available on request, and an interactive Runestone version has online homework sets.
Is it good for complete beginners, and do I need calculus first?
It is written for beginners. The author deliberately does not assume calculus, and the expressions studied use only the four basic arithmetic operations. The one place calculus appears is the Hessian discussion in Section 7.2, which you can skim. The book is not an introduction to proofs, so it will feel friendly if you want intuition and computation but incomplete if you need formal proof practice.
Does it cover eigenvalues, SVD and PCA?
Yes. Chapter 4 covers eigenvalues, eigenvectors, diagonalization, dynamical systems and Markov chains with PageRank. Chapter 6 covers orthogonality, Gram-Schmidt, QR factorization and least squares. Chapter 7 covers the Spectral Theorem, quadratic forms, principal component analysis, and singular value decompositions with applications such as image compression and denoising.
Do I need to install Sage to use this book?
No. The online edition has Sage cells you can run in the browser. For the PDF, the book’s own note says print readers can use the online Sage cell server or a page of cells provided by the author. Appendix A lists the Sage commands used in the book, and Section 1.3 introduces them from the start.
Related Books
- A First Course in Linear Algebra – Robert Beezer
- Linear Algebra – Jim Hefferon (4th Edition)
- Mathematics for Machine Learning – Deisenroth, Faisal & Ong
- Calculus Volume 3 – Strang & Herman (OpenStax)
Understanding Linear Algebra is a free, activity-driven first course that takes you from systems of equations to eigenvalues, least squares and singular value decompositions, with Sage cells and real applications along the way. Browse more Mathematics books for the rest of your semester.
Understanding Linear Algebra, 2026 Update, by David Austin, Grand Valley State University. Copyright 2026 David Austin. Free under the Creative Commons Attribution 4.0 International licence (CC BY 4.0). Official source: https://understandinglinearalgebra.org/
© Copyright Policy
Freebooks.pk is an educational resource for students in Pakistan. Textbooks and other books belong to their respective boards, authors and publishers, and all rights remain with them. If you are the copyright holder of any material on this page and want it changed or removed, please contact us or file a request under our DMCA policy. We review every request and remove reported content promptly.
