Neural Networks and Deep Learning – Free Book (BSCS/MS)

BSCS students taking a Neural Networks or Deep Learning elective course, and MS students looking for a rigorous first text, can read the complete book “Neural Networks and Deep Learning” by Michael A. Nielsen, free from its official source, neuralnetworksanddeeplearning.com. It is a 6-chapter book (plus one appendix) that builds a working neural network entirely from first principles — deriving the perceptron and sigmoid neuron, then backpropagation, then regularization and better weight initialization, then convolutional networks — and backs every idea with real, runnable Python code rather than treating any of it as a library call.

What makes this book distinctive among free deep learning texts is how much of it it actually derives rather than asserts: Chapter 2 proves the four fundamental equations of backpropagation from the chain rule, and Chapter 4 gives a fully visual, interactive proof that neural networks are universal function approximators. There is no official PDF or print edition — the author states on the book’s own FAQ page that none exists or is planned, because dozens of interactive JavaScript diagrams throughout the book, especially in Chapters 1 and 4, would not survive a static conversion. The book was first published in 2015 by Determination Press and was last revised on 26 December 2019 per its own page footer; its concepts remain foundational reading for deep learning even though its own example code (Python 2.7, and the discontinued Theano library in Chapter 6) reflects 2015-era tooling.

Book Overview

Chapter List

Chapter 1: Using Neural Nets to Recognize Handwritten Digits

Difficulty: Medium · Key topics: perceptrons, sigmoid neurons, feedforward architecture, MNIST dataset, quadratic cost function, stochastic gradient descent, hidden layers

Nielsen opens with the perceptron, an artificial neuron that fires 0 or 1 based on a weighted sum of inputs against a threshold, and shows perceptrons are computationally universal because they can implement NAND gates. He then introduces the sigmoid neuron, which smooths the perceptron’s step function into σ(w·x+b) so that small weight changes produce small output changes — the property that makes gradient-based learning possible. The chapter designs a three-layer, 784-input network for the MNIST handwriting dataset, defines the quadratic cost function, and derives stochastic gradient descent as the way to minimize it using small mini-batches instead of the full training set. It closes by walking through a complete 74-line Python implementation (network.py) that reaches over 96 percent test accuracy with no external neural-network libraries.

Key Points:

• A perceptron outputs 0 or 1 based on whether w·x+b is ≤0 or >0; the bias b = −threshold.
• Networks of perceptrons are computationally universal, because a perceptron can implement a NAND gate.
• A sigmoid neuron outputs σ(w·x+b) = 1/(1+e^−z); its smoothness makes a small Δoutput a near-linear function of Δw and Δb.
• The example network uses 784 input neurons (28×28 MNIST pixels), a hidden layer of n neurons, and 10 output neurons, one per digit.
• The quadratic cost function is C(w,b) = (1/2n)Σ||y(x)−a||².
• Stochastic gradient descent estimates the true gradient from small mini-batches instead of all 50,000 training images at every step, which is what makes training practical.
• The complete network.py implementation is only 74 lines and reaches over 96% accuracy on MNIST with no dedicated neural-network library.

Practice Tip: Clone the book’s own repository (github.com/mnielsen/neural-networks-and-deep-learning) and actually run network.py with different values of the hidden-layer size n and the learning rate η — watching accuracy change in real time teaches more than reading the chapter twice. Note the code is written in Python 2 syntax (xrange, the print statement); it needs a few small edits to run under current Python 3.

Common Mistake: Assuming a bigger hidden layer always means a better network; in the chapter’s own experiments, changing n from 15 to 30 barely moves accuracy compared to correctly tuning the learning rate η, and picking η too large makes the cost function diverge instead of decrease.

Important Questions:

• Why does a network of sigmoid neurons learn well where a network of pure perceptrons does not? Because a perceptron’s output can flip completely from a tiny weight change, while a sigmoid neuron’s output changes smoothly and near-linearly, letting small weight adjustments produce small, controllable improvements in the cost.
• Why does the MNIST network use 10 output neurons instead of 4 (enough bits to encode digits 0–9)? Empirically the 10-neuron encoding trains better; Nielsen’s heuristic is that each output neuron can correspond to evidence from simple component shapes detected by the hidden layer, which has no natural relationship to a single output bit.

Chapter 2: How the Backpropagation Algorithm Works

Difficulty: Hard · Key topics: backpropagation, error term δ, four fundamental equations (BP1–BP4), Hadamard product, weighted input z, chain rule

This chapter fills the gap left in Chapter 1 by deriving how to actually compute ∂C/∂w and ∂C/∂b for every weight and bias in the network. Nielsen defines the error δ of a neuron as ∂C/∂z, then derives the four fundamental backpropagation equations (BP1–BP4): one for the output layer’s error, one for propagating that error backward through the transposed weight matrix, and two connecting the error directly to the bias and weight gradients. He proves all four from the chain rule, restates them as a concrete algorithm, and walks through the matching numpy code (update_mini_batch and backprop) line by line. The chapter closes by explaining why backpropagation is fast — one forward pass plus one backward pass computes every gradient at once, instead of well over a million forward passes for the naive finite-difference approach — and gives an intuitive “demon in the network” picture of what the algorithm is really doing.

Key Points:

• The error of neuron j in layer l is defined as δˡ₀ ≡ ∂C/∂zˡ₀.
• BP1: δᶫ = ∇⃄C ⊙ σ'(zᶫ) gives the error in the output layer.
• BP2: δˡ = ((wˡ⁺¹)ᵀδˡ⁺¹) ⊙ σ'(zˡ) propagates the error backward one layer at a time.
• BP3 and BP4: ∂C/∂b = δ and ∂C/∂w = a_in·δ_out — the bias gradient equals the error, and the weight gradient is the input activation times the output error.
• ⊙ denotes the Hadamard (elementwise) product, not ordinary matrix multiplication.
• A weight learns slowly if its input neuron has low activation, or if its output neuron has saturated (activation near 0 or 1).
• Backpropagation costs about the same as two forward passes total, versus well over a million forward passes for the naive finite-difference method on a million-weight network.

Practice Tip: Before reading the backprop code, hand-derive BP3 and BP4 from the chain rule yourself, exactly as the chapter’s own exercise asks — it takes about ten minutes and makes the backprop() function (already seen in Chapter 1) read like something you wrote, not something you’re decoding.

Common Mistake: Mixing up the Hadamard product ⊙ (elementwise, used in BP1/BP2) with ordinary matrix multiplication is the single most common error when implementing backpropagation by hand from these equations — check that δˡ always has the same shape as zˡ, never a different one.

Important Questions:

• What does BP4 tell you about weights whose input neuron has an activation near zero? Such weights learn very slowly, because the gradient is directly proportional to the input activation; a near-zero input means a near-zero gradient regardless of the error.
• In what sense is backpropagation a “fast” algorithm? It computes the gradient with respect to every weight and bias using just one forward pass and one backward pass, instead of well over a million forward passes the naive finite-difference calculation would need for a million-weight network.

Chapter 3: Improving the Way Neural Networks Learn

Difficulty: Medium · Key topics: cross-entropy cost, softmax, overfitting, L1/L2 regularization, dropout, weight initialization, hyperparameters

Nielsen diagnoses why the quadratic cost function learns slowly when a neuron is badly wrong (because σ'(z) is tiny when a neuron saturates), and fixes it with the cross-entropy cost function, whose gradient removes the σ'(z) term so learning speed tracks the size of the error directly. He introduces the softmax output layer paired with the log-likelihood cost as an alternative that guarantees the output activations sum to 1, useful for treating them as probabilities. The chapter then tackles overfitting: it shows a network memorizing noise in a small training set, and introduces L2 and L1 weight-decay regularization, dropout, and artificially expanding the training data (via rotated/translated MNIST digits) as four separate cures, plus a smarter Gaussian weight-initialization scheme that avoids early saturation. It closes with a rewritten network2.py and a discussion of how to choose learning rate, mini-batch size, regularization parameter and number of epochs in practice.

Key Points:

• The cross-entropy cost C = −(1/n)Σ[y ln a + (1−y) ln(1−a)] avoids the learning slowdown caused by σ'(z) in the quadratic cost’s gradient.
• A softmax output layer normalizes its activations to sum to 1, letting them be read as a probability distribution over the 10 digit classes.
• L2 regularization adds (λ/2n)Σw² to the cost, shrinking weights and biasing the network toward simpler explanations that generalize better.
• Dropout randomly deletes about half the hidden neurons on each mini-batch during training, forcing the network to learn more robust features instead of relying on any single neuron.
• Expanding the training data (e.g. small rotations of MNIST digits) is often more effective at reducing overfitting than any single regularization technique — one cited experiment reached 99.3% accuracy this way.
• Good weight initialization uses a Gaussian with standard deviation 1/√n_in instead of standard deviation 1, which stops hidden neurons saturating right from the start of training.
• Validation data, not test data, should be used to tune hyperparameters like learning rate and regularization strength, so the test accuracy stays an honest final measure.

Practice Tip: Run network2.py with dropout switched on and off on the same small training set and watch the gap between training accuracy and validation accuracy shrink — that gap, not the raw training accuracy, is what tells you whether you’re overfitting.

Common Mistake: Treating a high training accuracy as good news by itself; a network that memorizes 100% of its training data while validation accuracy stalls is overfitting, not learning, and needs regularization or more data, not more epochs.

Important Questions:

• Why does the quadratic cost function cause a neuron to “learn slowly” when it is badly wrong? Its gradient contains a σ'(z) factor, and σ'(z) is nearly zero when the neuron is saturated near 0 or 1 — exactly the situation of a confidently wrong neuron.
• Why is expanding the training data by rotating or translating MNIST digits an effective regularizer? It exposes the network to realistic variation the original 50,000-image training set didn’t contain, so the network learns features that generalize beyond the exact training examples, directly reducing overfitting.

Chapter 4: A Visual Proof That Neural Nets Can Compute Any Function

Difficulty: Medium · Key topics: universality theorem, step functions, bump functions, tower functions, continuous function approximation, single hidden layer

This self-contained chapter proves, almost entirely with pictures instead of algebra, that a neural network with even a single hidden layer can approximate any continuous function to any desired accuracy. Nielsen shows that cranking up a sigmoid neuron’s input weight turns its output into a near-perfect step function whose position is controlled by the bias, then glues pairs of these step neurons together into “bump” functions of any height and width. Stacking enough bumps side by side builds an arbitrary one-dimensional function as a kind of lookup table; the same trick, extended to a second hidden layer, builds “tower” functions over two or more input variables, generalizing the construction to functions of many inputs and outputs. The chapter closes with two caveats — the approximation is never exact, only within any desired accuracy, and only continuous functions are covered — and previews why, despite this universality result for shallow networks, deep networks are still worth using.

Key Points:

• The universality theorem: a network with a single hidden layer can approximate any continuous function to within any desired accuracy ε, given enough hidden neurons.
• Cranking a sigmoid neuron’s weight toward a large value (e.g. w≈1000) turns σ(wx+b) into a near-perfect step function whose position is s = −b/w.
• Two step-function neurons with opposite-sign output weights (+h and −h) combine into a “bump” function of height h.
• Many bumps side by side let the network approximate the target function as a piecewise lookup table.
• Only continuous functions can be approximated this way; functions with sharp discontinuities are not covered by the theorem.
• Universality only proves that a suitable network exists — it says nothing about whether a learning algorithm like gradient descent can actually find it.

Memory Tip: Picture the construction in three steps — “a big weight makes a step, two opposite steps make a bump, many bumps make a lookup table” — and the whole chapter’s argument compresses into one sentence you can reconstruct the diagrams from.

Common Mistake: Concluding from this chapter that any function a network can represent is a function it can learn; universality is only an existence proof — Chapter 5 shows deep networks can fail badly to find those weights via gradient descent, even though a solution provably exists.

Important Questions:

• What two caveats does Nielsen attach to the claim that “a neural network can compute any function”? First, the network only approximates the function to within a chosen accuracy ε, never computing it exactly; second, only continuous functions can be approximated this way, not functions with sharp discontinuities.
• Why does the book still use deep networks later if a single hidden layer is already provably universal? Because universality only says a suitable shallow network exists, not that it is easy to design or learn; deep networks build up hierarchies of abstraction (edges → shapes → objects) that make learning practical, real-world functions far more tractable.

Chapter 5: Why Are Deep Neural Networks Hard to Train?

Difficulty: Hard · Key topics: vanishing gradient problem, exploding gradient problem, unstable gradient, layer-wise learning speed, σ'(z), weight initialization

Nielsen trains networks on MNIST with one, two, three and four hidden layers using otherwise identical hyperparameters, and shows accuracy stalls or even drops as more layers are added — extra depth is not paying off. Measuring the gradient’s length layer by layer reveals why: in a four-hidden-layer network the first hidden layer was measured learning roughly 100 times slower than the last, a pattern he names the vanishing gradient problem. Using a toy one-neuron-per-layer network he derives an explicit formula for ∂C/∂b₁ as a product of σ'(zᴡ)wᴡ terms, one pair per layer, and shows that because σ'(z) never exceeds ¼ and initial weights are typically |w|<1, this product shrinks exponentially with depth — while a different weight initialization can just as easily make it explode instead. The chapter concludes that this instability, not any one bug, is the fundamental reason plain gradient descent struggles to train deep networks, setting up Chapter 6’s fixes.

Key Points:

• Adding hidden layers to a plain sigmoid network on MNIST does not reliably improve accuracy, and can make it worse, despite deeper networks being provably more expressive.
• In a four-hidden-layer network, the first hidden layer was measured learning roughly 100 times slower than the last hidden layer.
• The gradient for an early bias is a product of terms wᴡσ'(zᴡ), one for every later layer, plus a final ∂C/∂a term.
• σ'(z) never exceeds ¼ (its maximum, at z=0), so with typical initial weights |w|<1 each term is usually well under 1, and the product shrinks exponentially with depth — the vanishing gradient problem.
• If weights are instead initialized large enough that |w·σ'(z)|>1, the same product grows exponentially instead — the exploding gradient problem.
• The underlying issue is an unstable gradient: without some mechanism to make the many per-layer terms balance out, they will not do so by chance, so different layers learn at very different speeds.

Memory Tip: Remember the chain as “each layer you go back multiplies in one more w·σ'(z) term, and σ'(z) tops out at ¼” — that single fact is the entire mechanical cause of vanishing gradients in a sigmoid network.

Common Mistake: Assuming a deep network that trains slower than a shallow one simply “needs more epochs” — the chapter’s own measurements show the early layers are barely moving at all, not moving slowly toward a good answer, so more training time alone does not fix an unstable gradient.

Important Questions:

• Why does σ'(z) having a maximum value of ¼ lead directly to the vanishing gradient problem? The gradient for an early layer is a product of many w·σ'(z) terms (one per later layer); with typical weights each such term is usually well under 1, and multiplying many small terms together shrinks the product exponentially as depth increases.
• What is the “exploding gradient problem” and how does it relate to vanishing gradients? It is the opposite failure mode, where the same product of w·σ'(z) terms grows exponentially instead of shrinking, if weights are initialized large enough; both are symptoms of the same underlying unstable-gradient problem in deep, gradient-trained networks.

Chapter 6: Deep Learning

Difficulty: Hard · Key topics: convolutional networks, local receptive fields, shared weights, pooling, ReLU, dropout ensembles, RNNs, LSTM, deep belief nets

The book’s longest chapter builds a convolutional neural network for MNIST from scratch, introducing local receptive fields, shared weights and biases (a “feature map”), and pooling layers that shrink and condense the convolutional output. Implemented in network3.py using the Theano library, the network climbs step by step from a 97.80% baseline fully-connected network to 98.78% with one convolutional-pooling layer, 99.06% with two, 99.23% after switching to rectified linear units, and 99.37–99.6%+ once the training data is algorithmically expanded and multiple networks are combined into a voting ensemble. Nielsen then surveys the wider deep learning landscape at a conceptual level — recurrent neural networks and LSTMs for sequence data like speech and text, and deep belief networks and Boltzmann machines as generative, unsupervised alternatives — before closing with speculation on intention-driven interfaces and the future role of deep learning. Note: network3.py depends on Theano, a library unmaintained since 2017 (see Practice Tip).

Key Points:

• A convolutional layer uses a small local receptive field (e.g. 5×5) slid across the image, with the same shared weights and bias at every position, forming one “feature map”.
• Pooling layers (e.g. 2×2 max-pooling) condense a feature map, reducing the number of parameters in later layers and building in some translation invariance.
• Switching from sigmoid neurons to rectified linear units (f(z)=max(0,z)) consistently outperformed sigmoids in the chapter’s own experiments, reaching 99.23% versus 99.06%.
• Algorithmically expanding the MNIST training data (e.g. by pixel translations) pushed accuracy from 99.23% to 99.37%, without changing the network architecture at all.
• The final MNIST system in the chapter misclassifies only 33 of the 10,000 test images, several of which are genuinely ambiguous even to a human reader.
• Recurrent neural networks (RNNs) add time-varying, feedback connections for sequence data; LSTMs (Hochreiter & Schmidhuber, 1997) were introduced specifically to fix the vanishing-gradient problem inside RNNs.
• Deep belief networks are generative models that can run “backward” to generate new images resembling their training data, and support unsupervised learning from unlabeled data.

Practice Tip: The chapter’s own code (network3.py) depends on Theano, a library that has been unmaintained since 2017 — treat this chapter as a conceptual and historical guide to convolutions, pooling and ReLU, and reproduce the actual experiments in a currently maintained framework such as PyTorch or TensorFlow/Keras, whose convolution and max-pooling layers implement exactly the ideas described here.

Common Mistake: Assuming a convolutional layer has one weight per pixel like a fully-connected layer; the entire point of “shared weights” is that a single small filter (e.g. 5×5 = 25 weights, plus one bias) is reused at every position in the image, which is what keeps the parameter count low and builds in translation invariance.

Important Questions:

• What is the practical difference between a convolutional-pooling layer and a fully-connected layer? A convolutional layer uses one small set of shared weights (a local receptive field) applied at every position across the image to detect a feature anywhere it appears, while a fully-connected layer gives every output neuron its own independent weight for every input, with no such position-sharing.
• Why do the chapter’s accuracy-improving techniques (ReLU, expanded data, extra layers, dropout, ensembles) stack rather than compete? Each technique attacks a different specific problem — activation saturation, limited training data, network capacity, overfitting, and prediction variance respectively — so combining them compounds their separate gains, taking the network from 97.80% to over 99.6% accuracy step by step.

Appendix: Is There a Simple Algorithm for Intelligence?

Appendix · Difficulty: Easy · Key topics: connectomics, human-chimp genetic difference, ferret rewiring experiment, Society of Mind, generative models, unsupervised learning

In this closing, largely non-technical essay, Nielsen asks whether human-level intelligence could ultimately rest on a small set of simple underlying principles, the way planetary motion, chemistry and evolution turned out to. He estimates that only about 125 million bits of genetic information separate human and chimpanzee brains — equivalent to roughly 25 million English words, or about 30 King James Bibles — a strikingly small number next to the roughly 70 quadrillion bits needed to describe the brain’s 100 trillion neural connections directly. As evidence for shared underlying learning principles he cites the well-known 2000 Sur-lab experiment, in which ferret brains rewired so the optic nerve fed the auditory cortex grew orientation columns there similar to a normal visual cortex, letting the ferrets learn a rudimentary kind of “sight” through their hearing circuitry. He weighs this against Marvin Minsky’s “Society of Mind” argument that many distinct brain regions imply irreducible complexity, and lands, optimistically but explicitly unproven, on the view that acting as though a simple algorithm for intelligence exists is itself the most productive research strategy.

Key Points:

• Human and chimpanzee genomes differ at roughly 125 million DNA base pairs out of about 3 billion; Nielsen estimates about half of that (125 million bits) plausibly relates to brain differences.
• 125 million bits is roughly equivalent to 25 million English words, or about 30 copies of the King James Bible — large, but graspable, next to the roughly 70 quadrillion bits needed to describe the brain’s neurons and connections directly.
• In a 2000 Nature study, Mriganka Sur’s team rewired newborn ferrets’ optic nerves to the auditory cortex, which then grew orientation columns resembling those of the visual cortex, and the ferrets showed rudimentary visual behaviour using that region.
• Marvin Minsky’s “Society of Mind” theory argues intelligence emerges from a large society of many individually simple but architecturally distinct “agents,” implying no single simple principle explains it.
• Nielsen takes no firm position on whether a truly simple algorithm for intelligence exists, but argues that optimistically acting as if it does is historically a more productive research strategy.

Important Questions:

• What evidence does Nielsen cite for different brain regions sharing common learning principles? The 2000 ferret-rewiring experiment, in which optic-nerve signals rerouted to the auditory cortex caused that region to grow orientation columns similar to the visual cortex, with the ferrets showing some ability to respond to visual stimuli using it.
• How does the human-chimpanzee genetic comparison support the idea that the brain’s architecture is simpler than its raw wiring suggests? The genetic difference between human and chimp brains, about 125 million bits (roughly 25 million words), is about nine orders of magnitude smaller than the ~70 quadrillion bits needed to describe the brain’s 100 trillion connections directly, suggesting the genome specifies broad architecture and principles rather than every connection.

Read Neural Networks and Deep Learning Online (Free)

This book has no PDF or print edition. The author states on the book’s own FAQ page that none exists or is planned, because the book relies on dozens of interactive JavaScript diagrams — used heavily in Chapter 1 and Chapter 4 — that would not survive conversion to a static PDF. Click below to start reading at its official source, neuralnetworksanddeeplearning.com.

How to Study This Book

Read Chapters 1 and 2 first, and in order. Chapter 1 builds the first working network (network.py) using perceptrons, sigmoid neurons and stochastic gradient descent, and Chapter 2 is its direct sequel: it derives backpropagation, the gradient-computing algorithm that Chapter 1’s own training code already depends on but never explains. Skipping Chapter 2 leaves a genuine gap in understanding, not just missing depth.

Chapter 3 is the natural next stop for anyone about to build a real project on top of Chapter 1’s network. It rewrites the network as network2.py with the cross-entropy cost, regularization, and better weight initialization, and is where the book actually earns its title, “Improving the way neural networks learn.”

Chapter 4 is explicitly written by Nielsen as a self-contained essay — you do not need Chapters 1–3 to follow it, and it can be read at any point, even first, if you prefer intuition before formulas. Its interactive sliders (best experienced in a real browser, not a printout) make the universality theorem visual rather than abstract, so read it on a laptop or tablet, not from notes copied out by hand.

Chapters 5 and 6 form a matched pair: Chapter 5 shows, with real measured numbers, why plain gradient descent fails to train deep sigmoid networks (the vanishing gradient problem), and Chapter 6 is the direct answer — convolutional networks, ReLU, expanded training data and ensembles, which is where the book’s own MNIST accuracy climbs from 96% to over 99.6%. Read them back to back; Chapter 6 makes far less sense without Chapter 5’s diagnosis first.

One caveat for anyone following along with actual code: network.py and network2.py (Chapters 1 and 3) use plain Python and numpy and are still easy to run today with light Python-3 edits. network3.py (Chapter 6) depends on Theano, a library that has been unmaintained since 2017. Treat Chapter 6’s code listings as a historical, conceptual guide to convolutions and pooling, not a drop-in script — reimplement the same ideas in PyTorch or TensorFlow/Keras if you want to actually run them.

The Appendix is genuinely optional. It has no code and no exercises, and is worth reading once you finish Chapter 6 out of curiosity, but it is entirely skippable if your course only covers Chapters 1–6.

For a one-semester Neural Networks or Deep Learning elective, Chapters 1, 2, 3 and 6 cover the practical core — build a network, train it correctly, then make it deep — while Chapters 4 and 5 are the “why it works, why it’s hard” chapters many courses assign as reading rather than as graded assignments.

Used In These Programs

This book is used for the Neural Networks / Deep Learning course in: BS Computer Science. It is also commonly assigned as first reading in graduate (MS) Deep Learning and Introduction to AI electives, where BS Software Engineering (BSSE) students taking a machine learning track will find it equally useful. Browse all Deep Learning books or all Computer Science books.

Who Should Read This

Neural Networks and Deep Learning is written for a reader who already has working knowledge of Python and enough calculus to be comfortable with partial derivatives and the chain rule — Nielsen assumes this from Chapter 1 onward and does not re-teach it. That makes it a strong fit for a junior- or senior-year BSCS Neural Networks or Deep Learning elective, or as first reading before a graduate (MS) deep learning course, but a weak starting point for a complete beginner to programming: readers who find Chapter 1’s network.py code hard to follow should first work through a Python fundamentals text such as Think Python, then come back. It is also not a broad survey of modern deep learning architectures — Transformers, diffusion models and large language models postdate the book entirely, and even its convolutional-network chapter is written from 2015-era practice, with Chapter 6’s own code using Theano, since superseded by PyTorch and TensorFlow. Readers who need current architecture coverage alongside these foundations are best served by pairing this book with a more recently maintained deep learning text once one is available on this site.

Applicable Universities

This book is useful for students at Pakistani universities offering BSCS, BSIT, BSSE and MS Computer Science programs with a Neural Networks, Deep Learning or Introduction to AI course, including Punjab University, Virtual University, COMSATS, FAST, UET, NUST, and other HEC-recognized institutions.

FAQs

Related Books

• Think Python – Allen B. Downey (this book’s own code, network.py through network3.py, is written from scratch in Python; Think Python is the natural prerequisite if that code is hard to follow)
• Calculus Volume 1 – Strang & Herman (OpenStax) (the derivatives, gradients, and chain rule behind backpropagation in Chapter 2, covered from first principles)
• Discrete Mathematics: An Open Introduction – Oscar Levin (probability and mathematical foundations useful for the statistical reasoning behind cost functions and regularization)
• Problem Solving with Algorithms and Data Structures using Python (general Python programming and data-structures grounding for readers who find Chapter 1’s from-scratch numpy code difficult)

Neural Networks and Deep Learning is a free, rigorously-written introduction that builds a working neural network from first principles across just six chapters and one appendix, deriving backpropagation, regularization, and convolutional networks along the way rather than treating them as library calls. Browse more Computer Science books for the rest of your semester.

Neural Networks and Deep Learning, by Michael A. Nielsen, Determination Press, 2015. Licensed under a Creative Commons Attribution-NonCommercial 3.0 Unported License (CC BY-NC 3.0). Free to read online at http://neuralnetworksanddeeplearning.com/ — no PDF or print edition exists.