Think Stats: Exploratory Data Analysis in Python

BSCS, BS Statistics, and BS Data Science students taking a Statistics for Programmers or Exploratory Data Analysis course can download the complete textbook “Think Stats: Exploratory Data Analysis in Python” by Allen B. Downey, free from its official source, Green Tea Press. It’s a 14-chapter book that teaches statistics the way a programmer would learn it — by writing and running Python code on a real dataset, rather than memorizing formulas from a traditional textbook.

The book works through a single real, messy survey dataset (the U.S. National Survey of Family Growth) from Chapter 1 all the way through advanced topics like regression, time series, and survival analysis — making it a practical companion for anyone who already knows some Python and wants to build real statistical intuition through code rather than derivation.

Book Overview

Chapter List

Chapter 1: Exploratory Data Analysis

Difficulty: Beginner · Statistics for Programmers — Week 1 · Key topics: Reading real datasets, the NSFG case study, records and cycles, using Python for statistics instead of formulas

This opening chapter sets the book’s approach: rather than teaching statistics as a sequence of formulas and hypothesis-testing recipes, it works through a real dataset (the U.S. National Survey of Family Growth, or NSFG) end to end using Python. It covers reading a real, messy dataset into code, understanding what a survey’s records and codebook actually represent, and framing exploratory data analysis as writing small programs to answer specific questions about the data, rather than a fixed statistical procedure.

Key Points:

• This book’s core philosophy is “compute, don’t derive” — concepts are taught by writing short Python programs on real data, not by memorizing closed-form formulas.
• The NSFG (National Survey of Family Growth) dataset used throughout the book is a real, complex survey with a codebook — understanding what each column actually measures is a prerequisite to analyzing it correctly.
• A statistical variable’s values only mean something in light of how it was collected — the same raw column can represent different things depending on the survey’s sampling and coding scheme.
• Exploratory data analysis (EDA) is the practice of investigating data to find patterns, check assumptions, and generate hypotheses, before jumping to formal hypothesis tests.
• Working with real, imperfect data (missing values, coding quirks, weighting) from the very first chapter is a deliberate choice — textbook “clean” data hides problems every real analysis has to handle.

Practice Tip: Actually download the NSFG data and run the book’s example code in Chapter 1 rather than just reading about it — this book is built around a philosophy of learning statistics by writing and running programs, and skipping the hands-on part undermines that approach from the start.

Common Mistake: Treating a survey dataset’s raw values as self-explanatory without consulting its codebook — a numeric code like 6 in a survey column might represent “refused to answer” rather than a real measurement, and silently including such codes in an analysis produces meaningless results.

Important Questions:

• Q1: What is the core philosophical difference between this book’s approach and a traditional statistics textbook? This book emphasizes computation over mathematical derivation — concepts are taught and explored by writing and running short Python programs on real datasets, rather than by proving formulas and plugging numbers into them by hand. The goal is building working intuition through code, not memorizing closed-form equations.
• Q2: Why does this book insist on using a real, messy dataset (the NSFG) from Chapter 1, rather than clean synthetic data? Real datasets have missing values, coding quirks, and sampling complexities that every genuine data analysis must handle. Starting with clean, artificial data would hide these issues and leave a reader unprepared for the realities of working with actual data outside a textbook.

Chapter 2: Distributions

Difficulty: Beginner · Statistics for Programmers — Week 2 · Key topics: Histograms, representing and summarizing a distribution of values

This chapter introduces the histogram as the fundamental tool for representing a distribution — how often each value (or range of values) occurs in a dataset. It covers building histograms from real data, comparing histograms across different groups (such as first babies versus others in the NSFG data), and the basic summary statistics (mean, variance, and their relationship to a histogram’s shape) that describe a distribution numerically.

Key Points:

• A histogram counts how many times each value (or bin of values) occurs in a dataset — it’s the most direct visual representation of a distribution.
• Comparing histograms between two groups (e.g. two survey subpopulations) is often more informative than comparing single summary numbers like the mean alone.
• The mean summarizes a distribution’s central tendency but can be misleading alone — two distributions can share a mean while looking completely different in shape.
• Variance measures how spread out a distribution’s values are around the mean; a larger variance means values are more dispersed.
• Outliers and unusual values often show up clearly in a histogram’s shape before any summary statistic reveals them.

Memory Tip: Before computing any summary statistic (mean, variance) on a new dataset, always plot its histogram first — a quick visual check catches obviously wrong data (like an impossible age or an unremoved missing-value code) that a summary number alone would hide.

Common Mistake: Comparing two datasets using only their means without checking their distributions’ shapes — two groups can have nearly identical means while having very different variances or shapes (one tightly clustered, one widely spread, or one skewed), a difference invisible from the mean alone.

Important Questions:

• Q1: What does a histogram show that a single summary statistic like the mean does not? A histogram shows the full shape of a distribution — how values are spread out, whether there are multiple peaks (modes), how skewed it is, and where outliers lie. A single number like the mean collapses all of this shape information into one value, potentially hiding important differences between datasets that share the same mean.
• Q2: Why might comparing histograms of a variable across two different groups reveal more than comparing their means alone? Two groups can have very similar means while differing substantially in variance, skew, or shape — a histogram comparison would reveal these shape differences directly, while a comparison of means alone would suggest the two groups are more similar than they actually are.

Chapter 3: Probability Mass Functions

Difficulty: Beginner · Statistics for Programmers — Week 2–3 · Key topics: PMFs, normalizing histograms, comparing distributions of different sizes, the class-size paradox

This chapter introduces the Probability Mass Function (PMF) — a histogram normalized so its values sum to 1, representing each outcome’s probability rather than its raw count. Normalizing this way makes it possible to fairly compare distributions built from datasets of different sizes. The chapter also covers the “class size paradox,” a classic example of how the way a quantity is sampled (per-student versus per-class) can produce two genuinely different, both correct, distributions from the same underlying data.

Key Points:

• A PMF is a histogram divided by the total count, so its values represent probabilities and sum to exactly 1.
• Normalizing to a PMF is essential when comparing distributions built from datasets of different total sizes — raw histogram counts aren’t directly comparable across different sample sizes.
• The class-size paradox: the average class size experienced by a randomly chosen student is larger than the actual average class size, because larger classes contain more students and are therefore over-represented from a student’s perspective.
• This kind of sampling bias — where the unit of observation (student vs. class) changes what “average” means — appears constantly in real data analysis, not just as a textbook curiosity.
• A PMF can be built directly from a dataset’s histogram in Python with a simple normalization step, reusing exactly the same underlying data structure.

Memory Tip: When a paradox like the class-size example feels counterintuitive, write the small Python simulation yourself rather than just reading the explanation — seeing the biased sampling reproduce the paradox in code makes the underlying mechanism concrete in a way the text alone often doesn’t.

Common Mistake: Directly comparing two raw histograms (unnormalized counts) from datasets of very different sizes and drawing conclusions about which distribution has “more” of some value — without normalizing to a PMF first, differences in total sample size can be mistaken for genuine differences in distribution shape.

Important Questions:

• Q1: What is a PMF, and how is it different from a histogram? A Probability Mass Function (PMF) is a histogram where each count has been divided by the total number of observations, so the resulting values represent probabilities and sum to exactly 1. A plain histogram shows raw counts, which are not directly comparable between datasets of different sizes; a PMF is.
• Q2: Explain, in one or two sentences, why the class-size paradox occurs. Larger classes contain more students, so when you ask a randomly chosen student what size their class is, students in larger classes are more likely to be the one you asked. This biases the student-perspective average class size upward compared to the simple average of all classes’ sizes, even though both numbers are computed correctly from the same underlying data.

Chapter 4: Cumulative Distribution Functions

Difficulty: Beginner · Statistics for Programmers — Week 3 · Key topics: CDFs, percentiles, comparing distributions, why CDFs are often better than PMFs/histograms for comparison

This chapter introduces the Cumulative Distribution Function (CDF), which for a given value x gives the probability that a random observation from the distribution is less than or equal to x. Unlike a histogram or PMF, a CDF has no binning choices to make and handles both discrete and continuous data uniformly, which the book argues makes it a better default tool for comparing distributions. The chapter also connects CDFs to percentiles and percentile ranks, a familiar everyday concept (like a test score’s percentile).

Key Points:

• A CDF at value x gives the probability that a randomly chosen observation is less than or equal to x — it’s a non-decreasing function that rises from 0 to 1.
• Unlike a histogram, a CDF requires no arbitrary choice of bin width, making it a more faithful, unambiguous representation of the underlying data.
• A percentile rank is just the CDF value at a given point, expressed as a percentage — e.g. scoring in the 90th percentile means your value’s CDF is 0.9.
• CDFs are often better than PMFs for visually comparing two distributions, since overlapping PMFs (especially with many bins) can be visually cluttered while CDFs overlay cleanly.
• The median of a distribution is simply the value where its CDF equals 0.5 — the 50th percentile.

Memory Tip: When comparing two distributions visually, plot their CDFs side by side before reaching for a formal statistical test — a visible horizontal gap between two CDF curves is often a clearer, more intuitive signal of a real difference than a p-value alone.

Common Mistake: Comparing two distributions’ histograms with different, arbitrarily chosen bin widths and drawing conclusions about their shapes — the choice of bin width can visually exaggerate or hide real differences, a problem CDFs avoid entirely since they have no binning parameter at all.

Important Questions:

• Q1: What does CDF(x) = 0.75 mean about a distribution? It means 75% of the observations in the distribution are less than or equal to x — equivalently, x is the distribution’s 75th percentile.
• Q2: Why does this book generally prefer CDFs over PMFs/histograms when comparing two distributions? A CDF requires no arbitrary choice of bin width or bin count, unlike a histogram or PMF, so it represents the data without any binning artifacts. It also tends to be visually cleaner when overlaying multiple distributions for comparison, since CDF curves don’t suffer the visual clutter that overlapping histogram bars often do.

Chapter 5: Modeling Distributions

Difficulty: Intermediate · Statistics for Programmers — Week 4 · Key topics: The exponential and normal distributions, fitting analytic models to empirical data, the normal probability plot

This chapter moves from purely empirical distributions (built directly from data) to analytic models — mathematical distributions like the exponential and normal (Gaussian) distributions — and how to check whether a real dataset is well approximated by one of these models. It covers plotting a dataset’s CDF against a model’s theoretical CDF, and the normal probability plot as a specific, widely-used visual technique for checking whether data is approximately normally distributed.

Key Points:

• An analytic distribution (like the exponential or normal distribution) is defined by a mathematical formula with a small number of parameters, unlike an empirical distribution built directly from raw data.
• The exponential distribution often models the time between independent, randomly occurring events (like customer arrivals), and has a characteristic straight line when its CDF is plotted on a log scale.
• The normal (Gaussian) distribution is defined by its mean and standard deviation and produces the familiar symmetric, bell-shaped curve.
• A normal probability plot compares a dataset’s sorted values against the values expected from a perfect normal distribution — a straight line indicates a good fit, and curvature indicates departure from normality.
• Fitting a model to data is useful for summarizing a distribution compactly and for extrapolating beyond the observed data, but should always be checked against the actual data rather than assumed.

Memory Tip: Never assume a dataset is normally distributed just because it’s roughly bell-shaped by eye — always check with a normal probability plot before applying a technique (like certain hypothesis tests) that assumes normality, since real data frequently has heavier tails or skew that a quick visual glance misses.

Common Mistake: Fitting a normal distribution to data that is actually skewed or has a different underlying shape (like exponential or lognormal) just because it’s a convenient, well-known default — this can produce a model that fits poorly in the tails, exactly where accurate predictions often matter most.

Important Questions:

• Q1: What is the purpose of a normal probability plot? It’s a diagnostic tool for checking whether a dataset is approximately normally distributed, by plotting the data’s sorted values against the values a perfect normal distribution would produce. If the data is approximately normal, the points fall close to a straight line; systematic curvature indicates the data departs from normality.
• Q2: What kind of real-world process does the exponential distribution typically model, and what does that imply about its shape? It typically models the time between independent, randomly occurring events, such as the gap between customer arrivals or equipment failures. This implies a distribution with most values clustered near zero and a long right tail — short gaps are common, long gaps are rare but possible.

Chapter 6: Probability Density Functions

Difficulty: Intermediate · Statistics for Programmers — Week 5 · Key topics: PDFs, kernel density estimation, the relationship between PMF, CDF, and PDF for continuous data

This chapter covers the Probability Density Function (PDF), the continuous-data counterpart to the PMF — rather than assigning probability directly to individual values (which have probability zero for continuous data), a PDF describes relative likelihood as a smooth curve, where probability corresponds to the area under the curve over an interval. It introduces kernel density estimation (KDE) as a practical technique for estimating a smooth PDF from a finite sample of data points, and ties together how PMF, CDF, and PDF relate to each other.

Key Points:

• For continuous data, a specific exact value has probability zero — a PDF describes relative likelihood as a density, with actual probability corresponding to the area under the curve over a range.
• A PDF is mathematically the derivative of a CDF; conversely, integrating a PDF over an interval gives the probability of falling in that interval.
• Kernel density estimation (KDE) builds a smooth estimated PDF from a finite sample by placing a small smooth bump (kernel) at each data point and summing them, rather than relying on discrete histogram bins.
• The bandwidth (smoothing parameter) chosen for a KDE trades off between an overly noisy, spiky estimate (too little smoothing) and an overly smoothed, detail-losing estimate (too much smoothing).
• PMF, CDF, and PDF are three different but mathematically connected views of the same underlying distribution, each more useful in different situations.

Memory Tip: Think of a PDF’s height at a point as relative likelihood, never as a probability itself — the actual probability of landing in a small interval is (roughly) the PDF’s height at that point multiplied by the interval’s width, which is why area under the curve, not height, is what corresponds to probability.

Common Mistake: Interpreting a PDF’s value at a specific point as “the probability of that exact value,” which is a conceptual error for continuous data — any single exact value has probability zero; only the area under the PDF curve over a range of values corresponds to an actual probability.

Important Questions:

• Q1: Why does a specific exact value have probability zero under a continuous distribution’s PDF? Because a continuous distribution has infinitely many possible values within any range, the probability mass must be spread across that infinite continuum. Any single exact point occupies zero width, and therefore captures zero probability — only a range of values (an interval) can have positive probability, corresponding to the area under the PDF over that interval.
• Q2: What problem does kernel density estimation (KDE) solve, and how does it work at a high level? It estimates a smooth, continuous PDF from a finite sample of discrete data points, without the arbitrary bin-width choice a histogram requires. It works by placing a small smooth bump (kernel) centered at each individual data point and summing all the bumps together, producing a smooth curve rather than a blocky histogram.

Chapter 7: Relationships Between Variables

Difficulty: Intermediate · Statistics for Programmers — Week 6 · Key topics: Scatter plots, covariance, correlation (Pearson’s and Spearman’s), when correlation misleads

This chapter covers how to explore and quantify the relationship between two variables. It starts with scatter plots as the direct visual tool, then introduces covariance and Pearson’s correlation coefficient as numeric summaries of linear association, along with Spearman’s rank correlation as a more robust alternative less sensitive to outliers and nonlinear-but-monotonic relationships. It also emphasizes correlation’s important limitations — it only captures linear association and says nothing about causation.

Key Points:

• A scatter plot is the most direct way to see the relationship between two variables and should generally be examined before computing any numeric summary of association.
• Covariance indicates the direction of a linear relationship (positive or negative) but its magnitude is hard to interpret because it depends on the variables’ units.
• Pearson’s correlation coefficient normalizes covariance to a value between -1 and 1, making it comparable across different pairs of variables regardless of their units.
• Pearson’s correlation only captures linear relationships and is sensitive to outliers; Spearman’s rank correlation, computed on the ranks rather than raw values, is more robust to both.
• Correlation, however strong, never by itself establishes causation — a correlation between two variables can arise from a third, unmeasured confounding variable.

Memory Tip: Always plot the scatter plot before computing a correlation coefficient — a famous demonstration (the Datasaurus / Anscombe’s quartet-style examples) shows wildly different-looking scatter plots can share nearly identical correlation coefficients, so the number alone can hide a misleading or nonlinear relationship.

Common Mistake: Concluding that one variable causes another purely because they’re strongly correlated — ignoring the possibility of a confounding third variable, reverse causation, or pure coincidence, any of which can produce a strong correlation with no causal relationship at all between the two measured variables.

Important Questions:

• Q1: What is the key limitation of Pearson’s correlation coefficient that Spearman’s rank correlation addresses? Pearson’s correlation only measures linear relationships and is sensitive to outliers, which can distort it significantly. Spearman’s rank correlation is computed on the ranks of the values rather than the raw values themselves, making it more robust to outliers and able to detect monotonic (consistently increasing or decreasing) but nonlinear relationships that Pearson’s correlation would understate.
• Q2: Why does “correlation does not imply causation” matter in practice, not just as a slogan? A strong observed correlation between two variables can arise from a third, unmeasured confounding variable that influences both, from reverse causation, or from coincidence in a small sample — acting as though correlation alone proves one variable causes the other can lead to incorrect conclusions and poor decisions based on a relationship that isn’t actually causal.

Chapter 8: Estimation

Difficulty: Intermediate · Statistics for Programmers — Week 7 · Key topics: Point estimation, sampling distributions, standard error, confidence intervals, bias in estimators

This chapter covers estimation — using a sample to estimate a property of the larger population it was drawn from, and quantifying how much uncertainty that estimate carries. It covers the idea of a sampling distribution (what you’d get if you repeated the sampling process many times), standard error as a measure of an estimate’s variability, confidence intervals as a range likely to contain the true population value, and the concept of bias — whether an estimator systematically over- or under-estimates the true value.

Key Points:

• A point estimate (like a sample mean) is a single best guess at a population parameter, computed from a limited sample rather than the entire population.
• The sampling distribution of an estimator describes how that estimate would vary if the sampling process were repeated many times — it’s central to understanding an estimate’s reliability.
• Standard error quantifies the typical amount an estimate would vary from sample to sample; a smaller standard error means a more precise (though not necessarily more accurate) estimate.
• A confidence interval gives a range of plausible values for the true population parameter, with a stated confidence level (e.g. 90%) describing how often such intervals would contain the true value across repeated sampling.
• An estimator is biased if it systematically over- or under-estimates the true population value on average — bias and variability (standard error) are two separate, both important, properties of an estimator.

Memory Tip: Simulate the sampling process yourself in Python — repeatedly draw random samples from a known distribution, compute the estimate each time, and look at the resulting spread — rather than just accepting the standard error formula. This book’s simulation-first approach makes abstract sampling-distribution concepts concrete.

Common Mistake: Confusing a confidence interval’s confidence level with the probability that the true value lies in one specific computed interval — a 90% confidence interval means that if the sampling process were repeated many times, about 90% of such intervals would contain the true value, not that there’s a 90% chance the true value is in this particular interval.

Important Questions:

• Q1: What is the difference between an estimate’s standard error and whether it is biased? Standard error measures how much an estimate would vary from sample to sample — its precision or variability. Bias measures whether the estimator systematically over- or under-shoots the true value on average across repeated sampling. An estimator can have low variability (low standard error) while still being biased, or vice versa — the two are independent properties.
• Q2: What does a 90% confidence interval actually mean, in terms of repeated sampling? If the sampling and estimation process were repeated many times, about 90% of the confidence intervals computed this way would contain the true population value. It does not mean there is a 90% probability that the true value falls within this one specific interval that was already computed from a single sample.

Chapter 9: Hypothesis Testing

Difficulty: Advanced · Statistics for Programmers — Week 8 · Key topics: Classical hypothesis testing, p-values, permutation/resampling tests, the difference from Bayesian approaches

This chapter covers hypothesis testing — the framework for deciding whether an observed effect in data (like a difference between two groups) is likely to be real or could plausibly have arisen by chance alone. Rather than leading with classical closed-form test statistics, the book emphasizes computational permutation and resampling tests, where the null hypothesis is simulated directly in code many times to build up a null distribution, against which the observed effect is compared — the p-value being the fraction of simulated null results as extreme as the actual one.

Key Points:

• A hypothesis test asks: if there were truly no effect (the null hypothesis), how likely would we be to see an effect as large as the one actually observed, just by chance?
• A p-value is the probability, under the null hypothesis, of observing an effect at least as extreme as the one actually found — a small p-value suggests the observed effect is unlikely to be due to chance alone.
• A permutation test directly simulates the null hypothesis by repeatedly shuffling or resampling the data and recomputing the test statistic, building an empirical null distribution without relying on a closed-form formula.
• A statistically significant result (small p-value) is not automatically a practically important one — effect size matters separately from statistical significance, especially with very large sample sizes.
• This computational, simulation-based approach to hypothesis testing generalizes far more easily to complex, non-standard test statistics than relying on a fixed catalog of named classical tests.

Memory Tip: Write the permutation test as actual working code — shuffle the labels, recompute the statistic, repeat thousands of times, then count how often the shuffled result was as extreme as the real one — rather than memorizing p-value formulas. This book’s approach makes the definition of a p-value concrete instead of abstract.

Common Mistake: Treating a small p-value as proof that an effect is real and important, without also considering the effect’s actual size and practical significance — with a sufficiently large sample, even a tiny, practically meaningless difference can produce a statistically significant (small) p-value.

Important Questions:

• Q1: What does a p-value of 0.03 actually mean? It means that, assuming the null hypothesis (no real effect) is true, there is a 3% probability of observing an effect at least as extreme as the one actually seen in the data, purely by chance. It does not mean there is a 3% probability the null hypothesis is true, nor a 97% probability the observed effect is real.
• Q2: How does a permutation test simulate the null hypothesis without using a closed-form statistical formula? It repeatedly shuffles or randomly reassigns the data’s group labels (destroying any real association while preserving the data’s other properties), recomputes the test statistic on each shuffled version, and builds up a distribution of what the statistic looks like when there truly is no effect. The p-value is then the fraction of these shuffled results that are at least as extreme as the statistic computed from the real, unshuffled data.

Chapter 10: Linear Least Squares

Difficulty: Advanced · Statistics for Programmers — Week 9 · Key topics: Fitting a line to data, least-squares regression, residuals, goodness of fit

This chapter covers fitting a straight line to a set of data points using the method of least squares — finding the line that minimizes the sum of squared residuals (the vertical distances between each data point and the line). It covers computing the line’s slope and intercept, interpreting residuals as what’s left unexplained after fitting the line, and assessing how well the line actually fits using the coefficient of determination (R²).

Key Points:

• The least-squares line minimizes the sum of squared residuals — the vertical distances between each observed point and the fitted line — rather than, say, minimizing absolute distances.
• A residual is the difference between an actual observed value and the value the fitted line predicts for that same x; examining residuals reveals whether a linear model is actually appropriate.
• R² (the coefficient of determination) measures what fraction of the variance in the dependent variable is explained by the fitted line, ranging from 0 (no explanatory power) to 1 (perfect fit).
• A systematic pattern in a residual plot (rather than random scatter around zero) is a strong signal that a straight line is not actually the right model for the data.
• Least-squares fitting minimizes squared error specifically, which makes it sensitive to outliers — a single far-off point can noticeably pull the fitted line toward it.

Memory Tip: Always plot the residuals after fitting a line, not just the fitted line itself — a residual plot with an obvious curved pattern (rather than random scatter) reveals a poor linear fit far more clearly than looking at R² alone, which can still look reasonably high even for a poorly-fitting line.

Common Mistake: Fitting a straight line to data with an obviously curved or nonlinear relationship just because least-squares regression is the most familiar technique, then trusting the fitted line’s predictions — when the residual plot shows a clear pattern, a linear model is fundamentally the wrong tool, no matter how it’s fit.

Important Questions:

• Q1: What quantity does a least-squares regression line minimize? It minimizes the sum of the squared residuals — the sum of the squared vertical distances between each actual data point and the value the fitted line predicts at that same x-value.
• Q2: What does it mean if a residual plot shows a clear curved pattern rather than random scatter around zero? It indicates that a straight line is not an appropriate model for the underlying relationship — the data likely has a genuinely nonlinear relationship that a linear fit is failing to capture, even if the line’s overall R² value looks reasonably high.

Chapter 11: Regression

Difficulty: Advanced · Statistics for Programmers — Week 10 · Key topics: Multiple regression, categorical variables in regression, logistic regression for binary outcomes

This chapter extends the single-variable linear fitting of Chapter 10 to multiple regression, where several explanatory variables jointly predict an outcome. It covers interpreting multiple regression coefficients, including categorical (non-numeric) variables in a regression model, and introduces logistic regression — the appropriate technique when the outcome being predicted is binary (yes/no) rather than continuous.

Key Points:

• Multiple regression predicts an outcome from several explanatory variables simultaneously, and each coefficient represents that variable’s effect holding the other variables constant.
• A categorical variable (like a survey response category) must typically be converted to one or more numeric indicator (dummy) variables before it can be used in a regression model.
• Adding more explanatory variables to a regression always increases R² on the training data, even if a variable is genuinely useless — this makes naive R² comparisons between models with different numbers of variables misleading.
• Logistic regression is used when the outcome variable is binary (e.g. survived/didn’t survive), modeling the log-odds of the outcome as a linear function of the explanatory variables, rather than the outcome itself directly.
• Multiple regression coefficients can shift substantially, or even change sign, when a new correlated variable is added — a sign that the added variable was capturing some of what an earlier variable’s coefficient had absorbed.

Memory Tip: When comparing two regression models with different numbers of explanatory variables, use adjusted R² rather than plain R² — plain R² always increases (or stays the same) when adding variables, even useless ones, so it’s not a fair comparison tool on its own.

Common Mistake: Adding many explanatory variables to a regression model simply because each one slightly increases R², without considering whether they’re theoretically meaningful or whether the model is now overfitting the specific sample rather than capturing a genuine, generalizable relationship.

Important Questions:

• Q1: How does the interpretation of a coefficient in multiple regression differ from simple (single-variable) regression? In multiple regression, each coefficient represents that variable’s estimated effect on the outcome while holding all the other variables in the model constant. In simple regression with only one explanatory variable, there’s nothing else to hold constant, so the coefficient simply represents that variable’s overall association with the outcome.
• Q2: Why is logistic regression used instead of ordinary linear regression when the outcome variable is binary? Ordinary linear regression can predict values outside the valid 0-to-1 probability range and doesn’t naturally model a binary outcome’s structure. Logistic regression instead models the log-odds of the outcome as a linear function of the explanatory variables, which keeps predicted probabilities properly bounded between 0 and 1.

Chapter 12: Time Series Analysis

Difficulty: Advanced · Statistics for Programmers — Week 11 · Key topics: Time series data, trends, seasonality, moving averages, autocorrelation, serial correlation

This chapter covers data indexed by time — time series — and the techniques specific to analyzing it, since standard statistical methods that assume independent observations often don’t directly apply. It covers identifying trends and seasonal patterns, smoothing a series with moving averages, autocorrelation (how a series correlates with a lagged version of itself), and why serial correlation between nearby time points complicates standard hypothesis testing techniques from earlier chapters.

Key Points:

• A time series is data indexed by time, where the order of observations matters — unlike most data covered in earlier chapters, time series observations are often not independent of each other.
• A moving average smooths out short-term noise in a time series to reveal an underlying trend, at the cost of introducing some lag and reducing responsiveness to genuine recent changes.
• Seasonality refers to a regularly repeating pattern in a time series (daily, weekly, or yearly), distinct from the series’ overall long-term trend.
• Autocorrelation measures how correlated a time series is with a lagged (shifted) copy of itself, revealing repeating patterns and helping identify the right lag for a seasonal or cyclic effect.
• Serial correlation between nearby time points violates the independence assumption many standard statistical tests rely on, so time series data often requires specialized techniques rather than the general-purpose methods from earlier chapters.

Memory Tip: Before applying any hypothesis test or confidence interval technique from earlier chapters to time series data, check for serial correlation first — treating consecutive time points as independent observations when they’re actually correlated can make an estimate’s uncertainty look much smaller than it really is.

Common Mistake: Applying standard hypothesis-testing or confidence-interval techniques directly to time series data without accounting for serial correlation between nearby observations, producing confidence intervals that are falsely narrow because the technique assumes independence the data doesn’t actually have.

Important Questions:

• Q1: What is the key assumption that many standard statistical techniques rely on, which time series data often violates? The assumption that observations are independent of each other. In time series data, nearby observations in time are often correlated with each other (serial correlation), violating this independence assumption and requiring specialized time-series-aware techniques instead of standard methods.
• Q2: What is the difference between a trend and seasonality in a time series? A trend is the long-term overall direction of the series (steadily rising, falling, or flat over an extended period). Seasonality is a regularly repeating short-term pattern (like a daily, weekly, or yearly cycle) layered on top of that longer-term trend.

Chapter 13: Survival Analysis

Difficulty: Advanced · Statistics for Programmers — Week 12 · Key topics: Survival curves, the Kaplan-Meier estimator, censored data, hazard functions

This chapter covers survival analysis — techniques for analyzing the time until an event occurs (originally developed for medical survival data, but applicable to any “time until X happens” question, like marriage duration in the NSFG data used throughout the book). It covers the survival curve (the complement of a CDF, showing the probability of “surviving” past a given time), the Kaplan-Meier estimator for handling censored data (where the event hasn’t yet occurred for some observations by the time data collection ended), and hazard functions.

Key Points:

• A survival curve shows, for each time t, the probability that the event of interest hasn’t yet occurred by time t — it’s the complement (1 minus) of the event-time’s CDF.
• Censored data occurs when the event of interest hasn’t happened yet for some subjects by the end of data collection (e.g. a marriage that’s still ongoing) — simply excluding these observations would bias the analysis.
• The Kaplan-Meier estimator correctly incorporates censored observations into a survival curve estimate, using the information they do provide (they survived at least this long) without treating them as if the event never happens.
• A hazard function gives the instantaneous rate of the event occurring at time t, given survival up to that point — distinct from, but mathematically related to, the survival curve.
• Survival analysis techniques apply far beyond medical contexts — any “time until an event” question (marriage duration, equipment failure, customer churn) can benefit from these same tools.

Memory Tip: Never simply drop censored observations (subjects for whom the event hasn’t happened yet) from a survival analysis — this systematically biases the analysis toward subjects with shorter times to the event, since longer-surviving subjects are disproportionately the ones still censored at the end of data collection.

Common Mistake: Treating censored observations (where the event of interest hasn’t occurred yet by the end of data collection) as if the event will never happen, or simply excluding them from the dataset, rather than using a technique like the Kaplan-Meier estimator that correctly accounts for the partial information censored observations do provide.

Important Questions:

• Q1: What does it mean for an observation to be “censored” in survival analysis? It means the event of interest (e.g. marriage ending, equipment failing) had not yet occurred for that subject by the time data collection ended. The subject’s eventual event time is unknown — all that’s known is that it’s at least as long as the observed duration so far.
• Q2: Why does the Kaplan-Meier estimator handle censored data better than simply excluding censored observations from the analysis? Simply excluding censored observations would bias the analysis toward subjects who experienced the event relatively early (since longer-surviving subjects are more likely to still be censored at the end of the study). The Kaplan-Meier estimator instead uses the partial information a censored observation does provide — that it survived at least up to its observed duration — without discarding it entirely.

Chapter 14: Analytic Methods

Difficulty: Advanced · Statistics for Programmers — Week 13 · Key topics: Closed-form (analytic) alternatives to simulation, the Central Limit Theorem, when to use analytic vs. computational methods

This closing chapter circles back to connect the book’s computational, simulation-based approach with the classical, closed-form analytic methods found in traditional statistics textbooks. It covers the Central Limit Theorem — why sums and means of many independent random variables tend toward a normal distribution regardless of the original distribution’s shape — and discusses when a fast analytic formula is preferable to a slower simulation, and when the reverse is true.

Key Points:

• The Central Limit Theorem states that the sum (or mean) of many independent random variables tends toward a normal distribution as the number of variables grows, regardless of the shape of the original distribution.
• This is why the normal distribution appears so often in statistics — many real-world quantities are effectively sums of many small independent effects.
• Analytic (closed-form formula) methods are typically much faster to compute than simulation, but often rely on simplifying assumptions (like normality) that may not hold for a given real dataset.
• Simulation-based (computational) methods, the book’s primary approach throughout, make fewer assumptions and generalize more easily to complex or unusual situations, at the cost of computation time.
• A practical approach is often to first check a problem with fast simulation to build intuition and verify results, and only reach for a faster analytic formula once its underlying assumptions have been confirmed appropriate.

Memory Tip: Use this closing chapter as a bridge, not a replacement — having worked through simulation-based methods for the whole book, the classical formulas covered here should now feel like recognizable shortcuts for problems where their underlying assumptions genuinely hold, not unexplained magic to memorize.

Common Mistake: Reaching for a classical analytic formula (which usually assumes normality or another simplifying condition) without checking whether that assumption actually holds for the dataset at hand — when in doubt, this book’s own philosophy suggests verifying with a quick simulation first.

Important Questions:

• Q1: What does the Central Limit Theorem say, and why does it explain the normal distribution’s prevalence in statistics? It states that the sum or mean of many independent random variables tends toward a normal distribution as their number grows, regardless of the shape of each individual variable’s own distribution. Since many real-world measurements are effectively the combined result of many small independent factors, this theorem explains why the normal distribution shows up so frequently across many different, unrelated real-world contexts.
• Q2: According to this chapter, what is the practical trade-off between analytic (formula-based) and simulation-based statistical methods? Analytic methods are typically much faster to compute, since they use a closed-form formula, but usually depend on assumptions (like normality) that may not actually hold for a given dataset. Simulation-based methods make fewer such assumptions and adapt more easily to unusual or complex situations, at the cost of requiring more computation.

Download Think Stats PDF (Free)

This book is free from its official source — author Allen B. Downey publishes it on his own Green Tea Press site under a Creative Commons licence. Click below to download the complete PDF — a free browsable HTML edition is also available on the official site.

How to Study This Book

This book is organized to build progressively: Chapters 1–6 (Exploratory Data Analysis through Probability Density Functions) establish how to represent and describe a single distribution, and should be worked through in order since each builds on the last (histogram → PMF → CDF → PDF).

Chapters 7 onward (Relationships Between Variables through Analytic Methods) shift to comparing and modeling relationships — correlation, estimation, hypothesis testing, and regression — and lean on the distribution concepts from the first half, so don’t skip ahead without a solid grasp of PMFs and CDFs.

This book teaches statistics by writing and running Python code on a real dataset (the NSFG), not by memorizing formulas — actually running the example code for each chapter, not just reading the text, is essential to how this book is meant to be used.

Chapter 9 (Hypothesis Testing) takes a distinctly computational approach — permutation and resampling tests rather than classical closed-form formulas — which can look unfamiliar if coming from a traditional statistics course; work through the code examples slowly rather than skimming.

Chapters 12 and 13 (Time Series Analysis and Survival Analysis) are more specialized topics that some courses may skip or treat as optional — check your own course syllabus for which chapters are assigned.

Used In These Programs

This book is used as an applied, programming-based statistics text in: BSCS, Programming, and Python-based Statistics or Data Analysis coursework. Browse all Python books or all Computer Science category books.

Who Should Read This

Think Stats is written for students and self-learners who already know some Python and want to learn statistics by writing code on real data, rather than through formula-heavy derivations. It suits BSCS, BS Statistics, and BS Data Science students taking a Statistics for Programmers, Data Analysis, or introductory Data Science course, and is especially useful for anyone who finds a hands-on, computational approach more intuitive than a traditional proof-driven statistics textbook.

Applicable Universities

This book is useful for students at Pakistani universities offering BSCS, BS Statistics, or BS Data Science programs with a Statistics for Programmers, Exploratory Data Analysis, or applied Statistics course, including Punjab University, Virtual University, COMSATS, FAST, UET, NUST, GIKI, and other HEC-recognized institutions, and for self-learners who already know Python and want a free, code-first statistics reference.

FAQs

Related Books

• Python for Everybody – Charles R. Severance (a gentler introduction to Python for working with data)
• Think Python – Allen B. Downey (the same author’s introductory Python programming text)

Think Stats is a free, code-first statistics textbook that teaches exploratory data analysis, estimation, hypothesis testing, and regression by writing Python programs on a real dataset. Browse more Computer Science books for the rest of your semester.

Think Stats: Exploratory Data Analysis in Python, by Allen B. Downey. Free under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 licence. Access for free at https://greenteapress.com/wp/think-stats-2e/