Statistical Thinking for the 21st Century – Free Online Book (Poldrack) BS Psychology/Data Science

Statistical Thinking for the 21st Century is a free textbook by Russell A. Poldrack, a psychologist at Stanford University, written for a first university course in statistics. It suits BS Psychology and BS Data Science students and fits an Introduction to Statistics or Statistics for Social Science course. The author keeps the whole book free to read online on the project website. There is no official PDF, and a separately published edition is sold by Princeton University Press.

The book is built around one idea: every analysis is a model of the data plus an honest statement of how uncertain that model is. It moves from data, summaries and probability to sampling, simulation and the bootstrap, then hypothesis tests, confidence intervals, Bayesian statistics, regression, the general linear model, ANOVA and multivariate methods, and ends with a chapter on the reproducibility crisis. Examples use real data such as the NHANES health survey, so it works for exam revision and as a first step into data analysis for a lab or thesis.

Book Overview

Chapter List

Chapter 1: Introduction

Difficulty: Easy · Key topics: statistical thinking, describe-decide-predict, aggregation, uncertainty, sampling, causality

The chapter defines statistical thinking as describing a complex world in simple terms while saying how uncertain that description is, and shows that raw intuition often answers such questions wrongly. It names three jobs for statistics (describe, decide, predict) and follows one question, whether saturated fat is harmful, from common sense to authority to a cohort study of more than 135,000 people. Four big ideas come out of it: learning from data, aggregation, uncertainty and sampling. It closes by separating observational studies from randomized controlled trials and explaining why only randomization supports claims about causes.

Key Points:

• The three goals of statistics are to describe, to decide and to predict.
• Aggregation means condensing many numbers into a few useful ones; the book calls statistics “the science of throwing away data”.
• Statistics can never prove a hypothesis; it gives evidence that stays tentative.
• A small sample can describe a whole population, but only if it is drawn in the right way.
• Observational studies show association; a randomized controlled trial is what supports a causal claim.

Memory Tip: Describe, Decide, Predict. Before choosing any method, ask which of these three jobs your question needs done.

Common Mistake: Reading a strong pattern in observational data as proof that changing the cause will change the outcome.

Important Questions:

• What are the three major things we can do with statistics? Describe the data, decide something in the face of uncertainty, and predict new situations from earlier ones.
• Why can randomization support causal conclusions when observation cannot? Random assignment spreads other influences evenly across groups, so a remaining difference can be traced to the factor that was manipulated.

Chapter 2: Working with Data

Difficulty: Easy · Key topics: qualitative and quantitative variables, binary, integer and real numbers, discrete vs continuous, reliability, validity, scales of measurement

Data are made of variables, and the chapter first separates qualitative variables, which describe a quality, from quantitative ones, and then sorts numbers into binary, integer and real. It then asks what makes a good measurement when the thing measured is an unobservable construct, such as an attitude. A measurement should be reliable (consistent across repeats or raters) and valid (it measures what it claims, judged by face, construct and predictive validity). The appendix introduces the four scales of measurement and what each allows you to compute.

Key Points:

• A discrete measurement takes one of a set of separate values; a continuous one can fall anywhere in a range.
• Reliability includes test-retest agreement and agreement between different raters.
• Two variables cannot be related more strongly than either is related to itself, so an unreliable measure can never show a strong relationship.
• Validity comes in three flavors in the book: face, construct and predictive.
• The four scales are nominal, ordinal, interval and ratio; only the ratio scale has a true zero.
• Counting statistics such as the mode work on any scale; other statistics need order or equal intervals.

Memory Tip: NOIR (Nominal, Ordinal, Interval, Ratio) lists the scales from weakest to strongest; each one keeps everything the previous one had and adds one feature.

Common Mistake: Treating reliability and validity as the same thing; a test can give the same wrong answer every time and still be perfectly reliable.

Important Questions:

• What is the difference between reliability and validity? Reliability is consistency of the measurement; validity is whether it actually measures the intended construct.
• What does a ratio scale have that an interval scale lacks? A true, meaningful zero point, which is why ratios such as “twice as heavy” make sense.

Chapter 3: Summarizing Data

Difficulty: Easy · Key topics: frequency tables, relative and cumulative frequency, histograms, bins, normal distribution, skewness, long tails

Summarizing always throws information away, and the chapter argues that this is the price of being able to generalize. It builds up from tables of absolute, relative and cumulative frequency to histograms, using the NHANES health survey for real examples such as hours of sleep, age and height. It shows how bin width changes the look of a histogram, then moves to idealized shapes: the bell-shaped normal distribution, skewed distributions with one long tail, and long-tailed distributions where a few huge values dominate and many standard tools start to fail.

Key Points:

• Relative frequency is a category’s count divided by the total count, so it can be read as a percentage.
• A cumulative distribution adds up frequencies up to and including each value.
• Number of bins = range of scores / bin width; the bin width is a judgment call.
• Many real measurements are roughly normal, symmetric around a rounded peak.
• Skewed and long-tailed distributions are common, and tools that assume a normal shape can fail on them.

Practice Tip: Take any numeric column you have, plot its histogram with three different bin widths, and watch which features survive; those are the ones worth trusting.

Common Mistake: Settling on the first bin width the software offers; too narrow looks jagged and too wide hides structure.

Important Questions:

• How do you compute a cumulative frequency? Add the frequencies of all values up to and including the one of interest.
• Why do long-tailed distributions cause trouble for standard statistics? A few extreme values dominate summaries such as the mean, so methods built for thin-tailed, normal-like data become unreliable.

Chapter 4: Data Visualization

Difficulty: Easy · Key topics: anatomy of a plot, violin and box plots, data/ink ratio, chartjunk, lie factor, perception, color blindness

The chapter opens with the Challenger shuttle disaster to argue that a poorly designed presentation of data can fail to persuade people even when the evidence is there. It explains the parts of a plot and compares bar graphs, bars with points, violin plots and box plots for showing group differences. Then it sets out principles: show the data, maximize the data/ink ratio, avoid chartjunk and avoid distortion such as misleading axes. It finishes with human limits, including color blindness and the weakness of pie charts, and with correcting for other factors such as inflation.

Key Points:

• A bar of means hides the spread of the data; violin or box plots show the distribution.
• Data/ink ratio = ink used for data / total ink; clutter lowers it.
• Chartjunk is decoration that is unrelated to the data, such as 3D bars and background textures.
• Tufte’s lie factor compares the size of an effect in the picture with its size in the data.
• Whether the Y axis must start at zero depends on the plot; bar charts usually need it, time series often do not.
• People judge length along a common scale better than area or color.

Practice Tip: Redraw a bar chart of group means as a box plot with the raw points laid over it, and note what the bars were hiding.

Common Mistake: Cutting the Y axis of a bar chart so a tiny difference looks huge.

Important Questions:

• What is the data/ink ratio? The share of a graphic’s ink that represents data; good graphics keep it high and remove clutter.
• Why does the book advise against pie charts? Humans compare angle and area poorly, whereas bar lengths on a common axis can be compared accurately.

Chapter 5: Fitting Models to Data

Difficulty: Medium · Key topics: data = model + error, sum of squared errors, overfitting, mean, median, mode, variance, degrees of freedom, Z-scores

This chapter gives the book its central equation, data = model + error. Using the heights of children in NHANES, it fits the simplest model (one number), compares the mode and the mean, then improves the model with age and sex. It explains that a good model must both describe the current data and generalize, which leads to overfitting. It then examines the mean, median and mode, variance and standard deviation (with a simulation showing why n – 1 is used), and Z-scores for comparing values on a common scale.

Key Points:

• error = observed value – predicted value; the model is the part of the data we can explain.
• The mean gives an average error of zero and the smallest sum of squared errors.
• The median is robust to outliers because it minimizes absolute, not squared, error.
• Sample variance divides by n – 1 because one degree of freedom is spent estimating the mean.
• Z = (x – μ) / σ says how many standard deviations a value sits from the mean.
• A model that fits one dataset very well can do worse on new data than a simpler one.

Memory Tip: The mean minimizes squared distance; the median minimizes absolute distance. That one sentence explains why the mean is pulled by outliers and the median is not.

Common Mistake: Judging a model by its average error; positive and negative errors cancel, so the squared error must be used.

Important Questions:

• Why is the sample variance divided by n – 1? The sample mean has already been estimated from the same data, which uses up one degree of freedom; dividing by n would underestimate the population variance.
• What does a Z-score tell you? How far a data point is from the mean, measured in standard deviations, which makes different variables comparable.

Chapter 6: Probability

Difficulty: Medium · Key topics: sample space, probability rules, law of large numbers, binomial distribution, conditional probability, independence, Bayes’ rule, odds

The chapter defines experiments, sample spaces and events, and gives the basic rules a probability must obey. It compares three ways to get probabilities: personal belief, empirical frequency (backed by the law of large numbers) and classical equally likely outcomes, and uses the gambler de Mere’s mistaken bets to teach the subtraction, intersection and addition rules. It then covers the binomial distribution, conditional probability, statistical independence, and Bayes’ rule through a medical screening example, with odds and odds ratios, ending with two meanings of probability.

Key Points:

• A probability lies between 0 and 1, and the probabilities of all outcomes in the sample space sum to 1.
• P(not A) = 1 – P(A); for any two events P(A or B) = P(A) + P(B) – P(A and B).
• The law of large numbers says empirical probability approaches the true value, but small samples can be far off.
• P(A|B) = P(A and B) / P(B); A and B are independent when knowing B does not change P(A).
• Bayes’ rule reverses a conditional: P(B|A) = P(A|B) P(B) / P(A).
• Odds are the chance an event happens relative to the chance it does not; an odds ratio compares two odds.

Memory Tip: For “at least one” questions, work out the chance of none and subtract from 1; that is exactly how Pascal fixed de Mere’s bets.

Common Mistake: Confusing P(A|B) with P(B|A); a positive test for a rare condition is still usually a false alarm.

Important Questions:

• Why was de Mere wrong to add the probabilities of separate throws? The events overlap, so the addition rule requires subtracting the probability that both happen.
• Why is a positive screening result for a rare disease often a false positive? The prior probability of the disease is very low, so most positives come from the large healthy group.

Chapter 7: Sampling

Difficulty: Medium · Key topics: population and sample, sampling error, sampling distribution, standard error of the mean, Central Limit Theorem

Starting from election polls, the chapter explains how a statistic computed on a small sample can stand in for a population value, and why how the sample is drawn matters more than how big it is. It separates sampling with and without replacement, defines sampling error, and uses the NHANES data as a pretend population to draw many samples and look at the sampling distribution of the mean. From there it introduces the standard error of the mean and the Central Limit Theorem, which says sample means become normal even when the raw data are not.

Key Points:

• A parameter describes a population; a statistic is computed from a sample.
• Sampling error is the difference between a sample statistic and the population parameter, and it changes from sample to sample.
• SEM = σ / √n, so quality of the estimate depends on both population variability and sample size.
• Estimating the standard deviation from a very small sample (under about 30) needs extra care.
• The Central Limit Theorem makes the sampling distribution of the mean approximately normal as samples grow.

Practice Tip: Take a skewed variable, draw many samples of 50, plot the sample means, and watch the bell shape appear.

Common Mistake: Mixing up the standard deviation (spread of the data) with the standard error (spread of the sample mean).

Important Questions:

• What happens to the standard error as the sample size grows? It shrinks in proportion to the square root of n, so larger samples give tighter estimates.
• Why is the Central Limit Theorem so useful? It lets us assume the sampling distribution of the mean is normal in most cases, even when individual data points are not.

Chapter 8: Resampling and Simulation

Difficulty: Medium · Key topics: Monte Carlo simulation, randomness, pseudo-random numbers, random seeds, bootstrap

Computing power lets us answer statistical questions by simulating them, and this chapter introduces Monte Carlo simulation, which grew out of work on the Manhattan Project. It lists four steps: define a domain, generate random numbers, compute something with them, and combine the results. It explains what randomness means in statistics, why people judge randomness badly, and how computers produce pseudo-random numbers that can be repeated with a seed. A quiz-time example shows simulation in use, and the bootstrap resamples the data with replacement to estimate uncertainty.

Key Points:

• Monte Carlo steps: define a domain, generate random numbers, compute, then combine across many repetitions.
• In statistics a process is random if it is unpredictable.
• Software uses pseudo-random numbers, which look random but follow a repeatable algorithm.
• Setting a seed reproduces exactly the same random numbers.
• The bootstrap repeatedly samples from the observed data with replacement to approximate a sampling distribution.

Practice Tip: Run any simulation twice with the same seed and then with different seeds, and compare; reproducible results come from fixing the seed.

Common Mistake: Resampling the data without replacement, which just reshuffles the original sample and gives no new variation.

Important Questions:

• What are the four steps of a Monte Carlo simulation? Define a domain of values, generate random numbers from a distribution, compute with them, and combine the results over many repetitions.
• What is the bootstrap used for? Estimating the sampling distribution of a statistic when its distribution is unknown or non-normal, by resampling the observed data with replacement.

Chapter 9: Hypothesis Testing

Difficulty: Hard · Key topics: NHST, null and alternative hypotheses, test statistic, p-value, randomization test, Type I and Type II errors, multiple testing

The chapter presents null hypothesis statistical testing and warns up front that it is widely misunderstood and, in the author’s view, deeply flawed. A six-step procedure is walked through with a question about physical activity and BMI in NHANES, where the t statistic is judged against a null distribution from the t distribution and from randomization. It then traces the 0.05 convention to Fisher, contrasts the Neyman-Pearson error-rate view, lists common misreadings of the p-value, and shows how multiple tests inflate false positives, with the Bonferroni correction.

Key Points:

• The p-value is the probability of data this extreme if the null hypothesis were true, not the probability that the null is true.
• A directional hypothesis uses one tail; the non-directional two-tailed test is more conservative.
• Randomization builds a null distribution by shuffling labels and assumes exchangeability.
• A Type I error (false alarm) has probability α; a Type II error (miss) has probability β.
• A statistically significant effect is not automatically practically important.
• Bonferroni correction divides α by the number of tests to control familywise error.

Memory Tip: Rejecting a true null is a false alarm (Type I); keeping a false null is a miss (Type II).

Common Mistake: Reading p = .01 as a 1% chance that the null hypothesis is true or a 99% chance the result will replicate.

Important Questions:

• What does a p-value actually measure? The probability of observing a result at least this extreme if the null hypothesis were true.
• Why is multiple testing a problem? If you run many tests at p < .05, about 5% will be significant by chance even when nothing is real, so the threshold must be corrected.

Chapter 10: Quantifying Effects and Designing Studies

Difficulty: Medium · Key topics: confidence intervals, t distribution, bootstrap intervals, effect size, Cohen’s d, Pearson’s r, odds ratio, statistical power

A reject-or-retain verdict from Chapter 9 says nothing about how big or how certain an effect is, so this chapter adds two tools. Confidence intervals express uncertainty about an estimate, computed with the normal distribution, the t distribution or the bootstrap, and the chapter stresses that a 95% interval describes the procedure, not a probability for one interval. Effect sizes (Cohen’s d, Pearson’s r and the odds ratio) express magnitude on a standard scale. Finally, statistical power and power analysis show how to pick a sample size before running a study.

Key Points:

• A confidence interval is estimate plus or minus a critical value times the standard error.
• When the population standard deviation is unknown, the critical value comes from the t distribution.
• Larger samples shrink the standard error, so the interval gets narrower.
• An interval that excludes the null value matches a significant test at the same level.
• Cohen’s d is a difference in means divided by the pooled standard deviation; Pearson’s r runs from -1 to 1.
• Power rises with larger samples and larger effects, and falls as the Type I error rate is made stricter.

Memory Tip: Significance asks “is there an effect?”, the effect size asks “how big?”, and the interval asks “how sure are we?” Report all three.

Common Mistake: Saying there is a 95% probability that the true value lies inside one particular computed interval.

Important Questions:

• What three factors affect statistical power? Sample size, effect size and the Type I error rate.
• Why run a power analysis before a study? It tells you the sample needed to detect the effect you care about, so you avoid an underpowered study that wastes effort and rarely finds anything.

Chapter 11: Bayesian Statistics

Difficulty: Hard · Key topics: generative models, prior, likelihood, posterior, MAP, credible interval, Bayes factor

This chapter offers the alternative to null hypothesis testing. It starts with generative models, in which a hidden process produces the data we see, and uses Bayes’ theorem to run that logic backward. A security-screening example walks through the four parts: prior, likelihood, marginal likelihood and posterior. A second example, a drug that works for 64 of 100 people, estimates a whole posterior distribution, with MAP estimates, credible intervals and the effect of weak or strong priors. The chapter ends with choosing priors and Bayes factors for testing hypotheses.

Key Points:

• Posterior is proportional to prior times likelihood; the marginal likelihood rescales it so it sums to 1.
• Frequentists do not assign probabilities to hypotheses; Bayesians do.
• A posterior can become the prior for the next analysis, so Bayesian updating is sequential.
• A 95% credible interval says there is a 95% probability the parameter lies inside it, which is what people wish confidence intervals meant.
• A strong enough prior can overwhelm the data; choosing the prior is the most debated step.
• A Bayes factor compares how well the data fit two hypotheses and can show evidence for the null.

Memory Tip: Prior × Likelihood, scaled to sum to 1, gives the Posterior: what you believed, times what the data say, equals what you now believe.

Common Mistake: Treating a significant p-value as strong evidence when the Bayes factor shows the data barely favor the alternative.

Important Questions:

• What are the four parts of Bayes’ theorem? The prior, the likelihood, the marginal likelihood and the posterior.
• How does a credible interval differ from a confidence interval? A credible interval gives the probability that the parameter lies within it, given the data and prior; a confidence interval describes long-run behavior of a procedure.

Chapter 12: Modeling Categorical Relationships

Difficulty: Medium · Key topics: chi-squared test, contingency table, expected counts, standardized residuals, odds ratio, Bayes factor, Simpson’s paradox

The chapter turns to variables that are counts of categories. A bag of candies tests whether observed counts differ from expected ones with Pearson’s chi-squared statistic, and a simulation confirms the shape of the chi-squared distribution. It then uses contingency tables to ask whether two categorical variables are related, using police-search data, with standardized residuals to see where the differences are, an odds ratio for magnitude and a Bayes factor for strength of evidence. Larger tables and Simpson’s paradox, shown with batting averages, close the chapter.

Key Points:

• Chi-squared = sum of (observed – expected)² / expected across all cells.
• The expected counts in a two-way table come from assuming the two variables are independent.
• A significant chi-squared test says the data differ from the null, not how; standardized residuals show where.
• The odds ratio and the Bayes factor add size and strength of evidence to the test.
• The same logic works for tables bigger than 2 × 2.
• Simpson’s paradox: a trend present in every subgroup can reverse when the groups are combined.

Memory Tip: Chi-squared measures the gap between what you saw and what independence predicts; no gap, no association.

Common Mistake: Combining subgroups into one summary table without checking the subgroups, which can flip the conclusion.

Important Questions:

• What does the chi-squared test for association test? Whether two categorical variables are related, by comparing observed cell counts with those expected under independence.
• What is Simpson’s paradox? A pattern that appears in each subgroup but disappears or reverses when the groups are pooled.

Chapter 13: Modeling Continuous Relationships

Difficulty: Medium · Key topics: covariance, correlation, testing r, outliers, Spearman correlation, causal graphs, mediation, Gini index

The chapter looks at how two continuous variables relate, using a published story that tied hate-crime rates to income inequality across US states. Covariance is turned into the correlation coefficient, which can be tested with a t statistic on N – 2 degrees of freedom, by randomization or by a Bayesian estimate. One outlier, the District of Columbia, changes the picture, so the rank-based Spearman correlation is introduced as a robust alternative. The last section separates correlation from causation and introduces causal graphs and mediation.

Key Points:

• Covariance is the mean cross-product of deviations from each variable’s mean; it depends on the units.
• Correlation r is the covariance rescaled, so it always lies between -1 and 1.
• Under H0: r = 0, the test statistic follows a t distribution with N – 2 degrees of freedom.
• Spearman correlation works on ranks and is far less sensitive to outliers.
• A causal graph shows variables as circles and causes as arrows; a mediator sits between a cause and its effect.
• The appendix explains the Gini index as a measure of inequality.

Practice Tip: Compute Pearson and Spearman correlations on the same two columns; a big gap between them usually means an outlier is steering the result.

Common Mistake: Reporting a correlation without plotting the data first; one extreme point can create or erase it.

Important Questions:

• Why use Spearman instead of Pearson correlation? Because ranks reduce the influence of outliers, giving a more robust measure of association.
• What does it mean for one variable to mediate a relationship? The effect of the first variable on the outcome passes through it, so holding the mediator constant removes the apparent link.

Chapter 14: The General Linear Model

Difficulty: Hard · Key topics: linear regression, regression to the mean, R-squared, multiple regression, interactions, residual checks, cross-validation

Here the book’s model of data becomes a general linear model, an outcome predicted by a weighted sum of predictors. Linear regression is built on a study-time and grades example that covers describing, deciding and predicting. The chapter explains regression to the mean, the link between correlation and slope, standard errors, t tests for parameters and R-squared, and extends to several predictors and interactions, using caffeine and anxiety in public speaking. It then checks assumptions with residuals and Q-Q plots, shows how overfitting fools prediction, and introduces cross-validation.

Key Points:

• The dependent variable (Y) is the outcome; independent variables (X) are used to explain it.
• Regression to the mean: extreme scores tend to be followed by less extreme ones, with no intervention needed.
• R-squared is the fraction of variance accounted for by the model.
• An interaction means the slope of one predictor depends on the level of another; interpret main effects with care.
• A Q-Q plot checks whether residuals are normally distributed.
• Cross-validation estimates predictive accuracy on new data and exposes overfitting.

Memory Tip: Same equation as Chapter 5: data = model + error. The GLM only changes what goes inside the model.

Common Mistake: Crediting an intervention for a rise in scores of people selected for low scores, when regression to the mean explains much of it.

Important Questions:

• What does R-squared tell you? The proportion of variability in the outcome that the model explains.
• Why is cross-validation needed? A complex model fit and judged on the same data looks accurate even when there is no real signal; cross-validation tests it on held-out data.

Chapter 15: Comparing Means

Difficulty: Medium · Key topics: sign test, one-sample t-test, two-sample t-test, t-test as a linear model, paired t-test, Bayes factor, ANOVA

The chapter surveys the common ways to ask about averages. It begins with a single mean, using the sign test and Student’s t-test, then compares two independent groups, and shows that the two-sample t-test is itself a linear model, with Cohen’s d and R-squared as effect sizes and a Bayes factor to quantify evidence. Paired observations are treated next, explaining why independent-sample tests are wrong for repeated measures. Analysis of variance then handles three or more groups, with dummy coding and an omnibus test.

Key Points:

• The t statistic compares the gap between the sample mean and a hypothesized value with the sampling variability of the mean.
• A large p-value does not show that the null hypothesis is true; a Bayes factor can.
• The two-sample t-test is a linear model with one dummy-coded predictor.
• Paired data need a paired test (or a one-sample test on the differences), not an independent-samples test.
• ANOVA splits total variance into model and error parts and tests whether the model part is larger than chance.
• The omnibus hypothesis is that no group differs from any other.

Memory Tip: One mean: one-sample t. Two separate groups: two-sample t. Same people twice: paired t. Three or more groups: ANOVA.

Common Mistake: Running an independent-samples test on before-and-after measurements from the same people.

Important Questions:

• When should you use a paired t-test? When the two measurements come from the same units, such as the same person measured twice.
• What does ANOVA test? Whether the variance explained by group membership is greater than expected by chance, which is the omnibus question of whether any means differ.

Chapter 16: Multivariate Statistics

Difficulty: Hard · Key topics: unsupervised learning, heatmaps, K-means, hierarchical clustering, PCA, factor analysis

Multivariate analysis involves many variables at once, and this chapter focuses on two aims: seeing the structure in the data and reducing many variables to fewer. It distinguishes supervised from unsupervised learning and introduces a real dataset of self-control measures from 522 participants. Scatterplot matrices and heatmaps visualize the relationships. Clustering follows, using Euclidean distance with K-means and agglomerative hierarchical clustering. Dimensionality reduction covers principal component analysis, with scree plots and loadings, and exploratory factor analysis, with the problem of choosing the number of factors.

Key Points:

• Supervised learning has a known target to predict; unsupervised learning looks for structure without one.
• Euclidean distance is the length of the straight line between two points (the Pythagorean theorem).
• K-means places K cluster centers and assigns each point to the nearest one; starting positions matter.
• Agglomerative clustering starts with each point as its own cluster and merges the closest ones step by step.
• PCA finds the combinations of variables that carry the most variance; a scree plot shows how much each explains.
• Factor analysis assumes hidden latent factors generate the observed variables.

Practice Tip: Run K-means for several values of K and a hierarchical clustering on the same data; if the groups agree, the structure is probably real.

Common Mistake: Expecting one correct answer from clustering; with no ground truth the number of clusters is a judgment.

Important Questions:

• What is the difference between supervised and unsupervised learning? Supervised learning predicts a known outcome; unsupervised learning finds groupings or structure with no outcome given.
• What is the goal of principal component analysis? To describe many variables with fewer composite ones that keep as much of the information (variance) as possible.

Chapter 17: Practical Statistical Modeling

Difficulty: Medium · Key topics: modeling workflow, ANOVA, residual checks, effect size, post-hoc comparisons, confounds

This chapter puts the earlier tools to work on one real question, comparing four weight-loss diets in a randomized study of 311 women. It lays out a seven-step modeling process: specify the question, get the data, prepare it, choose the model, fit it, criticize it and test hypotheses with effect sizes. Because the original data were not public, the chapter builds synthetic data from the paper’s summary numbers. It then checks a possible confound, rates of metabolic syndrome, with a chi-squared test and a Bayes factor, and ends with advice on getting help. The Princeton University Press edition carries an expanded version of this chapter.

Key Points:

• Seven steps: question, data, preparation, model choice, fitting, model criticism, then hypothesis test and effect size.
• Model choice depends on the kind of outcome (continuous or proportion), what is compared and whether observations are independent.
• Check residuals before trusting any p-value or F test.
• A significant omnibus F test does not say which groups differ; follow-up comparisons that adjust for the number of tests do.
• Possible confounds should be checked, here with a chi-squared test and a Bayes factor.
• Consult a trained statistician when analyzing real data.

Memory Tip: Ask, collect, clean, choose, fit, check, test. Run them in that order every time.

Common Mistake: Jumping to the hypothesis test before checking whether the model fits the data.

Important Questions:

• What are the steps of the statistical modeling process in this chapter? Specify the question, collect data, prepare data, determine the model, fit it, criticize it, then test the hypothesis and quantify the effect size.
• What does a significant omnibus F test not tell you? Which specific groups differ from one another.

Chapter 18: Doing Reproducible Research

Difficulty: Easy · Key topics: reproducibility crisis, positive predictive value, winner’s curse, p-hacking, HARKing, pre-registration, replication

The closing chapter compares how science is supposed to work with how it sometimes works, using a high-profile case of a food researcher whose papers were retracted. It explains the reproducibility crisis that followed 2010 and Ioannidis’s argument that low power and unlikely hypotheses make many significant findings false, with simulations of positive predictive value and the winner’s curse. It describes questionable research practices such as HARKing and p-hacking, and what is done about them: pre-registration, reporting rules, replication and scripted, shareable analyses.

Key Points:

• Positive predictive value is the share of significant results that are true; it falls when power is low or hypotheses are unlikely.
• The winner’s curse: low power inflates estimated effect sizes among the significant results.
• HARKing means hypothesizing after the results are known.
• P-hacking means trying many analyses until one gives p < .05.
• Pre-registration commits to the analysis plan before seeing the data.
• Scripted analyses shared with the data make a study reproducible.

Memory Tip: Decide first, then collect, then report everything. Pre-registration protects you from fooling yourself.

Common Mistake: Assuming a failed replication proves the original result was wrong; the p-value also says nothing about whether a finding will replicate.

Important Questions:

• What is p-hacking? Trying different analyses, variables, exclusions or stopping rules until a significant result appears, which inflates the false-positive rate.
• How does pre-registration help? It fixes the hypotheses and analysis plan in advance, so researchers cannot adjust them after seeing the results.

How to Study This Book

Read Chapters 1 to 4 first, in order. They are short and conceptual: what statistics is for, what a variable and a measurement are, how to summarize a distribution and how to draw an honest graph. Everything later reuses this vocabulary, and Chapter 1’s difference between observation and randomized experiment comes back in Chapters 13 and 17.

Treat Chapter 5 as the hinge of the book. Its equation, data = model + error, is the frame for the t-test in Chapter 15, the regression in Chapter 14 and the ANOVA in Chapter 17. Spend extra time on the mean, the sum of squared errors and the n – 1 correction, because the later chapters assume you are comfortable with them.

Chapters 6 to 8 form the toolkit for inference: probability rules and Bayes’ rule (6), sampling error and the Central Limit Theorem (7), and simulation and the bootstrap (8). Chapter 9 then uses them to build the null hypothesis test, getting a p-value both from the t distribution and from randomization, so do not skip ahead. Chapter 10 completes the frequentist set with confidence intervals, effect sizes and power. A course that is only frequentist can postpone Chapter 11 (Bayesian statistics), but Bayes factors reappear in Chapters 12, 15 and 17, so plan to come back.

Chapters 12 to 15 are the working core and are best read in this order: categorical data (12), correlation (13), the general linear model (14), then comparing means (15). Chapter 14 depends on Chapter 13, and Chapter 15 shows that the t-test and ANOVA are special cases of Chapter 14. If you already know the classic tests, read Chapter 15 last and watch for the idea that each of them is the same linear model with a different set of predictors.

Chapter 16 (multivariate statistics) goes beyond a typical first course. It is rewarding for BS Data Science students who want a first look at clustering, PCA and factor analysis, and a little linear algebra intuition helps. Chapter 17 is a worked case that ties the toolkit together, and Chapter 18 (reproducible research) can be read any time after Chapter 9; it is especially useful before writing a thesis or lab report in psychology.

Read the learning objectives at the end of each chapter as a checklist, because the chapters end with objectives and suggested readings rather than problem sets, and find practice problems elsewhere. The author says the book was shaped for a 10-week Stanford quarter, so on a 16-week semester you can slow down to one chapter a week and use the spare weeks for the exercises your instructor sets.

What the book does not cover: it is not a mathematical statistics text, so it does not derive distributions or prove theorems; it leaves out time series and survival analysis; and its Bayesian chapter stays at the level of simple grid and rejection-sampling examples. The preface warns that the datasets are heavily US-centric. Code examples use R, and the project lists companion books for R and Python that, according to the preface, were still works in progress.

Used In These Programs

This book is used for the Introductory Statistics / Statistics for Social and Data Science course in: BS Psychology · BS Data Science. Browse all Statistics-tagged books, the Statistical Inference books, or the full Statistics category on Freebooks.pk.

Who Should Read This

Statistical Thinking for the 21st Century is written for students who need to use statistics in their own work rather than prove theorems about it: first and second-year BS Psychology, BS Data Science and related social-science students, and anyone taking a first statistics course with a data-analysis lab. It assumes no earlier statistics and teaches ideas through real data, with simulation used alongside the classical formulas. It is not the right choice if your syllabus wants a calculus-based mathematical statistics course, and it has no problem sets, so pair it with a textbook that provides exercises, such as Introduction to Modern Statistics, which also teaches simulation-based inference. Students whose courses lean on probability can add Introduction to Probability, and those who want more on Bayesian thinking can continue with Bayesian Methods for Hackers.

Applicable Universities

This book is useful for students at Pakistani universities offering BS Psychology, BS Data Science, BS Statistics and related social-science and computing degrees, including Punjab University, Virtual University, COMSATS, FAST, UET, NUST, and other HEC-recognized institutions.

Official Copy of Statistical Thinking for the 21st Century

This book is free from its official site. The author publishes no PDF, so the legitimate free way to read it is the complete online edition. Click the button below to open the book on the official statsthinking21 website, where all 18 chapters are available.

FAQs

Related Books

• Introduction to Modern Statistics – Çetinkaya-Rundel & Hardin (2nd Edition)
• OpenIntro Statistics – Diez, Çetinkaya-Rundel & Barr (4th Edition)
• Bayesian Methods for Hackers – Cameron Davidson-Pilon
• Introduction to Probability – Grinstead & Snell (2nd Edition)

Statistical Thinking for the 21st Century teaches statistics as a way of reasoning about data and uncertainty, with simulation and real datasets doing much of the explaining, from the first summary table to regression and the reproducibility crisis. Browse more Statistics books on Freebooks.pk.

Statistical Thinking for the 21st Century by Russell A. Poldrack, Stanford University, copyright 2019. Licensed under a Creative Commons Attribution-NonCommercial 4.0 Generic (CC BY-NC 4.0) license. Official sources: statsthinking21.org and the online book. A separately published edition, Statistical Thinking: Analyzing Data in an Uncertain World, is available from Princeton University Press.