R for Data Science (2nd Edition) – Free Online Book (Wickham, Çetinkaya-Rundel, Grolemund)

R for Data Science (2nd Edition) by Hadley Wickham, Mine Çetinkaya-Rundel, and Garrett Grolemund is the standard introduction to doing data science with R and the tidyverse. Published by O’Reilly Media in 2023, the book teaches the full data science workflow — import, tidy, transform, visualize, and communicate — using a consistent set of R packages designed to work together.

The second edition is a substantial rewrite of the 2017 first edition. It restructures the book around a “whole game” opening part, expands data transformation into dedicated chapters on numbers, strings, factors, dates, and missing values, adds new chapters on databases, spreadsheets, web scraping, and rectangling messy hierarchical data, and replaces R Markdown with Quarto throughout. The book is written for readers with no prior programming experience and is used as a core text in introductory data science and applied statistics courses.

Book Overview

Chapter List

Introduction

Difficulty: Easy · Key topics: the data-science cycle (import/tidy/transform/visualize/model/communicate), tidyverse core packages, what this book does not teach, R/RStudio prerequisites

The Introduction defines data science as turning raw data into understanding through a repeatable cycle — import, tidy, then an iterative loop of transform/visualize/model, ending in communicate, with programming wrapping around every step of it. It deliberately scopes the book down: no deep coverage of statistical modeling (it points readers to Tidy Modeling with R instead), a focus on small in-memory data rather than big data (recommending the separate data.table package for 10–100GB workloads), and R only — not Python or Julia, on the argument that it is best to master one data-science tool at a time. It closes by listing exactly what is needed to run the book’s code: R 4.2.0+, RStudio 2022.02.0+, and the tidyverse package bundle, and explains the book’s own typographic conventions for functions, objects, and package-qualified names.

Key Points:

• The data science cycle: import → tidy → (transform ⇔ visualize ⇔ model) → communicate, with programming wrapping the whole cycle
• Tidying means each column is a variable and each row is an observation — the definition the book returns to throughout Part 1
• library(tidyverse) loads nine core packages: dplyr, forcats, ggplot2, lubridate, purrr, readr, stringr, tibble, tidyr
• The book explicitly does not teach modeling in depth, big-data tools, or Python/Julia — it recommends Tidy Modeling with R for modeling and data.table for genuinely large data
• Recommended minimum versions: R 4.2.0+ and RStudio 2022.02.0+
• package::function() syntax (e.g. dplyr::filter()) is used whenever it matters which package a function comes from

Memory Tip: Remember the cycle as Import → Tidy → (Transform / Visualize / Model, looping) → Communicate, with Programming wrapped around the outside of the whole diagram.

Common Mistake: Assuming the book teaches importing and tidying first because those steps come first in the real-world pipeline — it deliberately teaches visualization and transformation first, on already-tidy bundled data, since tackling the more tedious 20% of real data wrangling before you have any motivating payoff is a bad place to start learning.

Important Questions:

• What are the six steps of the data science process described in this book’s Introduction? Import, tidy, then an iterative cycle of transform, visualize, and model, followed by communicate — with programming used throughout every step, not as a separate stage.
• Why does this book focus on R rather than teaching R and Python together? The authors believe it is best to master one data-science tool at a time and consider R a great place to start, even though they note most real data-science teams end up using a mix of languages in practice.

Part 1 §1.1–1.2: Data Visualization & Workflow: Basics

Difficulty: Easy · Key topics: ggplot2 grammar of graphics, aes() mappings vs. setting, geom_point()/geom_smooth(), facet_wrap(), assignment operator <-, snake_case, RStudio autocomplete

Chapter 1.1 opens the “Whole Game” walkthrough by building a scatterplot of the penguins dataset step by step: an empty ggplot(data = penguins) canvas gets an aes(x = flipper_length_mm, y = body_mass_g) mapping, then a geom_point() layer, then color/shape aesthetics for species and a geom_smooth(method = "lm") trend line, arriving at the chapter’s own “ultimate goal” plot. It then covers visualizing single-variable distributions (bar charts for categorical variables, histograms/density plots for numerical ones) and relationships between two or three variables (boxplots, faceting with facet_wrap()), finishing with ggsave(). Chapter 1.2 is the first workflow interlude: R as a calculator, the <- assignment operator (Alt/Option + −), snake_case naming, # comments, and calling functions like seq() with RStudio’s autocomplete.

Key Points:

• ggplot(data, mapping = aes(x = , y = )) plus a geom_*() layer is the base template for every ggplot2 chart
• Aesthetic mappings go inside aes(); a fixed color/size/shape applies outside aes(), as a plain geom argument
• Mapping color at the global ggplot() level colors every layer; mapping it only inside geom_point() keeps a geom_smooth() line from being split by group
• facet_wrap(~var) splits a plot into one subplot per level of a categorical variable
• <- is the assignment operator; object names use lowercase snake_case with underscores between words
• ggsave() saves the most recently displayed plot to disk, using dimensions from the current plotting device unless width/height are given

Practice Tip: Type the penguins scatterplot from scratch instead of copy-pasting it — build it up one layer at a time (bare canvas, then aes(), then geom_point(), then color, then a trend line) so each addition’s visual effect is obvious before you move on.

Common Mistake: Writing ggplot2 code with the + at the start of a new line instead of the end of the previous one — ggplot2 requires the + to trail the line it belongs to, or R treats the next line as a new, incomplete statement.

Important Questions:

• What’s the difference between mapping a variable to an aesthetic inside aes() versus setting it as a fixed value outside aes()? Mapping inside aes() makes the aesthetic vary based on a variable’s values (e.g. color by species); setting it outside aes() (e.g. color = "blue") applies one fixed value to every point regardless of the data.
• Why does R show a + prompt in the console instead of running your code? R thinks the current expression is incomplete — usually a missing closing ) or " — and is waiting for you to finish it; pressing Escape aborts the expression so you can start over.

Part 1 §1.3–1.4: Data Transformation & Workflow: Code Style

Difficulty: Easy-Medium · Key topics: dplyr verbs (filter/arrange/distinct/mutate/select/rename/relocate), the pipe |>, group_by()/summarize(), tidyverse style guide, styler package

Chapter 1.3 introduces dplyr using nycflights13::flights, covering row verbs filter() and arrange(), column verbs mutate(), select(), rename(), relocate(), the native pipe |> (read as “then”), and group operations group_by() + summarize() plus the slice_*() family, closing with a baseball case study on aggregation and sample size. Chapter 1.4 covers the tidyverse style guide: snake_case names, spacing around operators, pipe formatting (each verb on its own line, indented two spaces), and the styler package for auto-reformatting code via RStudio’s command palette.

Key Points:

• filter() keeps rows matching a condition; arrange() reorders rows; both leave the columns untouched
• Every dplyr verb takes a data frame first, describes columns by bare name, and returns a new data frame — dplyr functions never modify their input
• x |> f(y) is equivalent to f(x, y); chaining verbs with |> reads as a left-to-right sequence of steps
• group_by() + summarize() collapses each group to one row; n() inside summarize() counts rows per group
• slice_min()/slice_max() keep ties by default — set with_ties = FALSE for exactly one row per group
• The tidyverse style guide puts a space before |> and normally puts it at the end of a line

Practice Tip: Install styler once, then use RStudio’s command palette (Cmd/Ctrl+Shift+P, type “styler”) to auto-reformat any pipeline you write by hand — comparing the before/after is a fast way to absorb the style rules.

Common Mistake: Writing filter(month == 11 | 12) to mean “November or December” — R evaluates month == 11 first, then treats | 12 as | TRUE (any nonzero number is truthy), so every row passes; the fix is filter(month %in% c(11, 12)).

Important Questions:

• What does the pipe operator |> actually do in a dplyr pipeline? x |> f(y) takes the value on its left and inserts it as the first argument of the function on its right, so it is shorthand for f(x, y) — letting a sequence of transformations read top-to-bottom instead of nested inside-out.
• Why does a filter on a column with missing values sometimes drop rows you did not expect it to? A comparison against NA produces NA, not TRUE or FALSE, and filter() only keeps rows where the condition is exactly TRUE, so rows with missing values in the compared column silently disappear without a warning.

Part 1 §1.5–1.6: Data Tidying & Workflow: Scripts and Projects

Difficulty: Medium · Key topics: tidy data’s three rules, pivot_longer()/pivot_wider(), RStudio script editor, RStudio Projects, relative vs. absolute paths

Chapter 1.5 defines tidy data by its three rules (each variable a column, each observation a row, each value a cell) and teaches pivot_longer() for data spread across column names — the billboard and who2 examples, including the special ".value" sentinel — and pivot_wider() for data spread across rows, using the cms_patient_experience dataset, showing both reshape without changing the underlying values. Chapter 1.6 moves from console to script editor: the Cmd/Ctrl+Enter and Cmd/Ctrl+Shift+S shortcuts, three file-naming principles (machine-readable, human-readable, sortable), RStudio Projects for fixing the working directory, and why scripts should use relative, forward-slash paths instead of absolute ones.

Key Points:

• Tidy data’s three rules: each variable is a column, each observation is a row, each value is a cell
• pivot_longer(cols, names_to, values_to) turns column names into row values, for data like the billboard dataset’s 76 wk1–wk76 columns
• pivot_wider(names_from, values_from) turns row values into column names, and needs an id_cols that uniquely identifies each output row
• Cmd/Ctrl+Enter runs the current statement in the script editor; Cmd/Ctrl+Shift+S runs the whole script
• RStudio Projects fix the working directory to the project folder, so a path like data/diamonds.csv works the same on every collaborator’s machine
• Scripts should use relative, forward-slash paths only — never an absolute path or a Windows-style backslash path

Practice Tip: Create a real RStudio Project for this book (File → New Project), turn off workspace-preserving between sessions, and get in the habit of restarting R before re-running a script from scratch — it is the only reliable way to catch code that secretly depends on objects left over in memory.

Common Mistake: Naming files code.R, Final Report.docx, and FinalReport.docx in the same folder — inconsistent capitalization, spaces, and vague names like “final” make it hard to tell which file runs first or what each one contains; the book recommends numbered, lowercase, hyphenated names instead.

Important Questions:

• What are the three rules that make a dataset “tidy”? Each variable must be its own column, each observation its own row, and each value its own cell — the same underlying data can be arranged in several valid ways, but only one arrangement is tidy.
• Why does the book recommend relative paths over absolute paths in scripts? An absolute path like /Users/hadley/Documents/r4ds/data/diamonds.csv only works on the machine it was written on; a relative path like data/diamonds.csv, combined with an RStudio Project setting the working directory, works identically for anyone who opens the project.

Part 1 §1.7–1.8: Data Import & Workflow: Getting Help

Difficulty: Easy · Key topics: read_csv(), col_types guessing, na argument, skip/comment, write_csv()/write_rds(), tibble()/tribble(), reprex, Google/Stack Overflow

Chapter 1.7 covers reading rectangular text files with read_csv(), including fixes for messy real files: the na argument for nonstandard missing-value strings, backtick-quoted non-syntactic column names, skip/comment for metadata rows, and readr’s nine col_types() helpers for overriding its type-guessing heuristic. It covers reading many files at once with an id column, writing back out with write_csv()/write_rds()/parquet, and hand-entering small tibbles with tibble() (by column) or tribble() (by row). Chapter 1.8 closes the Whole Game part with practical advice on getting help: searching Google/Stack Overflow (including forcing English error messages with Sys.setenv(LANGUAGE = "en")), and building a minimal reproducible example (a “reprex”) with the reprex package before asking anyone for help.

Key Points:

• read_csv() guesses each column’s type by sampling 1,000 evenly spaced values; override a wrong guess with the col_types argument
• The na argument tells read_csv() which extra strings (e.g. "N/A") count as missing, beyond the default empty string
• write_csv() loses column-type information on save; write_rds() (or parquet) preserves the exact R object
• tibble() builds a data frame column by column; tribble() lays it out row by row with ~name headers, easier to eyeball for small hand-entered data
• A good reprex is reproducible (includes every library() call and needed object) and minimal (stripped down to just what reveals the bug)
• 80% of the time, the act of writing a proper reprex reveals the bug yourself, before you ever post it anywhere

Practice Tip: Before asking for help on a real bug, run reprex::reprex() on your minimal example first — even if you never post it, its Markdown output is the fastest way to confirm your example is truly self-contained.

Common Mistake: Reporting that readr “read my numbers as text” without checking problems() first — read_csv() reports exactly which row and column failed to parse as the guessed type, which is usually enough to spot a stray non-numeric missing-value code like . or 99.

Important Questions:

• What does the na argument to read_csv() do, and when do you need it? It lists extra strings that should be treated as missing values beyond the default empty string, such as na = "N/A", and it is needed whenever a CSV encodes missing data with a text placeholder instead of leaving the cell blank.
• What are the two requirements of a good reprex? It must be reproducible — including every library() call and any data needed to run it standalone — and minimal — stripped down to only the code that actually demonstrates the problem.

Part 2 §2.1: Layers

Difficulty: Medium · Key topics: aesthetic mapping vs. setting, facet_grid(), stat_*()/default geoms, position adjustments (stack/fill/dodge/jitter), coord_polar()/coord_quickmap(), 7-parameter grammar template

Expands the grammar of graphics beyond Chapter 1.1, covering more than 40 ggplot2 geoms and facet_grid(rows ~ cols) for faceting by two variables at once. It explains statistical transformations — every geom has a default stat, e.g. geom_bar() uses stat_count() — the position adjustments for bar charts ("identity", "fill", "dodge", in addition to the stacking default) and position = "jitter"/geom_jitter() for scatterplot overplotting, and coordinate systems (coord_quickmap(), coord_polar()). It closes by assembling all seven parameters — data, geom, mappings, stat, position, coordinate system, facets — into one master ggplot2 template.

Key Points:

• Every geom has a default stat and every stat a default geom — geom_bar() uses stat_count() by default, computing count and prop
• facet_grid(rows ~ cols) facets by two variables at once in a grid, versus facet_wrap()‘s single-variable wrapping layout
• Bar chart position adjustments: "stack" (default), "fill" (equal-height, good for proportions), "dodge" (side by side)
• geom_jitter() is shorthand for geom_point(position = "jitter"), adding random noise to fight overplotting on scatterplots
• The group aesthetic draws a separate object per level without adding a legend — useful for un-labeled multi-line plots
• The full grammar template has seven parameters: data, geom, mappings, stat, position, coordinate system, and facets

Practice Tip: Take a bar chart made with the default position = "stack" and re-render it with "identity", "fill", and "dodge" back to back — seeing the same data reshape four different ways is the fastest way to internalize what a position adjustment actually does.

Common Mistake: Forgetting group = 1 on a proportion bar chart (y = after_stat(prop)) — without it, ggplot2 computes the proportion within each bar’s own group (always 100%) instead of across the whole dataset, silently producing a flat, meaningless chart.

Important Questions:

• What is a “stat” in ggplot2, and why does geom_bar() need one? A stat (statistical transformation) is the algorithm that computes new values to plot from raw data — geom_bar()‘s default stat, stat_count(), counts how many rows fall into each bar rather than plotting a raw column, since “count” is not a variable that exists in the original dataset.
• When should you reach for position = "jitter" instead of leaving points at the default? When many points in a scatterplot share the same rounded x/y values and overlap into indistinguishable clusters (overplotting) — jitter adds small random noise so the true density of points becomes visible, at the cost of slightly inaccurate exact positions.

Part 2 §2.2: Exploratory Data Analysis

Difficulty: Medium · Key topics: EDA as an iterative cycle, variation vs. covariation, coord_cartesian() zooming, geom_freqpoly()/geom_boxplot() for a categorical/numerical pair, geom_tile() for two categorical variables, modeling out a confound

Frames exploratory data analysis as an iterative loop — ask a question, answer it with a plot/transformation/model, use the answer to ask a sharper question — organized around two guiding questions: what variation exists within a variable, and what covariation exists between variables. It walks through spotting typical values and outliers in a histogram of diamonds$carat (using coord_cartesian() to zoom the y-axis without discarding data, unlike ylim()), then handling unusual values by converting them to NA rather than dropping whole rows. For covariation it compares a categorical and numerical variable with geom_freqpoly()/boxplots — discovering that Fair-cut diamonds are, counterintuitively, the most expensive on average — two categorical variables with geom_tile(), and two numerical variables with geom_bin2d()/geom_hex(), closing with a residual model that strips out the price/carat relationship to reveal the true cut/price pattern.

Key Points:

• EDA’s two guiding questions: what variation occurs within a variable, and what covariation occurs between variables
• coord_cartesian(ylim = ...) zooms into a plot region without discarding data; ggplot2’s own ylim()/xlim() throw away out-of-range data entirely
• Diamonds with a y (width) of 0mm reveal missing data coded as 0 — the kind of problem EDA catches that a plain is.na() search would miss
• geom_freqpoly()/boxplots comparing price by cut reveal that Fair-cut diamonds are, on average, the most expensive — a counterintuitive pattern only explained once carat is controlled for
• geom_tile() with a computed count/fill visualizes covariation between two categorical variables as a heatmap
• Fitting a model to log(price) ~ log(carat) and plotting its residuals removes the dominant carat effect, so a subtler pattern (cut vs. price) becomes visible

Practice Tip: When a plot suggests a surprising pattern, immediately run the book’s own three follow-up questions — could this be coincidence, how strong is the relationship, does it hold within subgroups — before treating the pattern as a real finding.

Common Mistake: Silently dropping every row with an outlier value instead of just the bad value — the book recommends replacing only the suspect value with NA via mutate() + if_else(), since one bad measurement in a row does not make the rest of that row’s data invalid too.

Important Questions:

• Why does coord_cartesian(ylim = c(0, 50)) reveal outliers that a default histogram hides, while ylim(0, 50) would not help? coord_cartesian() only changes what is displayed, zooming the axis while keeping every bar’s true height as computed; ylim() first discards all data outside the range before the histogram is even built, which can distort bin counts near the edges.
• What does it mean that Fair-cut diamonds are more expensive on average, and how does the chapter explain it? Cut and carat are correlated in the raw data (larger, pricier stones are less often cut to the highest quality), which confounds a simple cut-vs-price comparison; once the book models out the carat/price relationship and inspects the residuals, the expected pattern — better cuts cost more for a given size — reappears.

Part 2 §2.3: Communication

Difficulty: Medium · Key topics: labs(), geom_text_repel()/annotate(), scale breaks/labels, legend position/guides(), ColorBrewer & viridis palettes, three ways to zoom, theme(), patchwork package

Shifts from exploratory plots (made for yourself) to expository plots (made to communicate to others), starting with labs() for titles/subtitles/captions/axis labels, then geom_text()/geom_label()/annotate() for point labels, recommending ggrepel’s geom_text_repel()/geom_label_repel() to auto-avoid overlapping labels. It covers scale customization via breaks/labels (including scales::label_dollar()/label_percent()), legend placement with theme(legend.position = ) and guides(), and replacing default color scales with ColorBrewer or viridis palettes chosen for colorblind accessibility. It contrasts the three ways to zoom a plot — subsetting the data, setting scale limits, or coord_cartesian() — covers theme() for non-data plot elements, and introduces the patchwork package’s +/|// operators for combining several plots into one layout.

Key Points:

• labs(title = , subtitle = , caption = , x = , y = ) sets a plot’s text; a good title summarizes the finding, not just describes the chart type
• ggrepel’s geom_label_repel()/geom_text_repel() automatically reposition overlapping point labels
• breaks controls where axis ticks/legend keys fall; labels controls what text appears at each one
• Three ways to zoom: subset the data, set a scale’s limits (equivalent to subsetting — recomputes any smooth), or use coord_cartesian(xlim = , ylim = ) to zoom only the display
• ColorBrewer and viridis color scales are chosen to remain distinguishable under common forms of color blindness
• The patchwork package overloads + (side by side), | and / (explicit row/column layout) to combine separate ggplot objects into one figure

Practice Tip: Take a plot with a geom_smooth() line, then zoom it two ways — once by shrinking a scale’s limits, once with coord_cartesian() — and compare how the fitted line’s shape changes in one case but not the other.

Common Mistake: Zooming a plot by shrinking a continuous scale’s limits when a geom_smooth() line is present — because setting limits discards out-of-range data before any statistics are computed, the smoothed line gets refit on the smaller dataset and its shape changes, unlike coord_cartesian(), which zooms only the display.

Important Questions:

• What’s the practical difference between setting a scale’s limits and using coord_cartesian() to zoom into a plot? Setting limits discards data outside that range before any statistics (like a smooth line) are computed, changing the result; coord_cartesian() only changes what part of the already-computed plot is displayed, leaving every underlying calculation untouched.
• Why does the book recommend ggrepel’s geom_text_repel() over plain geom_text() for labeling multiple points? Plain geom_text() places labels exactly at each point’s coordinates, so nearby points end up with overlapping, unreadable labels; geom_text_repel() automatically nudges labels apart while keeping a line back to the original point.

Part 3 §3.1: Logical Vectors

Difficulty: Medium · Key topics: comparison operators, dplyr::near(), NA contagion, is.na(), Boolean algebra (&/|/!), %in%, any()/all(), sum()/mean() on logicals, if_else()/case_when()

Covers the three possible values of a logical vector (TRUE, FALSE, NA) and how comparisons like <, ==, != create them. It flags floating-point comparison traps (sqrt(2)^2 == 2 is FALSE; use dplyr::near()), explains why NA is “contagious” in comparisons and why is.na() — not == NA — is needed to find missing values, and covers Boolean algebra (&, |, !, xor()) with its own NA rules and an order-of-operations trap in expressions like nov | 12. It shows %in% as a safer alternative to chained |/==, logical summaries (any()/all(), and sum()/mean() since TRUE/FALSE become 1/0 in numeric context), and closes with conditional transformation via if_else() and case_when().

Key Points:

• == on floating-point numbers can fail unexpectedly (sqrt(2)^2 == 2 is FALSE) because of stored rounding error; use dplyr::near() instead
• NA is “contagious”: almost any comparison or arithmetic involving NA produces NA, so dep_time == NA never finds missing rows — use is.na()
• nov | 12 is not “November or 12” — R evaluates month == 11 first, then treats | 12 as | TRUE since any nonzero number becomes TRUE
• %in% avoids the |/== ordering trap; x %in% y is TRUE wherever a value of x appears anywhere in y, and NA %in% NA is TRUE (unlike ==)
• sum(x) on a logical vector counts TRUEs; mean(x) gives the proportion TRUE, since TRUE/FALSE coerce to 1/0
• if_else(condition, true, false, missing) handles two branches; case_when(condition ~ output, ...) handles many, using the first matching condition and .default for a catch-all

Practice Tip: Before trusting any filter() that uses | across several equality checks, rewrite the same condition with %in% and confirm you get the same row count — the two should agree unless the | version has an operator-precedence bug.

Common Mistake: Writing filter(month == 11 | 12), believing it reads like English “November or 12” — R actually computes month == 11, then ORs that logical vector with the number 12 (which becomes TRUE), matching every row.

Important Questions:

• Why does flights |> filter(dep_time == NA) return zero rows even when dep_time has missing values? Comparing anything to NA with == produces NA, not TRUE, because R does not know whether an unknown value equals another value; filter() only keeps rows where the condition is exactly TRUE, so is.na(dep_time) must be used instead.
• What’s the difference between if_else() and case_when()? if_else(condition, true, false) handles exactly one binary decision; case_when() takes any number of condition ~ output pairs and uses the first one that matches for each row, making it the right tool once there are more than two possible outcomes.

Part 3 §3.2: Numbers

Difficulty: Medium · Key topics: parse_double()/parse_number(), count()/n_distinct(), recycling rules, %/% and %%, round() banker’s rounding, cut(), cumsum(), min_rank()/lead()/lag(), mean()/median()/quantile()/IQR()

Covers turning strings into numbers with parse_double()/parse_number(), dplyr’s count() as a fast shortcut for group_by() + summarize() + n(), and R’s vector recycling rule, which silently repeats a shorter vector and can produce a wrong-but-error-free result when comparing a vector against a column with an even row count. It covers numeric transformations — pmin()/pmax() versus min()/max(), %/%/%% modular arithmetic (splitting a packed sched_dep_time column into hour/minute), log2()/log10(), round()‘s “round half to even” banker’s rounding, cut() for binning, and cumsum()/rolling aggregates — plus general transformations like min_rank(), lead()/lag(), and consecutive_id(). It closes with numeric summaries: mean vs. median for skewed data, quantile(), sd()/IQR() for spread, and position extractors first()/last()/nth().

Key Points:

• count(df, var, sort = TRUE) is dplyr’s shortcut for “group by and count”; n_distinct() counts unique values instead
• R’s recycling rule silently repeats a shorter vector to match a longer one, which can make a two-element comparison vector match wrong rows without any error if the column happens to have an even length
• %/% is integer division and %% is the remainder — together they split a packed time value like 1430 into hour 14 and minute 30
• round() uses “round half to even” (Banker’s rounding): round(0.5) gives 0, not 1, keeping repeated rounding statistically unbiased
• min_rank() handles ties like “1st, 2nd, 2nd, 4th”; lead()/lag() shift a vector to compare current vs. adjacent values
• For skewed distributions like flight delays, median() is usually the more representative center than mean()

Memory Tip: Read %/% and %% as “slash-slash gives the whole number of times it divides, percent-percent gives what’s left over” — 1430 %/% 100 gives hour 14, and 1430 %% 100 gives minute 30.

Common Mistake: Using min()/max() where pmin()/pmax() was needed — min()/max() collapse an entire vector to one overall value, while pmin()/pmax() compare vectors element-by-element and return a vector the same length as the input; using the wrong one gives every row the identical value.

Important Questions:

• What is R’s “recycling rule”, and why can it silently break a comparison? When two vectors of different lengths are compared or combined, R repeats (recycles) the shorter one until it matches the longer one’s length; if this happens without a warning against an even-length column, it can silently apply the wrong element to the wrong rows instead of erroring.
• Why does round(2.5) give 2 instead of 3? R’s round() uses “round half to even” (Banker’s rounding), where a value exactly halfway between two integers rounds to the nearer even integer, keeping repeated rounding statistically unbiased rather than always rounding .5 up.

Part 3 §3.3: Strings

Difficulty: Medium · Key topics: str_c()/str_glue(), raw strings, separate_wider_delim()/separate_longer_delim(), str_length()/str_sub(), UTF-8 encoding, locale-dependent casing/sorting

Covers creating strings (single vs. double quotes, \ escapes, and R 4.0+ raw strings like r"(...)" for avoiding “leaning toothpick syndrome”), then building strings from data with str_c() (tidyverse-recycling-aware, like paste0()) and str_glue() ({}-interpolated) versus flattening a vector to one string with str_flatten(). It covers extracting data crammed into one column with the separate_longer_delim()/separate_wider_delim()/separate_wider_position() family, including too_few/too_many debug modes, working with individual letters via str_length()/str_sub() (negative indices count from the end), and closes with non-English text: encodings (readr always assumes UTF-8), letter variations (a single ü character vs. u plus a combining diaeresis), and locale-dependent case/sort behavior (stringr defaults to the “en” locale unless told otherwise).

Key Points:

• Raw strings (r"(...)", R 4.0+) let you write literal backslashes and quotes without escaping — useful for regex or Windows paths
• str_c() recycles and propagates NA the tidyverse way; str_glue("Hi {name}") interpolates any {}-wrapped R expression into a string
• separate_wider_delim(col, delim, names) splits one column into several by a delimiter; too_few = "debug" reveals which rows did not split as expected
• str_sub(string, start, end) extracts a substring by position; negative positions count backward from the end and never error on a too-short string
• readr always assumes UTF-8; guess_encoding() helps diagnose garbled text from older, non-UTF-8 files
• stringr defaults to the “en” locale for case-changing and sorting so results do not silently vary by the user’s own OS settings

Practice Tip: Deliberately break a separate_wider_delim() call on messy data (some rows with too few pieces, some with too many) and read the too_few = "debug"/too_many = "debug" output columns before deciding whether to fix the data or the split.

Common Mistake: Assuming a printed string’s escapes (like \n) are its actual contents — copying a printed representation back into code can double-escape characters; str_view(), not print(), shows what a string really contains.

Important Questions:

• What problem do raw strings (r"(...)", R 4.0+) solve? They let you write a string containing many literal backslashes or quotes — like a Windows path or a regex — without escaping each special character with \, avoiding the “leaning toothpick syndrome” of stacked backslashes.
• Why might str_length() and str_sub() behave unexpectedly on accented non-English text? An accented letter like “ü” can be encoded as either one single character or two characters (a base letter plus a separate combining diacritic mark) that look identical when printed but have different lengths, so position-based functions can disagree on where a letter starts or ends.

Part 3 §3.4: Regular Expressions

Difficulty: Hard · Key topics: metacharacters, str_detect()/str_count()/str_replace(), quantifiers, anchors (^ $ \b), character classes (\d \s \w), grouping/backreferences, regex() flags, separate_wider_regex()

Builds regex from the ground up with str_view(), covering literal characters vs. metacharacters, quantifiers (?, +, *, {n}/{n,}/{n,m}), character classes ([abcd], [^abcd]), and alternation (|). It covers str_detect(), str_count(), str_replace()/str_replace_all(), and separate_wider_regex() for extracting several pieces from one messy column, then goes deeper into escaping, anchors (^, $, \b word boundary), character class shortcuts (\d/\D, \s/\S, \w/\W), operator precedence (quantifiers bind tighter than alternation), and grouping/capturing with backreferences (\1). It finishes with regex flags via regex() (ignore_case, dotall, multiline, comments), fixed()/coll() for opting out of regex entirely, and three problem-solving techniques: positive/negative test cases, combining regex with Boolean algebra instead of one giant pattern, and constructing a pattern programmatically from a known vector of values.

Key Points:

• Quantifiers: ? (0 or 1), + (1 or more), * (0 or more), {n}/{n,}/{n,m} for exact/minimum/range counts
• To match a literal ., you need the regex \., which as an R string requires "\\." — or use raw strings to skip one layer of escaping
• ^ anchors to the start of a string, $ to the end, \b to a word boundary — \bsum\b avoids also matching summarize or rowsum
• Character class shortcuts: \d digit, \s whitespace, \w word character (and their uppercase negations)
• Quantifiers bind tighter than alternation: ab+ means a(b+), and ^a|b$ means (^a)|(b$) — use parentheses to override
• str_count() matches never overlap — str_count("abababa", "aba") returns 2, not 3, since each match consumes its characters

Practice Tip: For any pattern more complex than a single character class, write down 2–3 strings it should match and 2–3 it should not, then test the pattern against all of them with str_view() before trusting it on real data.

Common Mistake: Trying to build one giant regex for a logical “and” (e.g. a word containing both “a” and “b”) — there is no native “and” operator in regex, so an unreadable alternation of orderings gets written instead of simply combining two separate str_detect() calls with &.

Important Questions:

• Why does the pattern ^a|b$ not mean “starts with a and ends with b”? Alternation has lower precedence than anchors, so ^a|b$ is actually parsed as (^a)|(b$) — it matches any string that starts with “a” OR any string that ends with “b”, not strings that do both; parentheses are needed to combine them correctly.
• What is the difference between str_detect() and str_count()? str_detect() returns one TRUE/FALSE per string, telling you whether the pattern matches at all; str_count() returns how many non-overlapping times the pattern matches within each string, which can be more than one.

Part 3 §3.5: Factors

Difficulty: Medium · Key topics: factor()/forcats::fct(), levels(), gss_cat dataset, fct_reorder()/fct_relevel()/fct_infreq(), fct_recode()/fct_collapse()/fct_lump_n(), ordered factors

Motivates factors with a month variable stored as a string, which allows typos and sorts alphabetically instead of chronologically, then shows factor(x, levels) and the safer forcats::fct(), which orders by first appearance rather than assuming alphabetical sort order is universal. Using the forcats::gss_cat General Social Survey dataset, it covers reordering levels with fct_reorder() (for arbitrarily-ordered factors like religion, plotted against average TV hours), fct_relevel() (pulling a special level like “Not applicable” to the front without disturbing a principled order), and fct_infreq() for bar-chart frequency order. It covers changing level values with fct_recode() (renaming individual levels), fct_collapse() (merging many old levels into fewer new ones), and the fct_lump_*() family for lumping small categories into “Other”, closing with ordered factors (ordered()), which print with < between levels and default to a ranked (viridis) color scale.

Key Points:

• A string-typed categorical variable allows typos and sorts alphabetically, not logically — factor(x, levels) fixes both, and any out-of-level value becomes NA
• forcats::fct() orders levels by first appearance in the data instead of assuming alphabetical order (which is not the same on every system)
• fct_reorder(.f, .x, .fun = median) reorders a factor by another variable — reserve it for arbitrarily-ordered factors, not ones with a real built-in order like income brackets
• fct_relevel() moves specific named levels to the front without reordering the rest
• fct_recode(f, "New Name" = "old_name") renames individual levels; fct_collapse() merges several old levels into one new one at once
• Ordered factors (ordered()) print with < between levels and default to scale_color_viridis() in ggplot2, implying a ranking

Memory Tip: “Recode renames, collapse combines, lump limits” — fct_recode() changes labels one-to-one, fct_collapse() merges groups many-to-one, and fct_lump_*() caps the total category count by rolling small ones into “Other”.

Common Mistake: Applying fct_reorder() to a factor that already has a principled order (like income brackets or education levels) — arbitrarily resorting a factor that already encodes a real sequence destroys that meaning; fct_reorder() is only for factors with no inherent order.

Important Questions:

• Why is forcats::fct() recommended over base R’s factor() when you do not explicitly supply levels? factor() without explicit levels sorts them alphabetically, but not every computer sorts strings identically, making results non-portable; fct() instead orders levels by the order they first appear in the data, which is deterministic everywhere.
• What’s the difference between fct_recode() and fct_collapse()? fct_recode() changes the label of individual levels one at a time (new name on the left, old name on the right); fct_collapse() is built for merging — you supply a vector of several old level names to collapse into one new combined level.

Part 3 §3.6: Dates and Times

Difficulty: Hard · Key topics: date/time/date-time types, ymd()/ymd_hms() lubridate helpers, make_datetime(), year()/month()/wday() accessors, floor_date()/round_date(), durations vs. periods vs. intervals, IANA time zones

Distinguishes R’s three date/time types (date, time, date-time/POSIXct) and covers the four ways to create them: automatically during read_csv() import of ISO8601 strings, from strings via lubridate’s ymd()/mdy()/ymd_hms() family (named for the order of components in the input), from separate numeric columns via make_date()/make_datetime() (used to reconstruct proper timestamps from packed HHMM-format columns), and by converting between types with as_date()/as_datetime(). It covers extracting components with year()/month()/wday()/hour() accessors, rounding with floor_date()/round_date()/ceiling_date(), and the three time-span classes — durations (exact seconds; can misbehave across a DST boundary), periods (human units, behaving intuitively across DST), and intervals (a start/end pair, for the most accurate span calculations) — closing with time zones, where R always uses IANA {area}/{location} names and treats the zone as only a display attribute of the same underlying instant.

Key Points:

• Three date/time types: date (<date>), time within a day (<time>, needs the hms package), date-time (<dttm>, a.k.a. POSIXct)
• lubridate helper names encode the order of components in your input string: ymd() parses “year-month-day” order, mdy() parses “month-day-year”, and so on
• make_datetime() builds a date-time from separate numeric year/month/day/hour/minute columns, using %/%/%% to unpack packed HHMM time columns
• Durations always represent an exact number of seconds and can give surprising results across a daylight-saving boundary
• Periods use “human” units (days, months) rather than fixed seconds, so date + days(1) behaves intuitively even across DST
• R always identifies time zones by IANA names like "America/New_York", never ambiguous local abbreviations like “EST” — the zone is only a display attribute of the same underlying instant

Memory Tip: “Duration = seconds, Period = calendar, Interval = a measured span” — pick a duration for pure physical time, a period when “add one human day/month” should behave the way a person expects, and an interval when you need the exact, unambiguous length between two real moments.

Common Mistake: Adding days(1)‘s duration counterpart, ddays(1), to a date-time near a DST transition and expecting exactly 24 hours later — because DST removes or adds an hour on the transition day, a duration-based “one day later” can land at an unexpected clock time; a period-based days(1) behaves the way a calendar expects instead.

Important Questions:

• What is the practical difference between a lubridate duration and a lubridate period? A duration always represents an exact number of seconds, so adding one can produce a surprising clock time across a Daylight Saving Time change; a period represents “human” units like days or months and adjusts for things like DST, so date + days(1) reliably means “the same clock time, tomorrow.”
• Why does R require IANA time zone names like "America/New_York" instead of abbreviations like “EST”? Common abbreviations are ambiguous — both Australia and Canada have their own “EST” — while the IANA {continent}/{city} scheme is unique worldwide and, because city names rarely change, can also encode a location’s full history of daylight-saving rule changes.

Part 3 §3.7: Missing Values

Difficulty: Medium · Key topics: tidyr::fill(), coalesce()/na_if(), NaN vs. NA, explicit vs. implicit missingness, complete(), anti_join() for cross-table missingness, .drop = FALSE for empty factor groups

Covers explicit missing values (a visible NA in a cell) — tidyr::fill() for “last observation carried forward” data entry, coalesce() for replacing NA with a known fixed value, na_if() for the opposite (turning a sentinel like 99 into NA), and NaN as a distinct “not a number” case from an indeterminate calculation like 0/0. It introduces implicit missing values — an entire absent row, described as “the absence of a presence” — and shows how pivoting, tidyr::complete(), and anti_join() each reveal implicit gaps, using anti_join() across flights/planes/airports to surface genuinely missing reference data. It closes with a third kind of missingness: empty factor groups that count()/group_by() drop by default, fixed with .drop = FALSE, and the resulting NaN/-Inf/Inf behavior of summary functions on zero-length vectors.

Key Points:

• tidyr::fill(cols) implements “last observation carried forward,” filling missing values with the value from the row above
• coalesce(x, 0) replaces NA with a known value; na_if(x, 99) does the reverse, turning a sentinel value like 99 into a proper NA
• NaN (“not a number”) arises from indeterminate math like 0/0, distinct from NA; is.nan() distinguishes the two when it matters
• Explicit missingness is a visible NA in the data; implicit missingness is an entire row that is simply absent — “the absence of a presence”
• tidyr::complete(df, var1, var2) makes implicit gaps explicit by generating every combination of the named variables that should exist
• anti_join(x, y) finds rows in x with no match in y — a practical way to discover implicitly missing reference data across two tables

Practice Tip: Before assuming a count of zero for some group is real, re-run the same count() with .drop = FALSE for factors — it will surface any group that is silently missing from the raw data rather than genuinely zero.

Common Mistake: Assuming is.na(x) alone always distinguishes an unknown value from an indeterminate one — is.na() actually returns TRUE for both NA and NaN in most contexts, which is exactly why is.nan() exists for the cases where the distinction genuinely matters.

Important Questions:

• What is the difference between an explicit and an implicit missing value? An explicit missing value is a cell that literally contains NA — “the presence of an absence” — while an implicit missing value is a whole row that simply does not appear in the dataset — “the absence of a presence” — and only becomes visible once you compare against what should exist.
• How can anti_join() help find missing values that never show up as NA? anti_join(x, y) returns every row of x with no matching key in y; run against related tables, it surfaces tail numbers or airports referenced in one table but genuinely absent from the other — a kind of missingness a simple is.na() check can never catch.

Part 3 §3.8: Joins

Difficulty: Hard · Key topics: primary/foreign/surrogate keys, left_join()/inner_join()/right_join()/full_join(), join_by(), semi_join()/anti_join(), many-to-many warning, non-equi joins (cross/inequality/rolling/overlap)

Introduces primary keys (a variable or compound key that uniquely identifies each row) and foreign keys (a matching variable in another table), and shows checking a primary key’s uniqueness with count() plus a check for missing values. It covers the four mutating joins — left_join() (keeps all rows of x, used almost always), right_join(), full_join(), and inner_join() — and explicit key specification with join_by() for cases where two tables share a column name with different meanings, like year in flights vs. planes. It covers the two filtering joins, semi_join() (keep matched rows, never duplicates) and anti_join() (keep unmatched rows), and the “many-to-many” warning dplyr raises when a row in x matches more than one row in y. It finishes with non-equi joins — cross joins (Cartesian product), inequality joins (<, >=, etc.), rolling joins (closest(), matching only the nearest row), and overlap joins (between(), within(), overlaps() for interval matching) — worked through a party-planning/employee-birthday example.

Key Points:

• A primary key uniquely identifies each row in its own table; a foreign key is a variable in another table that points to that primary key
• left_join() keeps every row of x, filling unmatched rows’ new columns with NA — it is the join used almost all of the time
• join_by(tailnum) is short for join_by(tailnum == tailnum); when two tables share a column name with different meanings, an explicit join_by() avoids an accidental compound-key join
• semi_join(x, y) filters x to rows with a match in y without duplicating rows; anti_join(x, y) filters x to rows with no match in y
• When a row in x matches more than one row in y, dplyr emits a “many-to-many” warning and duplicates that row once per match
• Rolling joins (join_by(closest(x <= y))) match only the single closest row instead of every row satisfying an inequality — useful for “most recent event before this date” problems

Memory Tip: “Left keeps left, inner keeps only the overlap, full keeps everything” — picture the classic Venn diagram: inner_join() is the intersection, full_join() is the union, and left_join()/right_join() each keep one whole circle plus whatever from the other circle matches it.

Common Mistake: Joining two tables on every shared column name by default (a “natural join”) without checking what each shared name means — flights and planes both have a year column, but one means year of departure and the other means year the plane was built, so an unqualified join silently produces near-zero matches instead of erroring.

Important Questions:

• Why does left_join(flights2, planes) return far fewer matches than expected if both tables have a year column? Without an explicit join_by(), dplyr uses every shared column as the join key by default, joining on tailnum AND year together — but the two year columns mean different things and almost never match for the same plane, so most rows fail to match.
• What’s the difference between semi_join() and inner_join() when a row in x matches multiple rows in y? inner_join() duplicates the row in x once for every matching row in y, potentially growing the row count; semi_join() only checks whether at least one match exists and keeps that row of x exactly once, never duplicating and never adding columns from y.

Part 4 §4.1: Spreadsheets

Difficulty: Easy-Medium · Key topics: readxl::read_excel(), skip/col_names/na/col_types arguments, reading multiple worksheets, cell ranges, writexl::write_xlsx(), googlesheets4::read_sheet()

Covers reading Excel files with readxl::read_excel(), working through a messy students spreadsheet to fix inconsistent column names (col_names), a misrecognized "N/A" string (na), and a mixed numeric/text age column — read as "text" then converted after the fact, since specifying col_types = "numeric" upfront silently turns the one bad cell into NA. It covers reading a specific worksheet with the sheet argument or discovering all sheet names with excel_sheets(), then combining several same-shaped worksheets with bind_rows(); reading only a cell range like "A5:F15" to skip surrounding presentation text; the four underlying Excel cell data types (boolean, number, datetime, text) that do not always match what is displayed; and writing back out with writexl::write_xlsx(). It closes with the parallel googlesheets4::read_sheet()/write_sheet() workflow for Google Sheets, including public (gs4_deauth()) versus authenticated access.

Key Points:

• read_excel() (and format-specific read_xls()/read_xlsx()) is readxl’s function for loading Excel spreadsheets, mirroring read_csv()‘s syntax
• Reading a genuinely mixed column as col_types = "numeric" upfront silently converts bad values to NA — read it as "text" first, then convert deliberately
• excel_sheets() lists all worksheet names in a file; the sheet argument to read_excel() selects one to read
• The range argument (e.g. "A5:F15") reads a fixed cell range, avoiding decorative text above/below the real data table
• Excel stores every number as a floating point and every date as a day-count from 1900-01-01 — what is displayed can differ from what is actually stored
• googlesheets4::read_sheet() mirrors read_excel()‘s interface; reading a public sheet needs gs4_deauth(), while writing or reading a private sheet requires gs4_auth()

Practice Tip: When a spreadsheet import does not look right, open a copy of the actual Excel/Sheets file and look at it directly first — unlike a CSV, a spreadsheet’s real formatting and cell types are not visible from the printed R output alone.

Common Mistake: Setting col_types = "numeric" on an Excel column with just one or two non-numeric entries (like a typo "five" instead of 5) — this silently turns those specific cells into NA instead of erroring, so the mistake is easy to miss unless row counts are compared before and after.

Important Questions:

• Why does the book recommend reading a spreadsheet’s messy age column as col_types = "text" rather than "numeric", even though age should be numeric? When a column has one non-numeric entry (like the string “five”), forcing col_types = "numeric" on import silently converts just that cell to NA with no error; reading it as text first lets you see and fix the actual bad value, then convert the whole column deliberately.
• What is the key underlying difference between how a CSV and an Excel file store data types? A CSV file has no type information at all — everything is plain text and readr must guess each column’s type from the values; an Excel file stores each cell as one of four real underlying types (boolean, number, datetime, or text) that can differ from what is visually displayed, such as a number formatted to look like a date.

Part 4 §4.2: Databases

Difficulty: Hard · Key topics: DBI::dbConnect(), client-server/cloud/in-process DBMS, duckdb, dbplyr lazy tables, tbl()/show_query()/collect(), SQL clauses, SQL evaluation order

Explains databases as collections of indexed, disk-stored tables and the three kinds of database management system — client-server (PostgreSQL, run on a shared server), cloud (BigQuery/Snowflake), and in-process (duckdb/SQLite, running entirely on one computer) — then connects to an in-memory duckdb database with DBI::dbConnect() and loads sample data with dbWriteTable(). It introduces dbplyr, a dplyr backend that translates familiar dplyr verbs into SQL: tbl() creates a lazy table reference that records operations without executing them, show_query() reveals the generated SQL, and collect() finally pulls the result into an in-memory tibble. It then teaches SQL itself through that translation — the five key clauses SELECT/FROM/WHERE/GROUP BY/ORDER BY, noting SQL’s clause-writing order differs from its actual evaluation order, how dplyr verbs map onto them, NULL vs. NA semantics, and when dbplyr needs a subquery, closing with SQL joins and a brief tour of function translation.

Key Points:

• Three DBMS categories: client-server (PostgreSQL, run on a shared server), cloud (BigQuery/Snowflake), and in-process (duckdb/SQLite, running entirely on your own computer)
• tbl(con, "table") creates a lazy dbplyr object — writing dplyr verbs against it builds up a query without executing it until collect() is called
• show_query() prints the actual SQL dbplyr generated from your dplyr pipeline — a practical way to learn SQL by comparison
• SQL clauses are written in the order SELECT, FROM, WHERE, GROUP BY, ORDER BY, but are actually evaluated in the order FROM → WHERE → GROUP BY → SELECT → ORDER BY
• SQL uses = for comparison (not ==), single quotes for strings (not double), and NULL instead of NA (silently dropped when summarizing, unlike NA)
• Filtering on a variable just created with summarize() produces a HAVING clause instead of WHERE, because WHERE is evaluated before SELECT/GROUP BY

Memory Tip: SQL’s written order runs SELECT, FROM, WHERE, GROUP BY, ORDER BY but its evaluation order runs FROM, WHERE, GROUP BY, SELECT, ORDER BY — the mismatch between the two orders is exactly why filtering a computed alias needs HAVING, not WHERE.

Common Mistake: Expecting a dbplyr pipeline that both creates and immediately filters on a new column to translate into one single SQL query — SQL’s SELECT clause cannot reference an alias it just created in that same clause, so dbplyr must silently generate a subquery (an inner query computing the new column, wrapped by an outer query that filters on it).

Important Questions:

• Why does SQL need both a WHERE clause and a HAVING clause, when dplyr only has filter()? SQL evaluates WHERE before GROUP BY/SELECT, so it cannot filter on an aggregate value that does not exist yet at that point; HAVING is evaluated after grouping and aggregation, so it is the clause used to filter on a summarized value — dbplyr picks between the two automatically.
• What does collect() do in a dbplyr pipeline, and why is it usually the last step? collect() triggers the database to actually run the SQL query dbplyr has been building lazily and pulls the full result back into R as an in-memory tibble; it is placed last so that as much filtering/aggregation as possible happens inside the database engine before any data crosses into R.

Part 4 §4.3: Arrow

Difficulty: Hard · Key topics: parquet format, arrow::open_dataset(), Hive-style partitioning, write_dataset(), lazy arrow dplyr backend, roughly a 100x speedup, to_duckdb()

Introduces the parquet file format and Apache Arrow as a faster alternative to CSV for large datasets, using a real 9GB, 41-million-row Seattle library checkout dataset. It shows opening a large CSV lazily with arrow::open_dataset() (scanning just a sample of rows to infer structure, rather than loading everything), then explains parquet’s four advantages over CSV — smaller size via compression, a rich stored type system, column-oriented storage matching how analysis usually filters/aggregates, and file “chunking” that lets some chunks be skipped entirely. It covers Hive-style partitioning (CheckoutYear=2005/ folder names) via dplyr::group_by() + arrow::write_dataset(), writing dplyr pipelines against a partitioned arrow dataset the same lazy way as a database (build up, then collect()), and measures roughly a 100x speedup from partitioning plus the columnar parquet format over the original single CSV. It closes by noting arrow::to_duckdb() hands an arrow dataset to DuckDB without copying any memory.

Key Points:

• arrow::open_dataset() scans only a sample of rows to infer a large CSV’s structure, without loading the whole file into memory
• Parquet’s four advantages over CSV: smaller (compressed), typed (unlike CSV, which stores everything as text), column-oriented (faster for analysis), and “chunked” (allows skipping irrelevant sections)
• Hive-style partitioning names folders key=value (e.g. CheckoutYear=2021/), so a query filtering on that variable can skip entire files unread
• write_dataset() writes a partitioned dataset; dplyr::group_by() beforehand defines which variable to partition by
• Like dbplyr, an arrow dplyr pipeline is lazy — dplyr verbs build up a query that only runs when collect() is called
• Partitioning by CheckoutYear plus the parquet format together produced roughly a 100x speedup over the original single 9GB CSV for a year-filtered query

Memory Tip: “CSV is a text file pretending to be a table; parquet is an actual table pretending to be a file” — parquet stores real column types and lets a query skip whole chunks of the file, the opposite of a CSV’s flat, must-read-it-all text.

Common Mistake: Crediting partitioning alone, or the parquet format alone, for the whole speedup — the book attributes its roughly 100x figure to both working together: partitioning skips whole files that do not match the filter, while the columnar parquet format then reads only the needed columns from the files that remain.

Important Questions:

• What is Hive-style partitioning, and why does it speed up a filtered query? It is a convention of naming folders as key=value (e.g. CheckoutYear=2021) so a dataset is physically split into files by a variable; when a query filters on that same variable, arrow can skip reading entire files that do not match the filter instead of scanning the whole dataset.
• Why is parquet described as “column-oriented”, and why does that matter for analysis? Parquet stores data column by column rather than row by row, so a query that only needs a handful of columns can read just those columns from disk instead of every column in every row, unlike a row-oriented CSV, which must be read essentially in full.

Part 4 §4.4: Hierarchical Data

Difficulty: Hard · Key topics: lists vs. vectors, list-columns, unnest_wider()/unnest_longer(), inconsistent-type list-columns, GitHub/Game-of-Thrones/geocoding case studies, JSON data types, jsonlite::read_json()/parse_json(), hoist()

Introduces lists as R’s way to hold mixed-type or nested data, how a list can live inside a tibble as a list-column, and str() for inspecting nested structure. It teaches the two core rectangling functions — unnest_wider() for named list-columns where every row has the same element names (turns names into new columns) and unnest_longer() for unnamed list-columns of varying length (turns elements into new rows), including how keep_empty = TRUE preserves rows with an empty list element, and how mixed-type elements within one list-column get preserved as a list-column of individually-wrapped elements rather than silently coerced. It works through three real case studies with increasing depth — very-wide GitHub repository data, relational Game of Thrones character data (extracting a titles list-column into its own joinable table), and deeply nested Google geocoding results (closing with hoist() once the target path is known) — before covering JSON’s six data types and reading JSON files with jsonlite::read_json()/parse_json().

Key Points:

• A list can mix types and be nested; a vector (made with c()) is always flat and homogeneous
• unnest_wider() turns a named list-column’s elements into new columns — use it when every row’s list has the same names
• unnest_longer() turns an unnamed list-column’s elements into new rows — use it when list length varies row to row
• An inconsistent-type list-column cannot be merged by unnest_longer() into one plain column, so it stays a list-column with each element wrapped individually
• names_sep disambiguates a naming collision when unnesting produces a duplicate column name (e.g. two id columns from nested owner/repo data)
• JSON has four scalar types (null, string, number, boolean) plus two container types: arrays ([], like an unnamed list) and objects ({}, like a named list)

Practice Tip: Before writing any unnest_*() call on real nested data, run str() on just one row’s list-column first — seeing whether its top level is named or unnamed immediately tells you whether unnest_wider() or unnest_longer() is the right first step.

Common Mistake: Reaching straight for unnest_wider() on data whose list length actually varies row to row — unnest_wider() expects every row to have the same set of names and will produce ragged, NA-filled columns or an error; unnest_longer() is the correct starting tool whenever element count varies.

Important Questions:

• How do you decide whether to use unnest_wider() or unnest_longer() on a given list-column? Check whether the list-column’s elements are named and consistent across rows — if every row’s list has the same names, unnest_wider() turns each named element into its own column; if the elements are unnamed and vary in count row to row, unnest_longer() turns each element into its own row instead.
• Why can’t a list-column containing a mix of numbers, strings, and logicals be unnested into one ordinary column? A single vector in R must hold values of one consistent type, so unnest_longer() cannot merge genuinely different types into a plain column; instead it keeps the result as a list-column where every element remains individually wrapped.

Part 4 §4.5: Web Scraping

Difficulty: Medium-Hard · Key topics: scraping ethics/legality (ToS, PII, copyright/fair use), HTML elements/attributes, rvest::read_html(), CSS selectors, html_elements()/html_element(), html_text2()/html_attr(), html_table(), dynamic (JS-rendered) sites

Opens with the ethics and legality of scraping — public/non-personal/factual data is usually lower risk, a site’s terms of service in a footer generally is not binding in the US without an explicit agreement (per HiQ Labs v. LinkedIn, though the rule differs in Europe), personally identifiable data is a serious ethical risk even when technically public (citing the 2016 OkCupid scraping controversy), and US copyright protects original expression but not bare facts, with fair use as a further defense for research/non-commercial scraping. It covers HTML basics, reading a page with rvest::read_html(), the three essential CSS selectors (p, .class, #id), html_elements() (returns a vector, possibly length 0) versus html_element() (always returns one result per input, using NA for no match), extracting data with html_text2()/html_attr(), and html_table() for pages already structured as an HTML table. It works through two case studies (Star Wars film data and IMDb’s top 250 films) using SelectorGadget/browser DevTools to find selectors, and closes by noting dynamic, JavaScript-rendered sites are not scrapable by rvest’s static HTML download alone.

Key Points:

• In the US, scraping public, non-personal, factual data is generally lower legal risk; a website’s terms of service alone usually is not binding without an explicit agreement (HiQ Labs v. LinkedIn)
• US copyright protects original expression but not bare facts — a list of ingredients is not copyrightable, though accompanying original narrative content is
• html_elements() returns a vector that can have zero matches; html_element() always returns exactly one result per input (using NA for no match), keeping extracted variables aligned row-by-row
• The three most useful CSS selectors: p (tag name), .title (class), #title (a unique id)
• html_text2() extracts an element’s visible text; html_attr() extracts an attribute’s value (like a link’s href), always as a string
• rvest downloads only the raw HTML and does not execute JavaScript, so dynamically-rendered page content is not reachable with the chapter’s techniques

Common Mistake: Using html_elements() where html_element() is needed when extracting a variable that might be missing for some rows (like an optional field on only some entries) — html_elements() silently returns fewer results than there are parent elements when some are missing, breaking the row-to-row correspondence with other extracted variables.

Important Questions:

• What is the key practical difference between html_elements() and html_element()? html_elements() returns every matching element and can return a vector shorter than its input if some parent elements have no match; html_element() is guaranteed to return exactly one result per input element, substituting NA when nothing matches — which is why it is used when extracting a variable that should align one-to-one with a set of already-selected elements.
• Under what circumstances does the book suggest scraping data carries less legal/ethical risk? When the data is public (no login or agreement required to view it), non-personal (not personally identifiable information about individuals), and factual (bare facts are not copyrightable, unlike original creative expression) — scraping outside those three conditions, or scraping to make money from the results, is flagged as needing actual legal advice.

Part 5 §5.1: Functions

Difficulty: Medium · Key topics: vector/data-frame/plot functions, function() template, tidy evaluation & indirection, embracing {{ }}, data-masking vs. tidy-selection, pick(), rlang::englue()

Motivates writing a function once code has been copy-pasted more than twice, using a real copy-paste bug (a forgotten rename across nearly identical mutate() lines) to show how a function eliminates that class of mistake. It covers three function categories: vector functions (input/output are vectors), data-frame functions (input/output are data frames), and plot functions (returning a ggplot2 object). For data-frame and plot functions it introduces indirection — a naive grouped_mean(df, group_var, mean_var) does not work because dplyr’s tidy evaluation looks for literal columns named group_var/mean_var — solved by “embracing” an argument with {{ }} to tell dplyr to look inside the argument’s stored value instead. It distinguishes data-masking arguments (used by filter()/mutate()/summarize()) from tidy-selection arguments (used by select()/rename()), shows pick() for using tidy-selection inside a data-masking context, and closes with rlang::englue() for auto-generating a plot label from an embraced variable name.

Key Points:

• A function needs three things: a name, its arguments (the parts that vary between calls), and a body (the code that is repeated)
• Wrapping range(x) as range(x, na.rm = TRUE, finite = TRUE) inside a custom function shows how fixing a bug in the function fixes it everywhere the function is used, unlike copy-pasted code
• Tidy evaluation’s “indirection” problem: a function argument like group_var is normally treated as a literal column name, not a stand-in for whatever column name was passed to it
• Embracing an argument with {{ var }} tells a dplyr verb to look inside var for the actual column to use, instead of searching for a literal column called “var”
• Data-masking arguments (in filter(), mutate(), summarize()) need embracing; tidy-selection arguments (in select(), rename()) follow different rules
• pick() lets you use tidy-selection syntax (like starts_with()) inside a data-masking function such as group_by()

Practice Tip: Write any function you are building for a data frame with a hard-coded column name first, get the logic right, and only then generalize it by wrapping the column-name argument in {{ }} — separating “does the logic work” from “does the indirection work” makes embracing bugs much easier to isolate.

Common Mistake: Forgetting to embrace a function argument passed into a data-masking verb like summarize() — the function then always operates on a literal column named e.g. mean_var (which usually does not exist) instead of the column the caller actually intended, producing an “object not found” error that looks unrelated to the real cause.

Important Questions:

• What problem does “embracing” a function argument with {{ }} solve? Without embracing, a dplyr verb inside a custom function treats an argument name like group_var as a literal column name to search for, not as a variable holding the actual column name the caller passed in; wrapping it as {{ group_var }} tells dplyr to look inside the argument for the real column to use.
• How do you know whether a function argument needs to be embraced with {{ }}? Check whether it is passed to a data-masking function (like filter(), mutate(), summarize(), which need embracing) or a tidy-selection function (like select(), rename(), which follow different rules) — the documentation for the tidyverse function being wrapped states which kind each argument is.

Part 5 §5.2: Iteration

Difficulty: Medium-Hard · Key topics: across() (.cols/.fns/.names), if_any()/if_all(), purrr::map()/list_rbind(), listing files with list.files(), data hidden in file paths, possibly() for handling failures, walk()/map2()/walk2() for saving multiple outputs

Introduces dplyr::across() for applying one or more functions to multiple columns at once inside summarize()/mutate(), covering .cols selection (including where(is.numeric) for type-based selection), calling multiple named functions via a list, the .names argument for custom output naming, and if_any()/if_all() as the filter()-compatible variants. It then covers purrr::map() for iterating a function over every element of a vector — listing files with list.files(), reading each with map(), combining with list_rbind(), and recovering data hidden only in a file name via set_names() + list_rbind(names_to = ) — plus the general strategy of preferring several simple iterations over one complex one. It covers possibly() for letting map() continue past individual failures instead of erroring on the whole batch, and closes with the opposite direction — saving multiple outputs — using walk()/walk2() to write multiple database rows, CSV files, or saved plots.

Key Points:

• across(.cols, .fns) applies one or more functions to multiple columns; .cols uses select()-style syntax including everything() and where(is.numeric)
• Passing multiple functions to across() via a named list produces columns named {.col}_{.fn} by default (customizable with .names)
• if_any()/if_all() are the filter()-compatible variants of across(), since across() alone does not combine multiple logical columns with |/&
• purrr::map(x, f) applies f to every element of x and returns a list; list_rbind() stacks a list of data frames into one
• possibly(f, otherwise = NULL) wraps a function so a single failure returns a placeholder instead of aborting the whole map() call
• walk()/walk2() behave exactly like map()/map2() but discard the return value — used for side effects like writing files

Memory Tip: “across() for columns, map() for elements” — across() iterates over the columns of one data frame inside a dplyr verb, while map() iterates over the elements of any vector or list, independent of any single data frame.

Common Mistake: Passing a function to across() or map() with trailing parentheses, e.g. across(everything(), median()) instead of across(everything(), median) — the parenthesized version calls the function immediately with no arguments (raising an error), instead of passing the function itself for across()/map() to call once per column/element.

Important Questions:

• What is the difference between map() and walk()? map(x, f) applies f to each element of x and returns the collected results as a list; walk(x, f) does exactly the same iteration and calls the same function, but discards the return value — used when f is called purely for a side effect, like saving a file.
• Why does map() fail entirely if even one file in a directory cannot be read, and how does possibly() fix that? map() succeeds or fails as a whole — a single error partway through aborts the entire call; wrapping the reading function in possibly(f, otherwise = NULL) changes a failure into a returned placeholder value instead of an error, letting map() finish the whole list and list_rbind()‘s automatic NULL-dropping clean up the failures afterward.

Part 5 §5.3: A Field Guide to Base R

Difficulty: Medium · Key topics: [ ] vs. [[ ]] vs. $, subsetting vectors (5 ways) and data frames, dplyr verbs as special cases of [, lapply()/sapply()/vapply()/tapply(), for loops, plot()/hist()

Positions base R as a complement to the tidyverse, not a replacement, since tidyverse functions are themselves built on base R. It covers [ for selecting multiple elements from a vector five ways (positive integers, negative integers, a logical vector, a character vector for named vectors, and nothing/x[]), extending this to df[rows, cols] for data frames (a plain data.frame can silently drop to a vector when selecting one column, unlike a tibble, unless drop = FALSE), and shows several dplyr verbs as literally special cases of [ (filter() ≈ logical-vector subsetting, arrange() ≈ integer-vector subsetting via order()). It contrasts [ (returns a smaller container of the same type) with [[/$ (extracts and “unwraps” a single element, dropping a level of hierarchy) using the book’s own pepper-shaker analogy, then tours the base “apply family” — lapply() (like map()), sapply() (simplifies output but can fail unpredictably), vapply() (a stricter, type-checked sapply()), and tapply() for a single grouped summary — before covering the basic for loop structure and closing with the two essential base plotting functions, plot() and hist().

Key Points:

• [ can subset a vector five ways: positive integers (keep), negative integers (drop), a logical vector (keep where TRUE), a character vector (named vectors), or nothing (x[], returns everything)
• df[rows, cols] subsets a data frame two-dimensionally; leaving rows or cols blank preserves that whole dimension
• filter() is essentially df[condition, ] with automatic NA-handling; arrange() is essentially df[order(col), ]
• [ always returns the same container type (a smaller list stays a list); [[/$ extract and unwrap a single element, removing one level of hierarchy
• lapply() is base R’s map(); sapply() tries to simplify its output (risky for programming); vapply() is a type-checked, safer sapply()
• A for loop needs pre-allocated output and iterates over seq_along(x), not x itself, to keep input/output positions aligned

Memory Tip: Think of the book’s own pepper-shaker analogy — pepper[1] is still a shaker containing one packet (same container, smaller), while pepper[[1]] is the packet itself, pulled out and unwrapped.

Common Mistake: Using [ when [[ was needed to pull a single element out of a list for computation — df["x"] on a data frame returns a one-column data frame (still a container), while df[["x"]] returns the bare vector inside it; passing the wrong one to a function expecting a plain vector produces a confusing type error.

Important Questions:

• What is the fundamental difference between [ and [[ when applied to a list? [ always returns a smaller list of the same type, no matter how many elements are requested — even l[1] is still a one-element list; [[ extracts and “unwraps” exactly one element, removing a level of hierarchy and returning that element’s own bare type.
• Why does a for loop need to iterate over seq_along(paths) rather than over paths directly when building up a list of results? Iterating over seq_along(paths) gives an integer index for each position, which lets you both read the i-th input and write to the i-th slot of a separately pre-allocated output list — iterating directly over the values of paths gives no way to write results back to matching positions in the output.

Part 6 §6.1–6.2: Quarto & Quarto Formats

Difficulty: Easy-Medium · Key topics: .qmd documents (code + prose + YAML), knitr → pandoc rendering pipeline, code chunks, visual vs. source editor, output formats (html/pdf/docx/revealjs/pptx), websites and books via _quarto.yml

Introduces Quarto as a unified authoring tool combining code, its output, and prose in one reproducible .qmd file. It covers the rendering pipeline (a .qmd file is executed by knitr into a plain markdown file, which pandoc then converts into the final output format), the Visual editor (a Google-Docs-like interface) versus the Source editor (plain Pandoc-Markdown text, better for catching syntax errors), inserting and running code chunks (Cmd/Ctrl+Enter for one expression, Cmd/Ctrl+Shift+Enter for a whole chunk), and YAML header options like execute: echo: false to hide code from a non-technical audience. Chapter 6.2 then covers overriding the output format either permanently (in the YAML) or transiently (via quarto::quarto_render()), document formats (pdf, docx, odt, gfm, ipynb), presentation formats (revealjs, pptx, beamer, where a new slide starts at each ## heading), interactive htmlwidgets versus server-dependent Shiny apps, and building a complete website or book from multiple .qmd files linked by one _quarto.yml navigation file.

Key Points:

• A .qmd file has three components: prose (Pandoc Markdown), code chunks, and a YAML header — rendered by knitr (executes code) then pandoc (produces the final format)
• Cmd/Ctrl+Enter runs the current line/expression; Cmd/Ctrl+Shift+Enter runs the whole current code chunk
• execute: echo: false in the YAML header hides all code from the rendered output — useful for a decision-maker audience that only wants conclusions
• The output format can be set permanently in the YAML format: field or overridden per-call with quarto::quarto_render(output_format = )
• Presentations start a new slide at each second-level (##) heading; Quarto supports revealjs (HTML), pptx (PowerPoint), and beamer (LaTeX PDF)
• A multi-file Quarto website or book needs one _quarto.yml file setting project: type: website (or book) plus navigation for all the .qmd source files

Practice Tip: Render the same short .qmd document to at least two different formats (e.g. html and pdf) and compare the two outputs side by side — seeing what survives the pandoc conversion and what does not makes format-specific options much less abstract.

Common Mistake: Assuming a Quarto document’s rendered output updates the moment code is edited — a .qmd file must actually be re-rendered (Render button, or quarto::quarto_render()) after every code change; editing the source alone does not regenerate the HTML/PDF/Word output automatically.

Important Questions:

• What does the knitr-then-pandoc rendering pipeline actually do to a .qmd file? knitr first executes every code chunk and produces a plain markdown (.md) file containing the code plus its results; pandoc then converts that markdown file into whichever final output format was requested — HTML, PDF, Word, and so on — which is why one .qmd source can produce many different output types.
• How does Quarto decide where one slide ends and the next begins in a presentation format like revealjs? A new slide begins at every second-level (##) heading in the document; a first-level (#) heading instead starts a new section, inserted as its own centered section-title slide before the slides that follow it.

Read R for Data Science Online (Free)

This book has no official free PDF — the authors publish the complete 2nd edition free as an HTML site at r4ds.hadley.nz. Its Creative Commons licence includes a No-Derivatives clause, so this page links directly to the official source rather than hosting a converted copy. Click below to read it.

How to Study This Book

This book is organized into six Parts, and a Data Science with R course will usually move through all of them roughly in order — but the book itself is deliberately sequenced so that Part 1 (the “Whole Game”) gives you a working, if shallow, tour of the entire pipeline before any single topic gets its full depth in Parts 2–6. Read Part 1 (§1.1–1.8, the Introduction plus eight chapters) in full and in order even if some of it feels basic — it is the only Part that teaches ggplot2, dplyr, tidyr, and readr all at once, and every later Part assumes you already have that working vocabulary. Do not skip the workflow interludes (§1.2, §1.4, §1.6, §1.8) just because they contain no new data-science function — RStudio Projects and relative paths in particular are the single most common thing a self-taught R user gets wrong for months.

Part 2 (Visualize, §2.1–2.3) depends directly on Part 1’s Chapter 1.1 and should be read next if your course spends real time on data visualization; it is the natural home of ggplot2’s full grammar, and Layers (§2.1) in particular is worth typing out by hand rather than reading passively, since the seven-parameter template it ends with is the mental model the rest of the book quietly assumes.

Part 3 (Transform, §3.1–3.8) is the densest Part in the book and the one that most benefits from hands-on typing rather than reading. Work through Logicals (§3.1) before Numbers (§3.2), since if_else()/case_when() and comparison operators come up constantly in every later chapter of this Part. Strings (§3.3) should be read before Regular Expressions (§3.4) — the two are explicitly a pair in the book’s own chapter order, and Regular Expressions is genuinely the hardest single chapter in the book, worth budgeting extra time for. Missing Values (§3.7) and Joins (§3.8) can be read in either order relative to the rest of Part 3, but Joins depends on a solid grasp of Logicals’ comparison operators and is worth reading only once filter/mutate/summarize from Part 1 feel automatic.

Part 4 (Import, §4.1–4.5) is the most situational Part for a typical undergraduate course — Spreadsheets (§4.1) is close to universally useful, but Databases (§4.2), Arrow (§4.3), and Web Scraping (§4.5) are genuinely advanced, infrastructure-heavy topics that many one-semester Data Science with R courses skip entirely or treat as optional reading. If your course does not require SQL or big-data tooling, it is reasonable to read Spreadsheets and Hierarchical Data (§4.4, since JSON shows up constantly from web APIs) and treat the other three chapters as a reference to return to later.

Part 5 (Program, §5.1–5.3) is where most R beginners hit their hardest conceptual wall in the whole book — not the syntax of writing a function, but tidy evaluation and “embracing” a variable with {{ }} in Functions (§5.1). Do not treat embracing as something to memorize on a first read; work through the chapter’s own grouped_mean() example by hand, deliberately breaking it (removing the braces) to see the actual error, before moving to Iteration (§5.2). A Field Guide to Base R (§5.3) is reference material more than sequential teaching — it is genuinely useful for reading other people’s R code later, but nothing in the rest of the book strictly depends on having read it first.

Part 6 (Communicate, §6.1–6.2) is the shortest Part and can be read at almost any point once Part 1 is comfortable — many courses assign it early, alongside the first homework, since students need to know how to turn in a rendered .qmd report long before they have learned regular expressions or joins.

Be honest with yourself about what this book does not cover, since a syllabus that expects it will leave real gaps. The Introduction says so explicitly: no statistical modeling or machine learning beyond the data-wrangling stage (the authors point to the separate Tidy Modeling with R and the tidymodels package family), no genuinely big-data tooling beyond what Arrow/duckdb can do on a laptop (recommending the separate data.table package for 10–100GB workloads), and no Python or Julia at all — it teaches R and only R, deliberately. If a course also requires model-fitting, deep learning, or a second language, a separate resource is needed for that unit specifically.

Used In These Programs

This book is used for the Data Science with R / Data Wrangling and Visualization course in: BS Computer Science · BS Information Technology. Browse all Data Science books, all Statistics books, or all Computer Science books.

Who Should Read This

This book is written for a reader with only basic numerical literacy and, ideally, some prior programming exposure in any language — unlike a book such as Python Data Science Handbook, it does not assume you already know R, and teaches the language itself from the very first chapter alongside the data-science workflow. It suits BS Data Science and BS Statistics students taking a dedicated Data Science with R, Data Wrangling, or Data Visualization course, as well as BSCS/BSIT students taking an R-based data-science elective, most naturally in their second or third year once basic programming logic is comfortable. It is a poor fit for a course that specifically needs statistical modeling, machine learning, big-data infrastructure at scale, or any language other than R — the book’s own Introduction explicitly rules all four out of scope and points elsewhere for them. If your course also needs core programming fundamentals taught first, from a language-agnostic starting point, pair this book with Think Python or a similar introductory text.

Applicable Universities

This book is useful for students at Pakistani universities offering BS Data Science, BS Statistics, BSCS, and BS Information Technology degrees, including Punjab University, Virtual University, COMSATS, FAST, UET, NUST, and other HEC-recognized institutions.

FAQs

Related Books

• Think Python (3rd Edition) – Allen B. Downey
• Database Management System – IT Series (SQL companion to this book’s Databases chapter)
• Problem Solving with Algorithms and Data Structures using Python

R for Data Science (2nd Edition) remains the most widely assigned starting point for learning the tidyverse, and its emphasis on a repeatable import-tidy-transform-visualize-communicate workflow makes it as useful as a reference after the course as it is during it. Browse more Computer Science books for the rest of your semester.

R for Data Science, 2nd Edition, by Hadley Wickham, Mine Çetinkaya-Rundel, and Garrett Grolemund (O’Reilly Media, 2023). Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 United States (CC BY-NC-ND 3.0 US). Read the official free edition at https://r4ds.hadley.nz/