# Data Science — progress
Running doc mirrored by the on-page tracker (`assets/js/progress-schema.json` + `localStorage`). Update this file
whenever a `/lecture` run adds or changes a module in this course; the on-page view is the live/interactive source
of truth for session-by-session status, this file is the durable narrative record.
## Module 01 — Foundations of Data Science
- **Status**: built 2026-09-09. 17 sessions across 6 arcs (Setup, The puzzle & the field, Clean & tidy, Describe,
See, Chance), one required project ("The readmission brief," 6 checkpoints), one optional mini-project
("Untangle the airline table").
- **Topics covered**: missing-data sentinels (`?` vs. literal `None`), duplicate semantics (visit-level vs.
patient-level), dtype conversion (ordered categoricals, numeric midpoints), tidy data / reshaping (wide-to-long),
mean/median/skew, variance/std (`ddof` conventions), quartiles/IQR, histograms (binwidth), box plots (Tukey
outliers), scatter plots (overplotting), empirical probability, PMFs and expected value, sampling variability.
- **Anchor dataset**: UCI "Diabetes 130-US Hospitals 1999-2008" (101,766 visits, verified messiness used
deliberately as the teaching hook). Supplementary: FiveThirtyEight `airline-safety.csv`.
- **Mastery**: defined by the 6 checkpoints in `project.html`, never by a score. Check the on-page tracker at
`modules/01-foundations/index.html` for current status.
## Module 02 — Statistics deep dive
- **Status**: built 2026-09-09. 12 sessions across 5 arcs (The shape of the wobble, How wrong could I be?, Is the
difference real?, Together is not because, The A/B test), one required project ("The A1C memo," 6 checkpoints),
one optional mini-project ("Fishing trip"). Builds forward from Module 01 — no cleaning/describing/basic-viz
content repeated.
- **Topics covered**: the central limit theorem (via Module 01's own sampling-wobble figure), confidence intervals
(construction + coverage + correct interpretation), two-sample hypothesis testing (Welch's t built by hand, cross-
checked with `scipy.stats.ttest_ind`), the permutation/null-world view of a p-value and its common
misinterpretations, effect size vs. statistical significance, correlation vs. causation (Pearson r, confounders,
partial correlation, an optional Simpson's-paradox hunt), and A/B testing as the synthesis point (proper
randomization vs. an observational split). Climax: discovering that the headline readmission gap (visit-level,
t ≈ 6.2) shrinks but survives (first-visit-per-patient, t ≈ 2.2, p ≈ 0.03) once repeat visits
from the same patient are no longer double-counted — the independence condition planted in Arc 1 and cashed in
here.
- **Anchor dataset**: same as Module 01 — UCI "Diabetes 130-US Hospitals 1999-2008" (101,766 visits from 71,518
distinct patients; 16,773 patients appear more than once; one patient appears 40 times — see
`data/DATA-README.md`).
- **Mastery**: defined by the 6 checkpoints in `modules/02-statistics/project.html` ("The A1C memo"), never by a
score. Check the on-page tracker at `modules/02-statistics/index.html` for current status.
## Module 03 — ML algorithms
- **Status**: built 2026-09-11. 15 sessions across 5 arcs (A model is a bet, Lines, Scorecards, Questions in a row,
No label) — 3 sessions marked optional (ROC/AUC, k-NN, hierarchical clustering) — one required project ("The risk
flag," 6 checkpoints), one optional mini-project ("The leak, measured"). Builds forward from Modules 01-02 — tidy
data, descriptive statistics, correlation, and hypothesis testing are assumed, not re-taught, and several exact
numbers (r=0.4661, the 40-visit patient, the 300-visit fishing-trip sample and its seed) are reused verbatim from
Module 02.
- **Topics covered**: supervised vs. unsupervised framing; train/test splits and overfitting (the fishing trip, run
again with a model); patient-safe (group-aware) splitting to prevent leakage from repeat visits; linear regression
by closed form (b = r·sy/sx, R²=r²) and its library check; logistic
regression fit by hand via gradient descent; the confusion matrix, precision/recall, and the threshold as a
business choice (optional: ROC/AUC); decision trees via Gini-scored splits, tree reading, and the depth-vs-overfit
dial (optional: k-NN); the mandatory three-way scaling contrast (OLS unaffected, penalised logistic regression
moves a little, distance-based k-means moves a lot); k-means by hand (assign/move/repeat), the elbow method
(honestly ambiguous on this data), and an after-the-fact check against a label never shown to the model (optional:
hierarchical clustering/dendrograms).
- **Anchor dataset**: same as Modules 01-02 — UCI "Diabetes 130-US Hospitals 1999-2008" (101,766 visits), restricted
to 9 numeric utilisation/clinical features per the module's light-touch feature-selection scope (no diagnosis-code
one-hot encoding).
- **Notable finding of this run**: the patient-repeat leakage Module 02 flagged (existence only) turns out, once
measured directly (s1-3, mini-project B), to be real but small in size for every model tried on this feature set
(accuracy/recall gaps of a few tenths of a percentage point) — a genuinely honest, data-driven answer to a question
the module's own research could not resolve in advance.
- **Mastery**: defined by the 6 checkpoints in `modules/03-ml-algorithms/project.html` ("The risk flag"), never by a
score. Check the on-page tracker at `modules/03-ml-algorithms/index.html` for current status.
## Module 04 — Deep learning
- **Status**: built 2026-09-13. 15 sessions across 5 arcs (The ceiling, One neuron then many, Learning from
mistakes, The real build, Where it wins and where it doesn't) — 1 session marked optional (the convolution
mechanics tangent) — one required project ("The second opinion," 6 checkpoints), one optional mini-project ("The
digit reader"). Builds forward from Modules 01-03 — tidy data, the patient-safe split, and Module 03's exact
hand-fit logistic-regression weights are reused verbatim, not re-derived.
- **Topics covered**: the linear ceiling and how a hidden ReLU layer crosses it (XOR, by hand); the neuron as
σ(w·x+b) and logistic regression's continuity with it; a full forward pass traced by hand on Mazur's
2-2-2 network with a live interactive slider; activation functions and their derivatives (saturation, dying ReLU);
backpropagation derived by hand on a 2-parameter chain and confirmed via finite differences and PyTorch autograd
(the module's one raw-autograd session, by design, before switching to `nn.Sequential` for every real build);
loss landscapes, learning rate choice, mini-batches, and Adam; building and training an MLP with PyTorch on the
diabetes data; overfitting discovered by the learner's own 300-epoch run, then three countermeasures (early
stopping, dropout, weight decay) compared head-to-head; the same MLP architecture family applied to MNIST pixels,
where the deep-learning-vs-classical gap is genuinely visible; convolution mechanics as an optional, untrained
tangent; and a closing four-model scoreboard plus a multi-cause reading of AlexNet (2012).
- **Anchor dataset(s)**: the diabetes 130-US-hospitals data (same 9-feature set and patient-safe split as Module 03)
for every tabular build in Arcs 1-4 and the Arc 5 scoreboard, so the module's central question — does stacking
neurons beat what Module 03 already built, on identical inputs? — gets an apples-to-apples answer. MNIST (LeCun et
al. 1998, vendored via a stable IDX mirror) is introduced fresh in Arc 5, with the *same* MLP shape family used on
the diabetes data, as the module's one deliberate contrast on unstructured input — the smallest dataset where raw
columns aren't meaningful features at all.
- **Notable finding of this run**: measured, not assumed — Module 03's logistic regression (AUC 0.6187), a
depth-limited tree (0.6142), the Arc 4 MLP with its chosen brake (0.6189), and an untaught gradient-boosted
ensemble (0.6156) land within two AUC points of each other on identical patient-safe test rows. Deep learning does
not clearly beat Module 03's simplest model here — it's essentially tied with it — confirming the module's opening
spoiler. The same architecture family, unchanged, beats logistic regression by 3.7 points of test accuracy on raw
MNIST pixels (94.79% vs. 91.11%), and neither model's accuracy moves at all when pixel order is shuffled
identically for every image — direct evidence that the diabetes table's columns, not any lack of "deep learning,"
are the ceiling on this data, and that the technique's real edge is on inputs without meaningful column structure.
- **Mastery**: defined by the 6 checkpoints in `modules/04-deep-learning/project.html` ("The second opinion"), never
by a score. Check the on-page tracker at `modules/04-deep-learning/index.html` for current status.
## Module 05 — Applied case studies
- **Status**: built 2026-09-14. 12 sessions across 5 arcs (The question comes first, Leaks you haven't met, Pick the
tool, The price of a wrong line, Tell the truth) — 1 session marked optional (the drift-detector tangent), one
required project ("Switch it on," 6 checkpoints), one optional mini-project ("The unseen attack"). Builds forward
from Modules 01-04 — cleaning, statistics, the full model toolbox (logistic regression, decision trees, k-means,
the MLP), and confidence intervals are assumed and reused, not re-taught; this is the course's capstone/synthesis
module, not a new algorithmic family.
- **Topics covered**: framing a problem (target, metric, success) before any model exists; target leakage as a proxy
for a decision a human already made (the Epic Sepsis Model story, cashed out on the diabetes data's
discharge-disposition codes); temporal leakage (a time-ordered split, and a leak the learner builds and watches
with their own hands); model selection under real constraints (volume, labels, human action, latency) rather than
"newest"; the Caruana pneumonia/asthma precedent for interpretability as a safety instrument; a dollar-denominated
cost matrix and proper threshold tuning (`TunedThresholdClassifierCV`) on a time-respecting validation slice;
population/distribution shift, measured directly via NSL-KDD's designed train/test attack-name gap; an optional
Module-02-style drift-detector tangent; and bootstrap confidence intervals used to decide whether a model
scoreboard has a real winner or a tie, with published precedent for both outcomes.
- **Anchor dataset(s)**: **Credit Card Fraud** (ULB Machine Learning Group with Worldline, via OpenML id 1597;
284,807 transactions, 492 frauds = 0.172%, `Time`/`Amount`/`V1`-`V28` PCA-anonymised features) for Arcs 1-4, Arc
5's comparison sessions, and the required project — chosen for genuine extreme imbalance and a real, per-row dollar
cost the diabetes/MNIST anchors couldn't supply. **NSL-KDD** (Tavallaee et al. 2009) for Arc 5's s5-1 and the
optional mini-project only — the one candidate with a documented, constructible train/deploy population shift.
**Diabetes 130-US Hospitals** (already vendored) returns for exactly one arc (s2-1/s3-2) to host a hands-on
target-leakage example on data the learner already owns. **Telco Customer Churn** appears only as a text brief
(7,043 customers, ~26.5% churn) — never loaded or vendored, since its license was unresolved.
- **Notable finding of this run**: measured, not assumed — several numbers came out differently than the module's
own design-stage expectations, and are reported as found rather than adjusted to match: (1) the three fraud models
did **not** tie the way Module 04's scoreboard did — the MLP led on PR-AUC (0.7898) over logistic regression
(0.7133) and a depth-limited tree (0.6436), and a bootstrap interval on the top-two difference **excludes zero**
([0.023, 0.136]) — a real winner, not a tie, which the module treats as an equally legitimate honest outcome, with
Module 03's elbow and Module 04's own tie kept as the precedent for the opposite case. (2) A hand-constructed
temporal leak (`hour_fraud_rate` computed over the whole stream) did **not** visibly inflate the random-split
PR-AUC (0.7657 vs. 0.7689 without it) — redundant with the already-strong PCA features — which the module reframes
as the sharper lesson: a leak's danger is about what information it could have used, not whether it happens to
move a favourite validation number today. (3) At a $500-per-review cost assumption, even the properly tuned
threshold still cost more than doing nothing on the held-out test slice — an honest finding about tuning on a
small, rare-event-dominated slice, kept in the project's own checkpoint guidance. (4) The tuned threshold at the
$5 review-cost assumption landed at 0.036–0.11, far below the naive 0.5 default, recovering $7,811.90 of
$10,839.13 at stake on the held-out test rows by flagging only 155 of 71,202 transactions — this one matched the
module's expectation closely. (5) NSL-KDD's tree scored AUC 0.997 on within-train validation and 0.828 on
`KDDTest+`; recall on attack names seen in training was 0.785 vs. 0.26 on names never seen — the seen/unseen gap
is real and large, computed directly rather than citing any externally-quoted percentage.
- **Mastery**: defined by the 6 checkpoints in `modules/05-applied-case-studies/project.html` ("Switch it on"), never
by a score. Check the on-page tracker at `modules/05-applied-case-studies/index.html` for current status.
## Planned modules (not yet built)
This closes out the data science course's currently-planned module list — Module 05 was the last module named in
the course's own planning. A Module 06 is not assumed here; that would be a future `/lecture` request.