Lecture Platform
Mini-project B — optional, non-gating The unseen attack

Run the full spine on NSL-KDD. A binary normal/attack target, a metric you state and defend, within-train validation scores vs. KDDTest+ scores, per-attack-name recall split by whether the attack's name was ever seen in training, and one honest paragraph on what the 1999-simulation caveat means for anyone using this data to claim something about real networks today.

Fully worked, end to end

Unlike the project's starter scaffold, mini-project.ipynb is a complete, already-executed implementation — open it to see (and re-run) the real numbers below, not a template to fill in.

What it found (measured, not asserted)

Python — notebook mini-project Open in notebook
tree = DecisionTreeClassifier(max_depth=6, random_state=0).fit(nsl_train[NSL_KDD_NUMERIC], y_subtrain)
# stated metric: AUC + recall at a 50% flagged rate (a SOC that can triage half of flagged traffic)
KDDTrain+: (125973, 44) KDDTest+: (22544, 44) attack share, train: 46.54% test: 56.92% within-train validation: AUC=0.9969 recall@50%=0.9925 KDDTest+: AUC=0.8280 recall@50%=0.7895 mscan n= 996 recall=0.472 (UNSEEN in train) apache2 n= 737 recall=0.000 (UNSEEN in train) processtable n= 685 recall=0.104 (UNSEEN in train) saint n= 319 recall=0.991 (UNSEEN in train) httptunnel n= 133 recall=0.820 (UNSEEN in train) ... (17 attack names total appear in KDDTest+ that never appear in KDDTrain+)

The within-train validation AUC (0.9969) looks excellent; KDDTest+ drops to 0.8280 — not a collapse to chance, but a real, measurable gap, and unevenly distributed: some never-seen attack names (saint, httptunnel) are still caught well by features that generalise; others (apache2, snmpguess, mailbomb) are missed almost entirely.

The honest paragraph

NSL-KDD's test file was deliberately built (Tavallaee et al. 2009) with attack types absent from its training file — a designed artifact of a 1999 network-intrusion simulation, not naturally occurring modern traffic drift. This mini-project's own computed numbers above (not any externally-quoted percentage) show the gap: recall on attack names the tree has seen in training vs. names it has never seen. Anyone using this 1990s-simulation dataset to claim something about today's network traffic needs to say so explicitly — the shift here is real, but it is a property of this specific historical benchmark's construction, not evidence about modern intrusion detection systems in general.

Marks

Completing this notebook marks population-shift as touched in the progress tracker — optional, non-gating, does not affect Project A's mastery status.