{
 "nbformat": 4,
 "nbformat_minor": 5,
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.x"
  },
  "colab": {
   "provenance": []
  }
 },
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lab 3 - Build Your First Spam Classifier 🌳\n",
    "### CIS 350: AI for Cybersecurity - Smart Defense for the Digital Business\n",
    "\n",
    "**Worth:** 50 points  ·  **Time:** ~60 minutes  ·  **Submit:** this `.ipynb` file in Canvas\n",
    "\n",
    "This is the moment from Wednesday's lecture: you will **train a real decision-tree classifier** to tell spam from legitimate email, test it the honest way (train/test split), and see how it makes decisions. The dataset and code are the same ones from the slides.\n",
    "\n",
    "**What you'll do**\n",
    "1. Create the labeled email dataset (features + label).\n",
    "2. Split features (X) and label (y), then into train and test sets.\n",
    "3. Train a `DecisionTreeClassifier` and measure its accuracy on the **test** set.\n",
    "4. Use the model to label a brand-new email.\n",
    "5. Experiment with tree depth to *see* overfitting, and read a confusion matrix.\n",
    "\n",
    "Run cells in order with **Shift + Enter** and fill every **`TODO`** blank (`____`).\n",
    "\n",
    "> **Academic integrity:** individual or pair work. **AI tools are not permitted** on labs."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 1 · Create the labeled dataset\n",
    "Each **row** is one email; the columns are **features**; `label` is the **answer** (spam/ham)."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "rng = np.random.default_rng(7)\n",
    "n = 240\n",
    "df = pd.DataFrame({\n",
    "    \"num_links\":       rng.integers(0, 11, n),\n",
    "    \"has_attachment\":  rng.integers(0, 2, n),\n",
    "    \"sender_mismatch\": rng.integers(0, 2, n),\n",
    "    \"urgent_words\":    rng.integers(0, 6, n),\n",
    "})\n",
    "score = (0.5 * df.num_links + 1.2 * df.urgent_words + 1.5 * df.sender_mismatch\n",
    "         + 0.8 * df.has_attachment + rng.normal(0, 1.6, n))\n",
    "df[\"label\"] = np.where(score > 8.5, \"spam\", \"ham\")\n",
    "\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "# TODO 1: how many emails are spam? (and how many ham?)\n",
    "# Hint: df[\"label\"].value_counts()\n",
    "print(____)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 2 · Features (X) and label (y)"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "# TODO 2: pick the four feature columns as X, and the label as y.\n",
    "X = df[[\"num_links\", \"has_attachment\", \"sender_mismatch\", ____]]\n",
    "y = df[\"label\"]\n",
    "print(X.shape, y.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 3 · Split into training and test sets\n",
    "The test set is held back so we can grade the model on emails it never trained on."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "# TODO 3: hold back 20% for testing (fill in test_size).\n",
    "X_train, X_test, y_train, y_test = train_test_split(\n",
    "        X, y, test_size=____, random_state=42)\n",
    "print(len(X_train), \"train /\", len(X_test), \"test\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 4 · Train a decision tree"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "from sklearn.tree import DecisionTreeClassifier\n",
    "\n",
    "# TODO 4: create a tree with max_depth=3, then train it on the TRAINING data.\n",
    "model = DecisionTreeClassifier(max_depth=3, random_state=0)\n",
    "model.____(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 5 · Test it: accuracy on unseen email"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "from sklearn.metrics import accuracy_score\n",
    "\n",
    "preds = model.predict(X_test)\n",
    "# TODO 5: compute accuracy by comparing preds to the true y_test.\n",
    "acc = accuracy_score(____, ____)\n",
    "print(\"test accuracy:\", round(acc, 3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 6 · Predict a brand-new email\n",
    "Features in this order: [num_links, has_attachment, sender_mismatch, urgent_words]."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "new_email = [[7, 1, 1, 4]]   # 7 links, attachment, mismatched sender, 4 urgent words\n",
    "# TODO 6: ask the model to predict the label of new_email.\n",
    "print(model.____(new_email))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 7 · See overfitting (run and observe)\n",
    "Watch how training accuracy climbs while test accuracy levels off as the tree gets deeper."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "for depth in [1, 2, 3, 5, 8, 12]:\n",
    "    m = DecisionTreeClassifier(max_depth=depth, random_state=0).fit(X_train, y_train)\n",
    "    tr = accuracy_score(y_train, m.predict(X_train))\n",
    "    te = accuracy_score(y_test, m.predict(X_test))\n",
    "    print(f\"depth={depth:2d}  train={tr:.2f}  test={te:.2f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 8 · Which features mattered? + confusion matrix (run and read)"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "print(pd.Series(model.feature_importances_, index=X.columns).sort_values(ascending=False))\n",
    "\n",
    "from sklearn.metrics import confusion_matrix\n",
    "print(confusion_matrix(y_test, model.predict(X_test)))   # rows=actual, cols=predicted"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 9 · What does it mean? (Guided questions)\n",
    "Double-click below and replace each *your answer here*."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Your answers**\n",
    "\n",
    "1. **How many emails total, and how many spam vs. ham?** *your answer here*\n",
    "2. **What test accuracy did your depth-3 tree get? Given the class balance, do you fully trust it? Why?** *your answer here*\n",
    "3. **In Part 7, what happened to training vs. test accuracy as depth grew? What is that called?** *your answer here*\n",
    "4. **Which feature was most important? Does that match your intuition about spam?** *your answer here*\n",
    "5. **In the confusion matrix, which error is worse for a business - flagging a real email as spam (false positive) or letting spam through (false negative)? Why?** *your answer here*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## ✅ Before you submit\n",
    "- [ ] **Runtime -> Run all** with no errors.\n",
    "- [ ] All `TODO` blanks filled.\n",
    "- [ ] All 5 guided questions answered.\n",
    "- [ ] **File -> Download -> Download .ipynb**, then upload to Canvas."
   ]
  }
 ]
}