{
 "nbformat": 4,
 "nbformat_minor": 5,
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.x"
  },
  "colab": {
   "provenance": []
  }
 },
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lab 8 - Adversarial Example Explorer 🖼️⚠️\n",
    "### CIS 350: AI for Cybersecurity - Smart Defense for the Digital Business\n",
    "\n",
    "**Worth:** 50 points  ·  **Estimated time:** ~60 minutes  ·  **Submit:** this `.ipynb` in Canvas\n",
    "**Required software:** Google Colab (or Jupyter) with `scikit-learn`, `numpy`, `matplotlib` (preinstalled in Colab).\n",
    "**Prerequisites:** Weeks 2-4 (features, train/test, evaluation) and Week 10 (attacks on ML).\n",
    "\n",
    "## ⚠️ Ethics & scope (read first)\n",
    "This is a **defensive, sandboxed** lab. You will attack a **toy** digit classifier **inside this notebook** to understand WHY models are fragile - so you can defend real ones. **Never apply these techniques to systems you do not own or have permission to test.** Doing so is an academic-integrity and legal violation.\n",
    "\n",
    "## Background\n",
    "An **adversarial example** is an input changed by a tiny, *crafted* amount that flips the model's prediction - often invisibly to a human. You will craft one against a digit classifier, see how small the change is, watch the model's **confidence** drop, and then try a simple **detection defense**.\n",
    "\n",
    "## Learning objectives\n",
    "By the end you can: (1) craft a small perturbation that fools a classifier; (2) visualize and measure how small the change is; (3) interpret the confidence drop; (4) apply and reason about a detection defense.\n",
    "\n",
    "Run cells top to bottom with **Shift + Enter** and fill every **`TODO`** blank (`____`).\n",
    "\n",
    "> **Academic integrity:** individual or pair work. **AI tools are not permitted** on labs."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 1 · Train a simple image classifier\n",
    "We use scikit-learn's 8x8 digit images and a logistic-regression classifier."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\")   # keep output clean\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.datasets import load_digits\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.metrics import accuracy_score\n",
    "\n",
    "digits = load_digits()               # 8x8 grayscale images of 0-9, pixel values 0-16\n",
    "X, y = digits.data, digits.target    # X: 1797 x 64 (flattened), y: the digit\n",
    "X_train, X_test, y_train, y_test = train_test_split(\n",
    "        X, y, test_size=0.3, random_state=0, stratify=y)\n",
    "clf = LogisticRegression(max_iter=5000).fit(X_train, y_train)   # our \"image classifier\"\n",
    "\n",
    "# TODO 1: measure test accuracy (true labels, predictions).\n",
    "acc = accuracy_score(y_test, clf.predict(____))\n",
    "print(\"test accuracy:\", round(acc, 3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 2 · Pick an image the model gets RIGHT\n",
    "We attack a correctly-classified digit. Here we target a **2** and try to make it read as a **3**."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "true_class, target = 2, 3\n",
    "# first correctly-classified test image of the true class\n",
    "idx = [i for i in range(len(X_test))\n",
    "       if y_test[i] == true_class and clf.predict([X_test[i]])[0] == true_class][0]\n",
    "x = X_test[idx].copy()\n",
    "conf = clf.predict_proba([x])[0].max()\n",
    "print(\"model predicts:\", clf.predict([x])[0], \" confidence:\", round(conf, 3))\n",
    "plt.imshow(x.reshape(8, 8), cmap=\"gray_r\"); plt.title(\"original\"); plt.axis(\"off\"); plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 3 · Craft the perturbation\n",
    "For a linear model, the most 'sensitive direction' toward another class is the difference of their weight vectors. We nudge the image a little that way."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "direction = clf.coef_[target] - clf.coef_[true_class]\n",
    "direction = direction / np.linalg.norm(direction)   # unit step\n",
    "# TODO 2: make the adversarial image by nudging x along `direction` by eps,\n",
    "#         then clip pixels back to the valid 0-16 range.\n",
    "eps = 3.5\n",
    "x_adv = np.clip(x + eps * ____, 0, 16)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 4 · Did it flip? Sweep the size\n",
    "See how big a nudge is needed to change the prediction."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "for e in [1, 2, 3, 3.5, 4]:\n",
    "    xa = np.clip(x + e * direction, 0, 16)\n",
    "    print(f\"eps={e}: predicted {clf.predict([xa])[0]}  (confidence {clf.predict_proba([xa])[0].max():.2f})\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 5 · Visualize the attack\n",
    "The original and adversarial images should look almost identical to you."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "pred_adv = clf.predict([x_adv])[0]\n",
    "fig, ax = plt.subplots(1, 2, figsize=(6, 3))\n",
    "ax[0].imshow(x.reshape(8, 8), cmap=\"gray_r\");     ax[0].set_title(f\"orig: {clf.predict([x])[0]}\")\n",
    "ax[1].imshow(x_adv.reshape(8, 8), cmap=\"gray_r\"); ax[1].set_title(f\"adv: {pred_adv}\")\n",
    "for a in ax: a.axis(\"off\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 6 · How small was the change?"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "# TODO 3: the biggest single-pixel change (out of a 0-16 scale).\n",
    "max_change = np.abs(x_adv - ____).max()\n",
    "print(\"max pixel change:\", round(max_change, 2), \"of 16\")\n",
    "print(\"clean confidence:\", round(clf.predict_proba([x])[0].max(), 2))\n",
    "print(\"adversarial confidence:\", round(clf.predict_proba([x_adv])[0].max(), 2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 7 · A detection defense\n",
    "Adversarial examples often sit near the boundary, so the model is **less confident**. A defender can flag low-confidence predictions for human review."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "def flag_if_unsure(image, threshold=0.90):\n",
    "    top = clf.predict_proba([image])[0].max()\n",
    "    return top < threshold, round(top, 2)\n",
    "# TODO 4: run the detector on the clean image and the adversarial image.\n",
    "print(\"clean:      \", flag_if_unsure(x))\n",
    "print(\"adversarial:\", flag_if_unsure(____))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 8 · Reflection questions\n",
    "Double-click and replace each *your answer here*."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Your answers**\n",
    "\n",
    "1. **What was the model's accuracy, and how confident was it on the clean image?** *your answer here*\n",
    "2. **At what `eps` did the prediction flip, and how big was the biggest pixel change (of 16)?** *your answer here*\n",
    "3. **Could you tell the original and adversarial images apart? Why does that matter for security?** *your answer here*\n",
    "4. **What happened to the model's confidence on the adversarial image, and how did the detector use that?** *your answer here*\n",
    "5. **Detection is only one layer. Name TWO other defenses against adversarial examples or data poisoning.** *your answer here*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## ✅ Before you submit\n",
    "- [ ] **Runtime -> Run all** with no errors.  - [ ] All `TODO` blanks filled.\n",
    "- [ ] All 5 reflection questions answered.  - [ ] **File -> Download -> Download .ipynb**, upload to Canvas.\n",
    "\n",
    "## Troubleshooting\n",
    "- `NameError` -> you skipped a cell; run from the top with **Run all**.\n",
    "- Prediction doesn't flip -> increase `eps` a little (try up to 5); make sure you used `x_adv`, not `x`.\n",
    "- Numbers slightly different -> tiny variation across scikit-learn versions is fine.\n",
    "\n",
    "## Extension (optional, ungraded)\n",
    "Pick a different `true_class`/`target` pair (e.g., 4 -> 9). Which pairs flip with the smallest change? What does that tell you about which digits the model confuses most easily?"
   ]
  }
 ]
}