{
 "nbformat": 4,
 "nbformat_minor": 5,
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.x"
  },
  "colab": {
   "provenance": []
  }
 },
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# CTF Challenge - Evade the Filter\n",
    "### CIS 350: AI for Cybersecurity  |  Week 8: Threats from the Inside - How Attackers Use AI\n",
    "\n",
    "**Category:** Evasion / Adversarial ML   **Points:** 15   **Difficulty:** Medium\n",
    "\n",
    "## Ethics and sandbox note\n",
    "This is an AUTHORIZED, sandboxed, defensive exercise. The 'filter' below is a TOY scikit-learn model trained on SYNTHETIC data inside this notebook - it is not a real email system, network, or service. Never apply these techniques to systems you do not own or have written permission to test.\n",
    "\n",
    "## Scenario\n",
    "A company deploys a machine-learning phishing filter that reads five simple, surface-level features of every email and outputs a phishing probability. Anything scoring >= 0.5 is quarantined. As a red-team analyst (Week 8 / Lab 7), you want to show that a filter built only on easily-forged features can be evaded - so the blue team learns to add signals that are hard to fake.\n",
    "\n",
    "## The five features\n",
    "- `num_links` (0-10): number of hyperlinks\n",
    "- `urgent_words` (0-5): count of pressure words (\"now\", \"immediately\", \"verify\")\n",
    "- `has_attachment` (0/1): a file is attached\n",
    "- `sender_mismatch` (0/1): display name does not match the true sending address\n",
    "- `cred_request` (0/1): the email asks for a password / login / payment info\n",
    "\n",
    "## Objective\n",
    "Craft an email (a feature dictionary) that the filter scores as BENIGN while it is STILL a credible phishing attempt.\n",
    "\n",
    "## Success condition (exactly what `check()` tests)\n",
    "Your email wins only if ALL of these hold:\n",
    "1. `score_email(email) < 0.5`  (the filter calls it benign), AND\n",
    "2. `cred_request == 1`         (it still steals credentials), AND\n",
    "3. `urgent_words >= 1`         (it still applies pressure), AND\n",
    "4. `num_links >= 1`            (it still has a link to click).\n",
    "\n",
    "A too-clean email (no links, no urgency, no credential request) does NOT count - that would just be a harmless email, not an evasion."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 1 - Build the toy filter (run this, do not edit)\n",
    "This cell trains the deterministic phishing filter and provides two helpers you will use:\n",
    "`score_email(email)` -> phishing probability, and `check(email)` -> grades your attempt.\n",
    "It also defines `STARTING_EMAIL`, a realistic phishing email the filter correctly flags."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\")   # keep output clean for beginners\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "\n",
    "RANDOM_SEED = 42\n",
    "FEATURES = [\"num_links\", \"urgent_words\", \"has_attachment\", \"sender_mismatch\", \"cred_request\"]\n",
    "\n",
    "\n",
    "def build_dataset():\n",
    "    \"\"\"Generate ~400 labeled synthetic emails deterministically.\n",
    "\n",
    "    Each email is 5 numeric features:\n",
    "      num_links       (0-10)  how many hyperlinks the email contains\n",
    "      urgent_words    (0-5)   count of pressure words (\"now\", \"immediately\", ...)\n",
    "      has_attachment  (0/1)   whether a file is attached\n",
    "      sender_mismatch (0/1)   display name does not match the real sender address\n",
    "      cred_request    (0/1)   the email asks for a password / login / payment info\n",
    "\n",
    "    Phishing (label 1) tends to have more links, more urgent words, more\n",
    "    credential requests, and more sender mismatch than benign (label 0) mail.\n",
    "    \"\"\"\n",
    "    rng = np.random.default_rng(RANDOM_SEED)\n",
    "    half = 200\n",
    "    phishing = np.column_stack([\n",
    "        rng.integers(2, 11, size=half),          # num_links 2-10\n",
    "        rng.integers(2, 5, size=half),           # urgent_words 2-4\n",
    "        (rng.random(half) < 0.5).astype(int),    # has_attachment\n",
    "        (rng.random(half) < 0.8).astype(int),    # sender_mismatch (usually spoofed)\n",
    "        (rng.random(half) < 0.85).astype(int),   # cred_request (usually yes)\n",
    "    ])\n",
    "    benign = np.column_stack([\n",
    "        rng.integers(0, 5, size=half),           # num_links 0-4 (overlaps phishing low end)\n",
    "        rng.integers(0, 4, size=half),           # urgent_words 0-3 (overlaps phishing)\n",
    "        (rng.random(half) < 0.5).astype(int),    # has_attachment\n",
    "        (rng.random(half) < 0.1).astype(int),    # sender_mismatch (rarely)\n",
    "        (rng.random(half) < 0.1).astype(int),    # cred_request (rarely)\n",
    "    ])\n",
    "    X = np.vstack([phishing, benign]).astype(float)\n",
    "    y = np.concatenate([np.ones(half, dtype=int), np.zeros(half, dtype=int)])\n",
    "    return X, y\n",
    "\n",
    "\n",
    "X, y = build_dataset()\n",
    "model = LogisticRegression(max_iter=5000, random_state=RANDOM_SEED).fit(X, y)\n",
    "\n",
    "\n",
    "def score_email(email):\n",
    "    \"\"\"Return the model's phishing probability for one email feature dict.\"\"\"\n",
    "    row = np.array([[float(email[f]) for f in FEATURES]])\n",
    "    return float(model.predict_proba(row)[0, 1])\n",
    "\n",
    "\n",
    "# A realistic starting phishing email the filter correctly flags (score >= 0.5).\n",
    "STARTING_EMAIL = {\n",
    "    \"num_links\": 6,\n",
    "    \"urgent_words\": 3,\n",
    "    \"has_attachment\": 1,\n",
    "    \"sender_mismatch\": 1,\n",
    "    \"cred_request\": 1,\n",
    "}\n",
    "\n",
    "\n",
    "def check(email):\n",
    "    \"\"\"Reveal the flag only if `email` evades the filter while staying a\n",
    "    credible phishing attempt.\n",
    "\n",
    "    Success condition:\n",
    "      score_email(email) < 0.5            (classifier calls it BENIGN)\n",
    "      AND cred_request == 1               (still steals credentials)\n",
    "      AND urgent_words  >= 1              (still applies pressure)\n",
    "      AND num_links     >= 1              (still has a link to click)\n",
    "    \"\"\"\n",
    "    prob = score_email(email)\n",
    "    realistic = (int(email.get(\"cred_request\", 0)) == 1 and\n",
    "                 int(email.get(\"urgent_words\", 0)) >= 1 and\n",
    "                 int(email.get(\"num_links\", 0)) >= 1)\n",
    "    if prob < 0.5 and realistic:\n",
    "        print(\"Classifier verdict: BENIGN (phishing probability = %.3f)\" % prob)\n",
    "        print(\"...but this email still requests credentials, applies urgency, and\")\n",
    "        print(\"contains a link. You slipped a real phishing email past the filter.\")\n",
    "        print(\"SUCCESS! Flag: CIS350{f34ture_ev4sion_beats_the_filter}\")\n",
    "    elif not realistic:\n",
    "        print(\"Phishing probability = %.3f, but this is not a credible phishing email.\" % prob)\n",
    "        print(\"Keep it dangerous: require cred_request == 1, urgent_words >= 1, num_links >= 1.\")\n",
    "    else:\n",
    "        print(\"Still flagged as PHISHING (probability = %.3f >= 0.5).\" % prob)\n",
    "        print(\"Reduce the easily-spoofed features the filter over-relies on and try again.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 2 - Confirm the filter catches the starting email\n",
    "The starting phishing email should score >= 0.5 (quarantined). Run this to see the baseline."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "print(\"Starting email:\", STARTING_EMAIL)\n",
    "print(\"Phishing probability:\", round(score_email(STARTING_EMAIL), 3))\n",
    "# >= 0.5 means the filter flags it as phishing."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 3 - Your attack (TODO)\n",
    "Start from `STARTING_EMAIL` and change feature values to push the probability BELOW 0.5 while keeping the email dangerous (cred_request=1, urgent_words>=1, num_links>=1).\n",
    "\n",
    "Hints:\n",
    "- Which features can a real attacker change freely without giving up the scam?\n",
    "- The number of links, whether there is an attachment, and whether the sender is spoofed are all cosmetic - the attacker can drop them and still phish.\n",
    "- Try lowering `num_links`, setting `has_attachment` to 0, and setting `sender_mismatch` to 0.\n",
    "- Watch `score_email(...)` after each change to see the probability fall."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "# TODO: build your evasion email. Copy the starting email, then adjust features.\n",
    "my_email = dict(STARTING_EMAIL)   # start from the known phishing email\n",
    "\n",
    "# Change the easily-spoofed features here (keep cred_request=1, urgent_words>=1, num_links>=1):\n",
    "my_email[\"num_links\"] = ____\n",
    "my_email[\"has_attachment\"] = ____\n",
    "my_email[\"sender_mismatch\"] = ____\n",
    "\n",
    "print(\"Your email:\", my_email)\n",
    "print(\"Phishing probability:\", round(score_email(my_email), 3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 4 - Grade it\n",
    "Run `check(my_email)`. If your email is both benign to the filter AND still a credible phishing attempt, it prints your flag. Otherwise it prints a hint - adjust and retry."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "check(my_email)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What to submit\n",
    "Submit (in Canvas): (1) your winning `my_email` feature dictionary, (2) the flag string printed by `check()`, and (3) one or two sentences explaining WHY the filter was fooled and naming ONE defense that would stop this evasion (see the README)."
   ]
  }
 ]
}