{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Demo: Gemini 3 Flash answering one CHNB question with the full movie\n",
    "\n",
    "Single-question end-to-end run. Picks the first question in the benchmark, sends the source mp4 to Gemini 3 Flash via Vertex AI, and prints the model's answer alongside the gold letter.\n",
    "\n",
    "## Gemini/Vertex specific\n",
    "\n",
    "Gemini official eval typically sets thinking level to `HIGH`, but empirically we find no statistically significant difference between `LOW` and `HIGH` for Gemini 3 Flash -- if you want to save time/budget for the demo, you might want to start with `LOW`.\n",
    "\n",
    "We recommend running this on Gemini Enterprise Agent Platform (Vertex AI) for stability and lower latency. We would also recommend turning on caching if you want to experiment with multiple videos (each associated with multiple questions); see:\n",
    "[https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/context-caching/intro_context_caching.ipynb](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/context-caching/intro_context_caching.ipynb).\n",
    "\n",
    "Vertex AI API changes every now and then -- check [official repo](https://github.com/GoogleCloudPlatform/generative-ai/tree/main/gemini) if things no longer work (we only support the codebase dirrectly related to CHNB).\n",
    "\n",
    "## Prerequisites\n",
    "\n",
    "```bash\n",
    "gcloud auth application-default login\n",
    "export GOOGLE_CLOUD_PROJECT=your-gcp-project\n",
    "```\n",
    "\n",
    "and at least one matching mp4 under `data/mp4/`. Run `bash scripts/download_movies.sh` or download just the first movie for the quickest demo path."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os, sys\n",
    "from pathlib import Path\n",
    "\n",
    "# Add project root to sys.path so `import util.*` works regardless of where Jupyter was launched.\n",
    "_PROJECT = Path.cwd().resolve()\n",
    "while not (_PROJECT / 'util').is_dir() and _PROJECT != _PROJECT.parent:\n",
    "    _PROJECT = _PROJECT.parent\n",
    "sys.path.insert(0, str(_PROJECT))\n",
    "print(f'Project root: {_PROJECT}')\n",
    "\n",
    "from util.benchmark import load_benchmark\n",
    "from util.paths import BENCHMARK_PATH, VIDEO_DIR\n",
    "from util.parsers import parse_mcq_json\n",
    "from util.prompts import HEADER_VIDEO_CLIP, mcq_prompt_json\n",
    "\n",
    "df = load_benchmark(BENCHMARK_PATH)\n",
    "print(f'{len(df)} questions across {df[\"video_file_id\"].nunique()} movies')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Pick the first question whose mp4 is on disk.\n",
    "available = []\n",
    "for vid in df['video_file_id'].drop_duplicates():\n",
    "    if (VIDEO_DIR / f'{vid}.mp4').exists():\n",
    "        available.append(vid)\n",
    "if not available:\n",
    "    raise FileNotFoundError(\n",
    "        f'No mp4s found under {VIDEO_DIR}. Run `bash scripts/download_movies.sh` first '\n",
    "        f'(or place a single mp4 there to test).'\n",
    "    )\n",
    "vid = available[0]\n",
    "row = df[df['video_file_id'] == vid].iloc[0].to_dict()\n",
    "print(f'video_file_id : {vid}')\n",
    "print(f'category      : {row[\"category\"]}')\n",
    "print(f'question      : {row[\"question\"]}')\n",
    "for L in 'ABCD':\n",
    "    print(f'  ({L}) {row[f\"option_{L}\"]}')\n",
    "print(f'gold          : {row[\"gold\"]}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Initialize Gemini client.\n",
    "from google import genai\n",
    "from google.genai import types\n",
    "from google.genai.types import GenerateContentConfig\n",
    "\n",
    "PROJECT_ID = os.environ.get('GOOGLE_CLOUD_PROJECT')\n",
    "if not PROJECT_ID:\n",
    "    raise RuntimeError(\n",
    "        'GOOGLE_CLOUD_PROJECT is not set. Run:\\n'\n",
    "        '  gcloud auth application-default login\\n'\n",
    "        '  export GOOGLE_CLOUD_PROJECT=your-gcp-project'\n",
    "    )\n",
    "\n",
    "client = genai.Client(vertexai=True, project=PROJECT_ID, location='global')\n",
    "MODEL_ID = 'gemini-3-flash-preview'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Build the request.\n",
    "video_path = VIDEO_DIR / f'{vid}.mp4'\n",
    "with open(video_path, 'rb') as f:\n",
    "    video_bytes = f.read()\n",
    "print(f'Loaded {len(video_bytes) / 1e6:.1f} MB from {video_path.name}')\n",
    "\n",
    "prompt = mcq_prompt_json(row, header=HEADER_VIDEO_CLIP)\n",
    "print('--- PROMPT ---')\n",
    "print(prompt[:500] + ('...' if len(prompt) > 500 else ''))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Call Gemini 3 Flash with the mp4 inlined as a Part. This may take 1–3 minutes.\n",
    "contents = [\n",
    "    types.Part.from_bytes(data=video_bytes, mime_type='video/mp4'),\n",
    "    prompt,\n",
    "]\n",
    "\n",
    "config = GenerateContentConfig(\n",
    "    system_instruction='You answer movie MCQs. Output only valid JSON.',\n",
    "    temperature=0.0,\n",
    "    max_output_tokens=24576,\n",
    "    response_mime_type='application/json',\n",
    "    thinking_config=types.ThinkingConfig(thinking_level=types.ThinkingLevel.HIGH),\n",
    ")\n",
    "\n",
    "response = client.models.generate_content(model=MODEL_ID, contents=contents, config=config)\n",
    "raw = (response.text or '').strip()\n",
    "print('--- RAW RESPONSE ---')\n",
    "print(raw)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Parse the answer.\n",
    "pred = parse_mcq_json(raw, qid=row['qid'])\n",
    "print(f'Gold: {row[\"gold\"]}')\n",
    "print(f'Pred: {pred}')\n",
    "print('Correct' if pred == row['gold'] else 'Wrong')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}