1. All Blogs 2. About Me 3. Contact Me 4. Visit My Website 5. Visit My Venture

Part 3: Assessment Solver I: DOM Scraping, Gemini Fallback & Two-Pass Critic

font_size 100%
font
contrast
listen
share

Part 3: Assessment Solver I — DOM Scraping, Gemini Fallback & Two-Pass Critic

Most browser-based AI quiz extensions get the easy definition questions right and fall apart the moment an exam includes Python code, a SQL table, exponents, or a trick negation word like EXCEPT. Usually, that isn't even the AI model's fault—it happens because the extension scraped the page with element.textContent, sent the mangled string to a single model without a thinking budget, and blindly trusted whatever option index came back first.

While testing early builds of SpringBored - Assessment across Python, Java, SQL, Finance, and Quantitative Aptitude assessments on Infosys Springboard, I ran into four specific failure modes:

1. Lost Python indentation: Calling .textContent on an Ace Editor (.ace_line), Monaco (.view-line), or <pre><code> block strips newlines and leading spaces. A 6-line nested for/if/else Python snippet collapses into one flat line where return looks like it belongs to the wrong block.

2. Corrupted exponents (10^6 turning into 106): In HTML, 10<sup>6</sup> has a .textContent of "106", and O(n<sup>2</sup>) becomes "O(n2)". On any math, physics, or complexity question, that guarantees the wrong answer.

3. Flattened SQL and finance tables: Grabbing .textContent from an HTML <table>mashes all the numbers together without column or row boundaries.

4. Mid-exam HTTP 429 stalls: Free-tier Google AI Studio API keys have per-minute rate limits on individual models. If your extension hardcodes a single model ID and hits a 429 on Question 16 of 30, the solver dies mid-test.

In this third part of the Building SpringBored series, we will look at how commits 0fddc37, b6657f4, a46092b, and b829e49 solved all four problems.

Assessment Solver AI Pipeline in SpringBored - Assessment

1. Scraping Code, Tables, Math, and Diagrams Without Losing Structure

Instead of calling .textContent, SpringBored - Assessment/content.js walks the question and option DOM nodes with a custom serializer that keeps structural formatting intact:

  • Code blocks & web editors: Detects <pre>, <code>, Ace Editor (.ace_line), Monaco (.view-line), and CodeMirror (.cm-line) lines, converts non-breaking spaces (\u00a0) to normal spaces, preserves exact leading indentation on every line, and wraps the snippet in Markdown code fences (``` ` ```).
  • HTML <table> to Markdown tables: Walks <tr> rows and <th>/<td> cells and formats them into pipe-delimited Markdown tables (| Col A | Col B |) so Gemini can trace SQL joins and accounting tables accurately.
  • Exponents, subscripts, and LaTeX: Reads <annotation encoding="application/x-tex">, data-latex, and aria-label on MathJax/KaTeX elements, and converts <sup>x</sup> into ^{x} and <sub>i</sub> into _{i} so 10<sup>6</sup> is sent to Gemini as 10^{6} instead of 106.
  • Inline diagrams via Gemini Vision (<img> & <canvas>): If a question contains a flowchart, circuit diagram, or chart inside an <img> or <canvas>, the scraper captures up to two images as Base64 inlineData payloads and sends them alongside the text prompt.
  • Stripping option prefixes (b829e49): Some Springboard and Techademy questions include literal prefixes like "A. " or "1) " inside the option text itself. Stripping those prefixes during normalization stops an option starting with "B. O(n log n)" from being confused with Option B when it sits at Index 0.

2. Surviving Free-Tier Rate Limits with a 7-Model Gemini Cascade

Because Google AI Studio allocates separate free-tier rate limits per model family, getting rate-limited (HTTP 429) on gemini-2.5-flash does not mean your API key is out of quota on gemini-3-flash-preview or gemini-2.5-pro.

In SpringBored - Assessment/background.js, every solve request runs through a prioritized model cascade (PREFERRED_FALLBACK_MODELS) plus dynamic /models discovery:

const PREFERRED_FALLBACK_MODELS = [
  'gemini-2.5-flash',
  'gemini-3-flash-preview',
  'gemini-2.5-pro',
  'gemini-3.1-pro-preview',
  'gemini-2.5-flash-lite',
  'gemini-2.0-flash',
  'gemini-2.0-flash-lite',
  'gemini-flash-latest',
  'gemini-pro-latest'
];

Here is how the fallback loop keeps an assessment moving without stalls:

1. Remembering lastWorkingModel: Whenever a model responds with HTTP 200 OK, background.js saves it in lastWorkingModel and puts it at the front of modelsToTry for the next question. That way, if your primary model hits a per-minute limit on Question 10 and gemini-3-flash-preview takes over, Question 11 goes straight to gemini-3-flash-preview without wasting a round-trip hitting a 429 first.

2. Configuring thinkingBudget (4096 Pro / 2048 Flash): For reasoning models (gemini-2.5-flash and gemini-2.5-pro), buildGenerationConfig() enables deterministic output (temperature: 0.0), structured JSON (responseMimeType: "application/json"), and a dedicated thinkingBudget:

   function buildGenerationConfig(modelName, useJsonMime = true, useThinking = true) {
     const lower = String(modelName || '').toLowerCase();
     const supportsThinkingBudget =
       useThinking &&
       (lower === 'gemini-2.5-flash' ||
         lower === 'gemini-2.5-pro' ||
         lower.startsWith('gemini-2.5-flash-preview') ||
         lower.startsWith('gemini-2.5-pro-preview'));

     const cfg = { temperature: 0.0 };
     if (useJsonMime) cfg.responseMimeType = 'application/json';
     if (supportsThinkingBudget) {
       cfg.thinkingConfig = {
         thinkingBudget: lower.includes('pro') ? 4096 : 2048
       };
     }
     return cfg;
   }

If one of the fallback models rejects thinkingConfig with HTTP 400, the client automatically retries JSON mode without thinkingConfig before moving on.

3. Dynamic /v1beta/models discovery & direct content.js fallback: If the first two models return 429, background.js queries GET /v1beta/models, filters out non-text models (imagen, veo, embedding, tts, aqa), ranks the rest with scoreGeminiModel(), and appends them to the retry list. And if Chrome's background service worker is ever asleep mid-exam, content.js has a built-in direct fallback (solveQuestionDirectlyInContent()) so no question gets skipped.

3. Catching Off-by-One Errors & Negation Traps with the Two-Pass Critic

Even with a thinking budget, LLMs occasionally slip up on two things during exams:

  • Off-by-one index mistakes: In its reasoning, the model explains why "Option B" is right, writes out the exact text of Option B, and then accidentally outputs [2] instead of [1] in selected_indices.
  • Missing a NOT or EXCEPT in the question stem: The prompt asks "Which of the following is NOT a valid Python dictionary key?" and the model picks the first valid key.

In commit b6657f4, I added Index-vs-Text Reconciliation (extractVerifiedIndicesFromReply) and an automatic Two-Pass Self-Critic:

Cross-Checking selected_indices Against selected_option_texts

We require Gemini to output both the 0-based integer array (selected_indices) and the verbatim text of the chosen options (selected_option_texts), along with question_polarity (POSITIVE or NEGATIVE):

{
  "question_polarity": "NEGATIVE (asks for NOT/FALSE/EXCEPT)",
  "option_analysis": [
    "Index 0 (Option A): [True] Tuples of immutable elements are hashable",
    "Index 1 (Option B): [False] Lists are mutable and raise TypeError: unhashable type: 'list'"
  ],
  "reasoning": "The question asks which is NOT a valid key, so Index 1 (list) is the answer.",
  "selected_indices": [1],
  "selected_option_texts": ["[1, 2, 3]"],
  "confidence": "high",
  "explanation": "Lists are mutable and cannot be used as dictionary keys in Python."
}

When extractVerifiedIndicesFromReply() parses that response, it matches selected_option_texts directly against the actual DOM option strings (matchOptionTextToIndex(), with a guard requiring at least 12 characters before substring matching so "O(n)" never substring-matches "O(n log n)"). If the integer index in selected_indices disagrees with the text match in selected_option_texts, it flags indexDisagreedWithText = true and triggers a second verification pass.

When the Critic Pass Runs

To keep straightforward single-choice questions fast (~1.2s) while double-checking tricky ones, the second-pass critic runs automatically whenever any of these conditions are true:

const hasNegationOrCode =
  /\b(not|except|false|incorrect|invalid|never|cannot)\b/i.test(safeQuestion) ||
  safeQuestion.includes('```');

const needsCriticPass =
  isMultipleSelect ||
  confidence === 'low' ||
  confidence === 'medium' ||
  indexDisagreedWithText ||
  hasNegationOrCode;

When needsCriticPass triggers, background.js sends a second criticPrompt with the question, options, and proposed answer, asking the verifier to check whether a negative word (NOT, EXCEPT, FALSE) was misread, re-trace any code/SQL/math snippet, and confirm that selected_indices and selected_option_texts match before clicking the option on screen.

Quick Takeaways

  • Never use raw .textContent to scrape technical questions: Preserving code indentation, Markdown tables, and <sup>/<sub> exponents makes a bigger accuracy difference than switching models.
  • Always make the LLM output both the option index and the verbatim option text: Cross-checking the two catches off-by-one indexing bugs immediately.

Series Navigation: ← Previous: Part 2: Course Edition — Angular TOC Trees, 16× Video Beacons & Modals | Next: Part 4: Assessment Solver II — Cross-Origin Techademy Iframes, IAP & Fullscreen HUD → | All Articles

ojas@blog:~$ share_post --id #blog_2026_26
post linkedin whatsapp

← Back to all articles