Part 3: Assessment Solver I: DOM Scraping, Gemini Fallback & Two-Pass Critic
Part 3: Assessment Solver I — DOM Scraping, Gemini Fallback & Two-Pass Critic
Most browser-based AI quiz extensions get the easy definition questions right and fall apart the moment an exam includes Python code, a SQL table, exponents, or a trick negation word like EXCEPT. Usually, that isn't even the AI model's fault—it happens because the extension scraped the page with element.textContent, sent the mangled string to a single model without a thinking budget, and blindly trusted whatever option index came back first.
While testing early builds of SpringBored - Assessment across Python, Java, SQL, Finance, and Quantitative Aptitude assessments on Infosys Springboard, I ran into four specific failure modes:
1. Lost Python indentation: Calling .textContent on an Ace Editor (.ace_line), Monaco (.view-line), or <pre><code> block strips newlines and leading spaces. A 6-line nested for/if/else Python snippet collapses into one flat line where return looks like it belongs to the wrong block.
2. Corrupted exponents (10^6 turning into 106): In HTML, 10<sup>6</sup> has a .textContent of "106", and O(n<sup>2</sup>) becomes "O(n2)". On any math, physics, or complexity question, that guarantees the wrong answer.
3. Flattened SQL and finance tables: Grabbing .textContent from an HTML <table>mashes all the numbers together without column or row boundaries.
4. Mid-exam HTTP 429 stalls: Free-tier Google AI Studio API keys have per-minute rate limits on individual models. If your extension hardcodes a single model ID and hits a 429 on Question 16 of 30, the solver dies mid-test.
In this third part of the Building SpringBored series, we will look at how commits 0fddc37, b6657f4, a46092b, and b829e49 solved all four problems.
1. Scraping Code, Tables, Math, and Diagrams Without Losing Structure
Instead of calling .textContent, SpringBored - Assessment/content.js walks the question and option DOM nodes with a custom serializer that keeps structural formatting intact:
- Code blocks & web editors: Detects
<pre>,<code>, Ace Editor (.ace_line), Monaco (.view-line), and CodeMirror (.cm-line) lines, converts non-breaking spaces (\u00a0) to normal spaces, preserves exact leading indentation on every line, and wraps the snippet in Markdown code fences (```````). - HTML
<table>to Markdown tables: Walks<tr>rows and<th>/<td>cells and formats them into pipe-delimited Markdown tables (| Col A | Col B |) so Gemini can trace SQL joins and accounting tables accurately. - Exponents, subscripts, and LaTeX: Reads
<annotation encoding="application/x-tex">,data-latex, andaria-labelon MathJax/KaTeX elements, and converts<sup>x</sup>into^{x}and<sub>i</sub>into_{i}so10<sup>6</sup>is sent to Gemini as10^{6}instead of106. - Inline diagrams via Gemini Vision (
<img>&<canvas>): If a question contains a flowchart, circuit diagram, or chart inside an<img>or<canvas>, the scraper captures up to two images as Base64inlineDatapayloads and sends them alongside the text prompt. - Stripping option prefixes (
b829e49): Some Springboard and Techademy questions include literal prefixes like"A. "or"1) "inside the option text itself. Stripping those prefixes during normalization stops an option starting with"B. O(n log n)"from being confused with Option B when it sits at Index 0.
2. Surviving Free-Tier Rate Limits with a 7-Model Gemini Cascade
Because Google AI Studio allocates separate free-tier rate limits per model family, getting rate-limited (HTTP 429) on gemini-2.5-flash does not mean your API key is out of quota on gemini-3-flash-preview or gemini-2.5-pro.
In SpringBored - Assessment/background.js, every solve request runs through a prioritized model cascade (PREFERRED_FALLBACK_MODELS) plus dynamic /models discovery:
const PREFERRED_FALLBACK_MODELS = [
'gemini-2.5-flash',
'gemini-3-flash-preview',
'gemini-2.5-pro',
'gemini-3.1-pro-preview',
'gemini-2.5-flash-lite',
'gemini-2.0-flash',
'gemini-2.0-flash-lite',
'gemini-flash-latest',
'gemini-pro-latest'
];
Here is how the fallback loop keeps an assessment moving without stalls:
1. Remembering lastWorkingModel: Whenever a model responds with HTTP 200 OK, background.js saves it in lastWorkingModel and puts it at the front of modelsToTry for the next question. That way, if your primary model hits a per-minute limit on Question 10 and gemini-3-flash-preview takes over, Question 11 goes straight to gemini-3-flash-preview without wasting a round-trip hitting a 429 first.
2. Configuring thinkingBudget (4096 Pro / 2048 Flash): For reasoning models (gemini-2.5-flash and gemini-2.5-pro), buildGenerationConfig() enables deterministic output (temperature: 0.0), structured JSON (responseMimeType: "application/json"), and a dedicated thinkingBudget:
function buildGenerationConfig(modelName, useJsonMime = true, useThinking = true) {
const lower = String(modelName || '').toLowerCase();
const supportsThinkingBudget =
useThinking &&
(lower === 'gemini-2.5-flash' ||
lower === 'gemini-2.5-pro' ||
lower.startsWith('gemini-2.5-flash-preview') ||
lower.startsWith('gemini-2.5-pro-preview'));
const cfg = { temperature: 0.0 };
if (useJsonMime) cfg.responseMimeType = 'application/json';
if (supportsThinkingBudget) {
cfg.thinkingConfig = {
thinkingBudget: lower.includes('pro') ? 4096 : 2048
};
}
return cfg;
}
If one of the fallback models rejects thinkingConfig with HTTP 400, the client automatically retries JSON mode without thinkingConfig before moving on.
3. Dynamic /v1beta/models discovery & direct content.js fallback: If the first two models return 429, background.js queries GET /v1beta/models, filters out non-text models (imagen, veo, embedding, tts, aqa), ranks the rest with scoreGeminiModel(), and appends them to the retry list. And if Chrome's background service worker is ever asleep mid-exam, content.js has a built-in direct fallback (solveQuestionDirectlyInContent()) so no question gets skipped.
3. Catching Off-by-One Errors & Negation Traps with the Two-Pass Critic
Even with a thinking budget, LLMs occasionally slip up on two things during exams:
- Off-by-one index mistakes: In its reasoning, the model explains why
"Option B"is right, writes out the exact text of Option B, and then accidentally outputs[2]instead of[1]inselected_indices. - Missing a
NOTorEXCEPTin the question stem: The prompt asks "Which of the following is NOT a valid Python dictionary key?" and the model picks the first valid key.
In commit b6657f4, I added Index-vs-Text Reconciliation (extractVerifiedIndicesFromReply) and an automatic Two-Pass Self-Critic:
Cross-Checking selected_indices Against selected_option_texts
We require Gemini to output both the 0-based integer array (selected_indices) and the verbatim text of the chosen options (selected_option_texts), along with question_polarity (POSITIVE or NEGATIVE):
{
"question_polarity": "NEGATIVE (asks for NOT/FALSE/EXCEPT)",
"option_analysis": [
"Index 0 (Option A): [True] Tuples of immutable elements are hashable",
"Index 1 (Option B): [False] Lists are mutable and raise TypeError: unhashable type: 'list'"
],
"reasoning": "The question asks which is NOT a valid key, so Index 1 (list) is the answer.",
"selected_indices": [1],
"selected_option_texts": ["[1, 2, 3]"],
"confidence": "high",
"explanation": "Lists are mutable and cannot be used as dictionary keys in Python."
}
When extractVerifiedIndicesFromReply() parses that response, it matches selected_option_texts directly against the actual DOM option strings (matchOptionTextToIndex(), with a guard requiring at least 12 characters before substring matching so "O(n)" never substring-matches "O(n log n)"). If the integer index in selected_indices disagrees with the text match in selected_option_texts, it flags indexDisagreedWithText = true and triggers a second verification pass.
When the Critic Pass Runs
To keep straightforward single-choice questions fast (~1.2s) while double-checking tricky ones, the second-pass critic runs automatically whenever any of these conditions are true:
const hasNegationOrCode =
/\b(not|except|false|incorrect|invalid|never|cannot)\b/i.test(safeQuestion) ||
safeQuestion.includes('```');
const needsCriticPass =
isMultipleSelect ||
confidence === 'low' ||
confidence === 'medium' ||
indexDisagreedWithText ||
hasNegationOrCode;
When needsCriticPass triggers, background.js sends a second criticPrompt with the question, options, and proposed answer, asking the verifier to check whether a negative word (NOT, EXCEPT, FALSE) was misread, re-trace any code/SQL/math snippet, and confirm that selected_indices and selected_option_texts match before clicking the option on screen.
Quick Takeaways
- Never use raw
.textContentto scrape technical questions: Preserving code indentation, Markdown tables, and<sup>/<sub>exponents makes a bigger accuracy difference than switching models. - Always make the LLM output both the option index and the verbatim option text: Cross-checking the two catches off-by-one indexing bugs immediately.
Series Navigation: ← Previous: Part 2: Course Edition — Angular TOC Trees, 16× Video Beacons & Modals | Next: Part 4: Assessment Solver II — Cross-Origin Techademy Iframes, IAP & Fullscreen HUD → | All Articles