Week 13
Agentic Router (in progress)
Month 4 focus: we are designing and trying an agentic query router on top of QUID. The goal is not to replace diffusion expansion, but to learn when to call it, when to try HyDE-style expansion, and when to leave the query alone. Early pilots are underway; the preprint is still forthcoming.
Place in the arc
Month 4 focus: we are designing and trying an agentic query router on top of QUID. The goal is not to replace diffusion expansion, but to learn when to call it, when to try HyDE-style expansion, and when to leave the query alone. Early pilots are underway; the preprint is still forthcoming.
Committee asked for an agentic AI angle. QUID already showed domain-dependent effectiveness (helps medical/finance; can hurt SciFact-style claims). The research bet we are pursuing:
QUID is a tool. An agent should choose when to call it.
This Month 4 work is active research - plans and early pilots, not a finished product claim.
Objectives
These objectives define what success looked like for Week 13. Meeting them either unlocked the next experiment, closed a risk, or produced evidence for the thesis claim that diffusion expansion is domain-dependent.
- [x] Design agentic layer on top of QUID (not a rewrite of the core thesis)
- [x] Implement first-pass query router: observe → decide → act
- [x] Wire Modal benchmark entrypoint
- [x] Fix Modal import/GPU blockers (A100 Dream-7B registration, transformers pin)
- [ ] Expand pilots and confirm routing behavior (continues into Week 14+)
What we built and measured
Below is the detailed record for the week - methods, implementation notes, and experimental setup - expanded from the team log so a reader can follow the technical path without the repository open.
Week 13 Progress Report - Agentic Router (Month 4 kickoff)
Project: QUID - Diffusion-Based Query Expansion for Dense Information Retrieval
Student: Shreyas S
Week: 13 (Month 4 · starting)
Focus: Design and stand up an agentic query-router layer on top of QUID
Objectives for This Week
- [x] Design agentic layer on top of QUID (not a rewrite of the core thesis)
- [x] Implement first-pass query router: observe → decide → act
- [x] Wire Modal benchmark entrypoint
- [x] Fix Modal import/GPU blockers (A100 Dream-7B registration, transformers pin)
- [ ] Expand pilots and confirm routing behavior (continues into Week 14+)
Motivation (Review feedback)
Committee asked for an agentic AI angle. QUID already showed domain-dependent effectiveness (helps medical/finance; can hurt SciFact-style claims). The research bet we are pursuing:
QUID is a tool. An agent should choose when to call it.
This Month 4 work is active research - plans and early pilots, not a finished product claim.
What we are building
1. Agentic Query Router (src/agentic_router.py)
First-pass routes each query to one of:
| Action | When (heuristic draft) |
|---|---|
vanilla | Query already specific / high retrieval confidence |
quid | Vocabulary gap (medical/finance cues, short queries) |
hyde | Knowledge/claim gap (science-claim cues, questions) |
Signals we are trying: token length, medical/finance/science term hits, question form, optional max corpus similarity.
2. Modal Benchmark (run_agentic_router_benchmark)
Comparing on BEIR slices:
always_vanilla/always_quid/always_hydeagentic_router(heuristic agent)oracle_router(upper bound)
Lean entrypoint: src/run_agentic_benchmark.py (avoids registering unrelated A100 functions).
3. Infrastructure Fixes
- Dream-7B A100 decorator blocked the whole app without A100 billing → lean entrypoint + lazy experiment imports
- LLaDA + new transformers broke on
all_tied_weights_keys→ pintransformers==4.38.2for agentic image - OpenAI HyDE quota exhausted → LLaDA document-style HyDE fallback
Challenges
| Challenge | Resolution |
|---|---|
| Modal token missing in env | Re-authenticated (modal token new) |
| A100 billing gate | L4 GPU + lean entrypoint |
| OpenAI 429 | Local LLaDA HyDE prompt |
| Over-routing to QUID | Added retrieval-confidence gate (still tuning) |
Plan for Week 14+
- Run NFCorpus + SciFact agentic pilots on Modal
- Treat early numbers as signals, not final claims
- Scale experiments; move toward a learned router
- Polish preprint materials (coming soon)
Weekly Status: MONTH 4 IN PROGRESS
Numbers and what they mean
Metrics are only useful with interpretation. Where tables appear, read the deltas as claims about when QUID helps (vocabulary gap) versus when HyDE or vanilla wins (knowledge / claim gap). Agentic weeks emphasize tool-call policies over blind expansion.
Challenges
| Challenge | Resolution |
|---|---|
| Modal token missing in env | Re-authenticated (modal token new) |
| A100 billing gate | L4 GPU + lean entrypoint |
| OpenAI 429 | Local LLaDA HyDE prompt |
| Over-routing to QUID | Added retrieval-confidence gate (still tuning) |
Key learnings
See work completed for takeaways.
Plan for next week
- Run NFCorpus + SciFact agentic pilots on Modal
- Treat early numbers as signals, not final claims
- Scale experiments; move toward a learned router
- Polish preprint materials (coming soon)
Weekly Status: MONTH 4 IN PROGRESS