Week 01
Environment & Literature
This week sits in Month 1 - building the substrate for QUID: environment, LLaDA porting, BEIR wiring, and the first signals of domain-dependent retrieval gains.
Place in the arc
This week sits in Month 1 - building the substrate for QUID: environment, LLaDA porting, BEIR wiring, and the first signals of domain-dependent retrieval gains.
Objectives
These objectives define what success looked like for Week 1. Meeting them either unlocked the next experiment, closed a risk, or produced evidence for the thesis claim that diffusion expansion is domain-dependent.
- [ ] Set up development environment and repository structure
- [ ] Review LLaDA paper and codebase in detail
- [ ] Begin porting LLaDA generation code
- [ ] Set up BEIR benchmark infrastructure
What we built and measured
Below is the detailed record for the week - methods, implementation notes, and experimental setup - expanded from the team log so a reader can follow the technical path without the repository open.
1. Environment Setup
- Initialized project repository with proper structure:
```
D-Search/
├── src/
│ ├── models/
│ ├── retrieval/
│ └── evaluation/
├── experiments/
├── data/
└── configs/
```
- Set up Python environment with PyTorch 2.x, HuggingFace Transformers
- Configured Modal.com account for GPU compute (A10G instances)
- Verified CUDA compatibility and bfloat16 support
2. Literature Deep Dive
- Completed detailed reading of LLaDA paper (Nie et al., 2025)
- Key insight: LLaDA uses a masked diffusion approach where tokens are progressively unmasked based on confidence scores
- Identified critical difference from standard HuggingFace
generate(): LLaDA requires custom sampling logic
3. LLaDA Code Analysis & Initial Port
- Cloned LLaDA repository and analyzed generation code
- Identified core functions needed for porting:
- forward_process() - adds masks to input
- reverse_process() - iterative unmasking
- confidence_sampling() - selects tokens to unmask
- Started porting generation code - 60% complete
4. BEIR Infrastructure Started
- Downloaded BEIR benchmark package
- Began setting up evaluation pipeline structure
- Identified all four target datasets
Numbers and what they mean
Metrics are only useful with interpretation. Where tables appear, read the deltas as claims about when QUID helps (vocabulary gap) versus when HyDE or vanilla wins (knowledge / claim gap). Agentic weeks emphasize tool-call policies over blind expansion.
1. Environment Setup
- Initialized project repository with proper structure:
```
D-Search/
├── src/
│ ├── models/
│ ├── retrieval/
│ └── evaluation/
├── experiments/
├── data/
└── configs/
```
- Set up Python environment with PyTorch 2.x, HuggingFace Transformers
- Configured Modal.com account for GPU compute (A10G instances)
- Verified CUDA compatibility and bfloat16 support
2. Literature Deep Dive
- Completed detailed reading of LLaDA paper (Nie et al., 2025)
- Key insight: LLaDA uses a masked diffusion approach where tokens are progressively unmasked based on confidence scores
- Identified critical difference from standard HuggingFace
generate(): LLaDA requires custom sampling logic
3. LLaDA Code Analysis & Initial Port
- Cloned LLaDA repository and analyzed generation code
- Identified core functions needed for porting:
- forward_process() - adds masks to input
- reverse_process() - iterative unmasking
- confidence_sampling() - selects tokens to unmask
- Started porting generation code - 60% complete
4. BEIR Infrastructure Started
- Downloaded BEIR benchmark package
- Began setting up evaluation pipeline structure
- Identified all four target datasets
Challenges
- Non-standard generation API: LLaDA doesn't use HuggingFace's
generate()method. Significant porting effort required.
- Documentation gaps: The LLaDA codebase has minimal documentation. Spent time tracing through code.
Key learnings
- Masked diffusion models generate text fundamentally differently from autoregressive models
- This global view is what I hypothesize will lead to more "anchored" expansions
Plan for next week
- Complete LLaDA generation port
- Finish BEIR evaluation infrastructure
- Download and preprocess all datasets
- Implement vanilla baseline
Hours logged
| Activity | Hours |
|---|---|
| Environment setup | 4 |
| Literature review | 6 |
| Code analysis & porting | 12 |
| BEIR setup | 6 |
| Documentation | 2 |
| Total | 30 |
Notes for advisor
Aggressive start to the project. LLaDA porting is complex but progressing well. On track for accelerated timeline.