Projects

Research projects and artifacts. New work is being added here.

Small and Smaller

How Sub-10B Agent Behaviour and Errors Change with Size

2026

Asks what changes in an agent's behaviour and in its errors when one model family grows from 0.8B to 9B parameters and nothing else changes. Evaluates four Qwen3.5 checkpoints, each with thinking on and off on the same weights, across the three τ²-bench domains — 6,672 conversations in all. Two behavioural frameworks are joined turn by turn: the ACT-onomy action taxonomy applied by an LLM coder, and the τ²-bench error reviewer run with two judge models from different families. The mix of behaviours holds at the level of ten action groups but shifts across 46 sub-actions, and the kind of error shifts more than the amount: comprehension errors fall by an order of magnitude while guideline violations do not, appearing in six of ten of the 9B's successful conversations.

τ²-benchQwen3.5LLM-as-a-JudgeACT-onomy TaxonomyBehavioural Analysis
Under review — IAB Workshop, NeurIPS 2026
6,672 conversations, three domains
Four checkpoints, 0.8B to 9B
Project details in progress

CT Report Generators are Good at Gatekeeping

Mechanistic Interpretability of a Volumetric Vision-Language Model

2026

Opens CT-CHAT's decoder with linear probing, attention analysis, and activation steering, all at the single position where the model commits to its first report token. 78.6% of the findings present in its evaluation set go unreported (8,176 of 10,402), yet the evidence for most of them stays linearly decodable there from layer 1 through layer 32, over and above patient metadata and scan protocol — the model has the evidence and does not report it. Within volumes where a finding is present, the attention difference between reports that name it and reports that miss it correlates negatively with recall, and steering lifts lung nodule from 0 to 49.6% against 0.4% for a matched random direction while consolidation stays below 1%.

Linear ProbingActivation SteeringAttention AnalysisCT-CHAT3D Vision-Language Models
Under review — Interpretability for Discovery, NeurIPS 2026
78.6% of present findings unreported
Evidence decodable through layer 32
Project details in progress