Language-model features can predict both held-out model activity and human fMRI responses. This project asks the harder question: do those targets rely on the same features in the same way?
01 · Evolution
One research idea, developed in two forms.
The thesis and the paper belong to the same research program: testing whether sparse language-model features provide a meaningful bridge between model activity and human responses to spoken stories. They differ in scope and purpose.
Bachelor thesis · broad and exploratory
Build the research system and investigate the space.
The thesis was the more open-ended version of the work. It included dataset and stimulus selection, exploratory fMRI analysis, transcript and timing infrastructure, SAE feature discovery, qualitative inspection, and focused feature probing.
Compared candidate naturalistic fMRI datasets before selecting the paired Shapes stories.
Built auditable feature evidence from transcript activations, Dolma contexts, judge summaries, and local alignment records.
Tested individual feature hypotheses with synthetic probes, counterfactual edits, and optional steering interventions.
Explored complementary model-side, sensitivity, and behavioral-benchmark analyses.
What the thesis found
In the cleaned 48-run shapesphysical analysis, the three completed model analyses converged at about r = .17 on the brain side with a small positive R² ≈ .034. Feature-importance correlations looked high (.909–.961), but sample-level RSA remained low (.019–.067). The exploratory result suggested overlap in feature salience, not matching moment-by-moment representational geometry.
Paper · focused and polished
Turn the strongest idea into one defensible test.
The paper trims the exploratory branches and centers the cleanest question: after a sparse feature bank transfers to a second description, do the brain and the model depend on those coordinates in the same order?
Freezes feature selection on the source description before either target or the held-out description is examined.
Measures reliance through held-out conditional and grouped ablation, not coefficient magnitude.
Tests correspondence against temporal and feature-identity nulls.
Uses simulations with planted agreement to check whether the test can detect correspondence at all.
What the paper found
The source-selected basis still transferred, although BOLD prediction was weak (r = .068–.092). But the held-out reliance rankings did not agree: at 256 features, the mean across twelve model-direction conditions was ρ = −.056 and no better than the fully retuned temporal null (p = .805). Simulations then show why this null is inconclusive: with 256 features, the test detects planted agreement in at most about 10% of runs, even when it is strong. The result cannot rule out shared organization between brain and model.
Why the result changed
The thesis result came from an exploratory, cleaned single-story comparison of fitted feature importance. The paper raises the bar: features are selected on one description, transferred unchanged to the other, and judged by the loss caused by removing them in held-out data. It also controls for shared story timing and correlated features. Under that stricter test, the earlier apparent salience agreement does not generalize. That difference is the intellectual progression of the project, not an attempt to hide the first result.
02 · Question
Prediction is not the same as correspondence.
The tempting claim
If one sparse feature bank predicts activity in a language model and in the brain, it is tempting to call that shared representation.
The actual test
Freeze the features selected on one spoken description, transfer them unchanged to a second description of the same events, and compare which features each target truly depends on.
03 · Pipeline
From spoken stories to a calibrated correspondence test.
01
Select a defensible paired stimulus
Exploratory analysis compared candidate naturalistic fMRI datasets and their timing quality. The final design uses shapesphysical and shapessocial: different spoken descriptions of the same animation, heard by the same 59 participants.
02
Build a shared time axis
Audio and transcripts are normalized, long recordings are chunked for Whisper word timing, and Gentle/Kaldi forced alignment refines word and phoneme boundaries. Words and model-specific tokens are then mapped to the scanner's 1.5-second TR grid.
03
Construct clean brain targets
fMRIPrep outputs are summarized into 200 Schaefer cortical parcels. Motion and physiological nuisance terms are regressed out; high-motion, high-DVARS, and non-steady-state response rows are censored while the complete timing grid remains intact.
04
Extract model states and sparse features
Each timed transcript is processed in causal context windows. Hidden states and residual-stream SAE activations are retained across six depth-spaced layers, then aggregated from tokens to TRs alongside the model's final hidden-state target.
05
Discover, inspect, and probe features
The thesis ranks transcript-relevant features by temporal activity and preserves their strongest tokens, spans, sentences, corpus contexts, and judge summaries. Selected features can then be tested with synthetic examples, counterfactual edits, and steering as an auditable interpretability track.
06
Freeze the source-only feature bank
For the paper's decisive test, feature identities are chosen from one description without looking at BOLD, model targets, or the second description. Nested banks from 64 to 1,024 features are carried unchanged into the paired telling.
07
Fit matched ridge models
The same sparse coordinates predict two different outcomes: lagged cortical BOLD and the model's unlagged final hidden state. Contiguous temporal folds, nested penalty selection, training-only scaling, and a purge around BOLD test blocks reduce leakage.
08
Ablate, compare, and calibrate
Each feature—or correlated feature group—is removed only in held-out data to measure the loss in predictive performance. Brain and model reliance rankings are compared in both transfer directions, against fully retuned temporal nulls, identity nulls, robustness sweeps, and a semi-synthetic power analysis.
04 · Brain data
What the fMRI signal actually looks like.
The analysis does not treat “the brain” as one score. It begins with quality-controlled time series, preserves the full story timeline through censoring, and models responses across a 200-parcel cortical atlas.
From raw movement to usable signal. The top panels show motion-based quality control and the cleaned parcel-by-time BOLD matrix. The cortical views show where participants share a reliable stimulus-locked response, with the strongest consistency around superior temporal regions.Where the transferred feature bank predicts. Held-out prediction is weak overall but not spatially uniform. The cortical maps and parcel distribution show where the source-selected sparse basis carries predictive signal on the second description.
05 · Scale
Enough variation to test the idea across systems.
Data
59 participants from Narratives fMRI
200 Schaefer cortical parcels
Two spoken descriptions of the same animation
Models and controls
Six SAE systems across three model families
Model sizes from 1B to 12B parameters
Temporal and feature-identity nulls
06 · Result
Transfer survives; the comparison is inconclusive.
≤ 10%
Detection of planted agreement at 256 features, against an 80% target: the test is underpowered.
r ≈ .07–.09
The frozen feature basis weakly predicts held-out BOLD responses.
ρ = −.056
The observed BOLD–model feature-profile agreement is no better than the temporal null (p = .805).
07 · Interpretation
A narrower, more defensible conclusion.
What the work supports
A feature basis selected from one description contains signal that transfers to another description for both targets.
What it does not establish
It does not show that brain and model rely on different features. Because the test rarely recovers planted agreement, the null cannot separate shared organization from its absence. A recovery check has to come before such a null is interpreted.
