Where does information actually flow in a transformer?
This post walks through our NeurIPS 2026 paper, Transfer Entropy as a Measure of Information Flow in VLMs and LLMs (Liang & Shi, 2026), joint work with Jianbo Shi at Penn. The paper asks a simple question about transformers: not where attention is placed, but how much new predictive information actually moves between layers, tokens, and heads. It then uses the answer to decide what to prune.
If you only want the headline: a cheap, closed-form estimate of transfer entropy recovers meaningful structure inside LLaVA, CLIP, and several language models, and pruning guided by that structure beats attention- and saliency-based baselines at matched budgets.
The poster is available if you prefer the one-page version.
The motivation
Transformers power both vision-language models (VLMs) and large language models (LLMs), yet the contributions of individual layers, tokens, and attention heads remain surprisingly opaque. The tool most people reach for is attention. Attention weights tell you where a token spreads its probability mass. They do not tell you whether that mass carries anything the receiving token didn’t already know. Static similarity between representations has the same problem: it exposes association without direction.
We wanted a quantity that is directed, that conditions on what the target already knows, and that can be measured at several structural scales using the same definition. Transfer entropy (TE) is exactly that quantity.
Transfer entropy, briefly
Transfer entropy from a source process \(X\) to a target process \(Y\) is the reduction in uncertainty about the target’s next state that comes from knowing the source, beyond what the target’s own history already provides:
\[\mathrm{TE}_{X \to Y} = I\!\left(Y_{t+1};\, X_t \mid Y_t\right).\]Two things make this a good fit for transformers. First, it is asymmetric, so \(\mathrm{TE}_{X \to Y}\) and \(\mathrm{TE}_{Y \to X}\) can differ, which lets us talk about senders and receivers. Second, the conditioning on \(Y_t\) discounts information that was already present in the target. A layer that just copies its input forward has high similarity to the next layer but low TE.
A tractable approximation
The obstacle is that hidden states live in thousands of dimensions, and estimating a conditional mutual information there is hopeless. We derive a surrogate under three local assumptions: Gaussian perturbations around the current representation, a linearized (Jacobian) response from one layer to the next, and a rank-one approximation of the covariance of those perturbations. Under these assumptions the TE across the transition from layer \(\ell\) to layer \(\ell+1\) collapses to an average of squared cosines:
\[\widehat{\mathrm{TE}}_{\ell} \;\approx\; \frac{1}{2(B-1)} \sum_{i=1}^{B-1} \cos^2\!\left(\Delta z_{\ell+1,i},\, \Delta z_{\ell,i}\right), \qquad \Delta z_{\ell,i} = z_{\ell,i+1} - z_{\ell,i}.\]Here \(z_{\ell,i}\) is the hidden state of probe item \(i\) at layer \(\ell\), and \(\Delta z\) is the difference between neighboring items in a fixed probe batch of size \(B\). In practice the recipe is:
- Capture activations. Cache layer representations for a fixed probe batch around an optimizer step.
- Form local differences. Compute the neighboring-item differences \(\Delta z_{\ell,i}\) at every layer.
- Estimate directional flow. Average ridge-stabilized squared cosines across pairs and steps.
No density estimation, no explicit Jacobians, and the cost is a handful of dot products per layer. This is what makes it practical to track TE during fine-tuning rather than only after.
From diagnostic to pruning decision
The same number gives three different pruning signals depending on what you compute it over:
- Layers. High averaged \(\mathrm{TE}_\ell\) means the transition into layer \(\ell+1\) is highly predictable from layer \(\ell\), so the next layer may add limited unique information and is a candidate for removal.
- Tokens. Computing TE in and out of each token position gives \(\mathrm{TE}_{\text{out}}\) and \(\mathrm{TE}_{\text{in}}\). Tokens with a large positive gap \(\mathrm{TE}_{\text{out}} - \mathrm{TE}_{\text{in}}\) are broadcasters: they push information outward but absorb little. We prune those.
- Attention heads. Combining head-level TE with CLS/EOT routing patterns classifies each head as a sink, a broadcaster, or benign. Low-TE benign heads are the ones we remove.
What the flow looks like
LLaVA-1.5-7B: where vision meets text
We track layerwise TE through the vision tower, the text tower, and the vision-to-text pathway while LoRA fine-tuning (rank 4) on MSCOCO and Flickr8k. Curves below show mean and standard deviation across 5 epochs.
The three pathways have distinct profiles. The cross-modal curve in particular is far from flat, which already tells you that “vision information enters the language model” is not something that happens uniformly across depth.
CLIP ViT-B/16: vision and language take different paths
The same measurement on CLIP during LoRA (rank 8) fine-tuning shows modality-dependent profiles. The vision and text towers do not share a redundancy pattern, which is the motivation for pruning them with separate budgets rather than symmetrically.
Tokens do not all play the same role
Zooming in to the token level on CLIP’s \(14 \times 14\) vision patches, we compare outgoing TE, incoming TE, average attention, and the resulting role classification.
The attention map and the TE maps disagree in instructive ways. Attention highlights where mass goes. The TE maps separate patches that genuinely transmit new information from patches that merely receive it, and that distinction is what the pruning step relies on.
Information flow becomes an efficiency tool
Decoder-layer pruning in LLaVA
We prune decoder layers of LLaVA-1.5-7B using the layer-level TE signal and evaluate VQAv2 soft accuracy. The unpruned model scores 0.6480. At every matched budget, TE-guided pruning retains more accuracy than Streamline and Short-LVLM.
| Layers pruned | TE pruning | Streamline | Short-LVLM |
|---|---|---|---|
| 2 | 0.5370 | 0.4498 | 0.4548 |
| 4 | 0.3798 | 0.2937 | 0.2832 |
| 6 | 0.2867 | 0.1352 | 0.2248 |
| 8 | 0.2737 | 0.1220 | 0.1472 |
Vision-token pruning in LLaVA
Using the broadcaster criterion (\(\mathrm{TE}_{\text{out}} - \mathrm{TE}_{\text{in}}\)), we prune vision tokens and compare against Top-ViM and SCOPE on VQAv2. The masks below show which patches are removed at 40% pruning; yellow marks a pruned position.
Layer pruning in CLIP
On MSCOCO retrieval, we remove matched numbers of vision (V) and text (T) layers from CLIP ViT-B/16 and compare against Short-LVLM and Streamline. TE pruning is best at every budget and in every recall column.
| Method | Pruned layers | I→T R@1 | I→T R@5 | I→T R@10 | T→I R@1 | T→I R@5 | T→I R@10 |
|---|---|---|---|---|---|---|---|
| No pruning | — | 42.20 | 66.21 | 74.94 | 26.66 | 48.94 | 58.91 |
| TE pruning | 2V + 2T | 27.36 | 51.44 | 62.78 | 16.67 | 36.50 | 47.03 |
| Short-LVLM | 2V + 2T | 25.65 | 49.34 | 60.63 | 15.91 | 35.16 | 45.65 |
| Streamline | 2V + 2T | 26.83 | 50.26 | 60.83 | 16.18 | 35.38 | 45.66 |
| TE pruning | 4V + 4T | 15.95 | 34.81 | 45.51 | 9.53 | 24.08 | 33.11 |
| Short-LVLM | 4V + 4T | 13.10 | 30.33 | 40.28 | 7.95 | 20.85 | 29.29 |
| Streamline | 4V + 4T | 12.02 | 28.49 | 38.18 | 7.57 | 20.03 | 28.37 |
| TE pruning | 5V + 5T | 11.79 | 28.42 | 38.68 | 7.07 | 19.49 | 27.92 |
| Short-LVLM | 5V + 5T | 8.02 | 20.62 | 28.81 | 4.87 | 14.19 | 20.91 |
| Streamline | 5V + 5T | 9.34 | 23.07 | 31.63 | 5.67 | 16.15 | 23.54 |
Classifying and pruning attention heads in CLIP
Head-level TE, combined with routing patterns through the CLS and EOT tokens, sorts every head in each tower into sink, broadcaster, or benign. Benign heads with low TE are removed first.
Pruning this way retains more retrieval performance than SmartTrim at both budgets we tested. Here \(x\)V + \(y\)T means \(x\) of the 12 vision heads and \(y\) of the 8 text heads are pruned per layer, with the first and last layers left intact.
| Pruning per layer | Method | I→T R@1 | I→T R@5 | I→T R@10 | T→I R@1 | T→I R@5 | T→I R@10 |
|---|---|---|---|---|---|---|---|
| None | No pruning | 42.20 | 66.21 | 74.94 | 26.66 | 48.94 | 58.91 |
| 3V + 2T | TE | 36.97 | 61.94 | 71.82 | 23.49 | 45.40 | 55.67 |
| 3V + 2T | SmartTrim | 34.92 | 59.79 | 69.78 | 21.78 | 43.12 | 53.34 |
| 5V + 3T | TE | 32.41 | 56.64 | 66.89 | 20.40 | 41.41 | 51.77 |
| 5V + 3T | SmartTrim | 30.53 | 54.42 | 64.87 | 19.16 | 39.48 | 49.66 |
Beyond vision-language: LLMs
None of this is specific to multimodal models. The same estimator applied to an encoder (RoBERTa-base), a decoder component (T5), and two decoder-only LLMs (Qwen-2.5-7B and Llama-3.2-3B) across eight GLUE datasets gives layerwise flow statistics for each architecture.
Takeaways
- TE reveals where information flows through VLMs and LLMs, with a direction and a magnitude, at the layer, token, and head level.
- The same signal identifies what is redundant: layers whose transitions add little, broadcaster tokens, and benign attention heads.
- TE-guided structured pruning preserves more predictive performance than the attention- and saliency-based baselines we tested, at matched budgets, on both VQA and retrieval.
The broader point I take from this project is that interpretability and efficiency do not have to be separate research programs. A measurement that is honest about direction and about what the target already knows tells you something true about the model, and that truth is directly actionable.
If you want to dig in, the poster has every figure above at full size, and the full paper is on OpenReview. Questions and pushback are welcome at jeliang@mit.edu.
References
- In Advances in Neural Information Processing Systems (NeurIPS), 2026