What Attention Is Missing

A small transformer that adds numbers kept failing on longer numbers for reasons that had nothing to do with addition. Every failure traced back to one property of softmax attention: it can prefer, but never exclude, and every preference is relative to everything else in the sequence. Patching it by hand moves the failure around. Letting the model learn its own coordinates gets surprisingly far, until it runs into the precision of counting.

The earlier log ended with a fix: 2D rotary positions, a fixed decay in column distance and a position-free attention sink took a one-digit-per-token adder from failing at about 40 digits to 100% at 200 on every seed. This log follows what happened next: the same fix on 3-digit tokens, a new failure, and what it says about attention itself. The tokenization side of the study is written up separately in How Tokenization Decides Whether a Transformer Can Add.

A failure that wasn't aliasing

On 3-digit right-cut tokens, the fix gave two seeds near-perfect reach (.99 and 1.00 at 200 digits) and one seed far less (.49). Scoring every decision of 150 random 100-digit problems on its own, given the correct answer so far, seed 2 got 591 of 5250 decisions wrong; seeds 0 and 1 got 0 and 0. Its errors start abruptly around answer chunk 17, only where the first number has already ended, and attention to the target digit itself is unchanged.

What changed is where the remaining attention goes. The second number's chunks look for "the first number's chunk in my column", and once the first number has ended there is nothing to find. The decay pulls every far score toward zero, but a score of zero still gets weight e⁰ = 1 in the softmax, and the number of far keys grows with the column:

In every model, some heads end up giving most of their attention to the crowd of far keys. Only seed 2 decides "the first number is missing, treat it as 0" with those heads, so only seed 2 breaks, and removing far keys at evaluation removes all of its errors. Nothing here was aliasing: no single far key scores high. Many keys each score a little.

One sum explains every failure

Softmax gives key j a share proportional to e^score_j, so only score gaps matter. A key that beats each of n others by a gap g keeps a share of 1 / (1 + n·e^−g): to keep a majority it needs g > ln n.

Computed, not measured. A gap of 2 keeps a majority against at most about 7 other keys; holding against a thousand needs a gap of about 7.

Write the target's share as 1 / (1 + Σ_j e^−gap_j). Every failure in this project is a way for that leak sum to grow with length: aliasing is one far key with a gap near zero; drift is gaps at untrained distances that nothing constrains; dilution is many keys with modest gaps. Training on 10 digits only asks for the gaps 10 digits need, and the measured sink scores (about 0.7–2.5, against 50–100 for real lookups) show it learns exactly that much. For a head to behave the same at any length, the leak sum must stay bounded as the sequence grows, which requires a far key's gap to grow faster than log(distance). Rotary scores are periodic and bounded, so with rotation alone the sum always grows.

The same inequality is behind long-context methods in language models: Scalable-Softmax multiplies scores by a multiple of log(context length), and YaRN adds an attention temperature that grows with the log of the context-extension factor, both to keep attention from flattening as the context grows.

Patching it by hand moves the failure

Three eval-time patches, each a no-op at training lengths: raise the sink's score by log(n / n_train) or twice that, or drop all keys beyond the largest training distance (a window). Accuracy at 200 digits, seeds 0 / 1 / 2:

A stronger sink starves real lookups in models that did not need it; the window fixes the diluted seed and nothing else. Each patch encodes an assumption about how one particular model solved the task, and seeds solve it differently because 10-digit training cannot tell their mechanisms apart.

Can attention exclude?

Sparsemax and entmax replace softmax with a rule that gives keys sufficiently below the leaders exactly zero weight, however many there are. Swapped in at evaluation, seed 2 improves only partly (200 digits: .51 → .67): its sink scores about 0.7, and beyond the trained range the best far keys score as high or higher, so they are leaders, not a crowd to exclude. Trained with entmax-1.5 from scratch, fewer sink heads collapse past the trained range (6 of 23 vs 12 of 23 with softmax), but a head that does collapse loses everything, and reach is worse overall (200 digits: .07 / .94 / .93 vs .99 / 1.00 / .49). The normalization changes how scores become weights; the failure is in the scores at distances training never saw.

Content addressing and location addressing

Stepping back: a person adding on paper looks up digits by location ("same column, one row up"), an exact operation that means the same at any length. Attention only has content addressing (find the key most similar to the query) and fakes location by encoding positions as content, then reconstructing "one column ahead" as a similarity over distance from what training showed. Every failure above lives in that reconstruction. Earlier architectures had both kinds explicitly: the Neural Turing Machine shifted its read head by −1/0/+1, and the convolutional Neural GPU learned binary addition on short instances that generalized to much longer ones. Transformers traded explicit location for parallel training and generality.

Long training data does not close the gap either: in the tokenization log, adding 20–40 digit problems moved this model family's cliff from about 40 to about 160 digits, four times the longest training length both times.

Attention that counts its own coordinates

So the last experiment keeps attention general and gives it a way to build exact locations: no rotary encoding, no place index, no positions at all. Each head keeps two counters, c_t = (1 − reset_t)·c_t−1 + gate_t, with the gate and reset learned from the token's content (the same idea as contextual position encoding, CoPE). Position enters only through learned kernels of counter differences, free in shape within ±8 and forced to fall beyond, and each head has a position-free null key to attend to when nothing matches.

• It invents the place-value index. 13 / 11 / 6 counters per seed track "position within its own number" with correlation above 0.9, many at exactly one count per digit, and their kernels peak at 0 and −1: the same-column and next-column lookups addition needs. Nobody told it that columns, rows or separators exist.
• The null key is the sink. 18 / 18 / 21 heads per seed put over 30% of their attention on it, most of them unchanged from 8 to 120 digits; no head ever uses the first token as a sink.
• It learns addition (in-distribution 1.000 / 1.000 / 1.000, 24–26 digits 0.986 / 0.995 / 0.952), and reaches .93 / .92 / .24 at 40 digits, .74 / .69 / 0 at 60, 0 / 0 / 0 at 200.

The limit is the counting itself. A gate is a sigmoid, so a column counter adds slightly less than one per digit, and differently in the operand and answer rows. Lookups compare counts, so the error in "same column" grows with the column, and somewhere past 50 columns the head reads the neighbouring digit. Rounding only the column counters' gates to exactly 0 or 1 at evaluation, with no retraining:

Seeds 0 and 1. Seed 2 does not route its lookups through its column counters and is unchanged.

Required precision grows with length (an error of ε per digit is half a column off after 1/(2ε) digits) while training at 10 digits only asks for about 5%. It is the leak sum again, in the counters: the model learns exactly the precision its training lengths need. Building exactness into training is not yet solved here. Switching a trained model's column counters to exact (straight-through) gates and continuing training knocked two of three seeds off their solution (reach 0); training from scratch with one exact and one soft counter per head, the model did its column counting with the soft counters (12 column counters, 0 of them exact) and extended worse (.39 at 40 digits).

Where this leaves things

• Softmax attention is a competition decided by Σ e^−gap. Generalizing over length means keeping that sum bounded as sequences grow, and nothing in short training asks for it.
• Every repair that worked made a length-dependent quantity length-independent by construction (a sink with no position, distances beyond the trained range that cannot matter, exact counts). Every repair that changed only how scores are normalized, or added data, moved the failure elsewhere.
• Attention can learn its own coordinates, but they must be exact to work at length, and gradient descent learns only the precision the data demands. The next step is a schedule that anneals every counter from soft to exact, so there is no soft path left to fall back on.

Limits

• One task, one model size; three seeds per variant except the exact-counting runs (one seed for the from-scratch run).
• The dilution analysis is on 3-digit tokens with a task-structured encoding (rows and columns); the learned-coordinate model uses none.
• Sparse attention was tested with entmax-1.5 only.

References

1. A. F. T. Martins, R. F. Astudillo, From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classification, ICML 2016 (arXiv 1602.02068).
2. B. Peters, V. Niculae, A. F. T. Martins, Sparse Sequence-to-Sequence Models, ACL 2019 (arXiv 1905.05702).
3. O. Golovneva et al., Contextual Position Encoding: Learning to Count What's Important, arXiv 2405.18719 (2024).
4. K. M. Nakanishi, Scalable-Softmax Is Superior for Attention, arXiv 2501.19399 (2025).
5. B. Peng et al., YaRN: Efficient Context Window Extension of Large Language Models, arXiv 2309.00071 (2023).
6. G. Xiao et al., Efficient Streaming Language Models with Attention Sinks, ICLR 2024 (arXiv 2309.17453).
7. S. Li et al., Functional Interpolation for Relative Positions Improves Long Context Transformers (FIRE), ICLR 2024 (arXiv 2310.04418).
8. A. Graves, G. Wayne, I. Danihelka, Neural Turing Machines, arXiv 1410.5401 (2014).
9. Ł. Kaiser, I. Sutskever, Neural GPUs Learn Algorithms, ICLR 2016 (arXiv 1511.08228).