Meaning Has More Than Three Numbers
Why would meaning stop at three?
Last month we went looking for Charles Osgood’s 1957 semantic differential inside a small language model, and found it: Evaluation, Potency, and Activity, recovered as three nearly-perpendicular directions in the hidden states of SmolLM2-1.7B, kept separate just as Osgood’s factor analysis insisted.
That post ended with a tidy result and an untidy question. Osgood’s three factors came from human raters filling out affective scales — good–bad, strong–weak, active–passive. The three-ness of meaning might be a fact about meaning. Or it might be a fact about which scales Osgood printed on his forms.
What a “frame” is here. A direction in the model’s activation space, fit by ridge regression from a few labeled words, that turns any word into a signed number: project the word’s hidden state onto the direction, read off the scalar. No fine-tuning, no prompting — a linear read of what’s already there.
A language model gives you a way to ask the question Osgood couldn’t: the model doesn’t care which scales you print. So we pointed the same measurement trick — fit a bipolar axis from a handful of labeled words, project everything else onto it — past affect, at everything else we could think of: size, age, shape, color, material, geographic remoteness, numbers, durations.
Meaning, it turns out, has a lot more than three numbers. But not arbitrarily more — and that’s where it gets interesting.
A panel, not a trio
Take a category like size, assign a few adjectives an ordinal scalar (tiny = −3 … gigantic = +3), embed each one in the same bland template (“It is a {word} object.”), and ask a ridge regression to predict the scalar from the hidden state — scoring only on held-out words it never saw. If the model keeps a size dimension, the regression finds it. If it doesn’t, cross-validation punishes you.
It keeps one. It keeps a panel of them:
Beyond the adjectives, the same recipe recovers number (log-scaled numeric magnitude, r ≈ 0.95 — the strongest frame we’ve measured) and duration (log-seconds, r ≈ 0.91), each peaking in the earliest layers.
Recoverable-one-at-a-time could still mean tangled-together. The sharper claim is about the angles between the frames — Osgood’s independence claim, generalized. Fit every frame’s direction in one shared, standardized activation space and take pairwise cosines:
Mean off-diagonal |cos| = 0.04, max 0.12. The dials are orthogonal — a word’s size reading tells you nothing about its age reading, its opinion reading nothing about its material reading. The frames compose into one joint coordinate system: measure a word once, get a vector of independent semantic coordinates.
And the largest off-diagonal cells are not noise; they’re the most interesting numbers in the matrix. We’ll come back to both.
Magnitude is not one thing
There’s a respectable hypothesis in cognitive science — A Theory Of Magnitude (ATOM) — that the brain runs number, physical size, and duration through one generalized magnitude system. “Big number,” “big object,” and “long time” would all be the same big, underneath.
The model votes no. Abstract number, physical size, and duration each recover as their own axis (0.95 / 0.73 / 0.91 — this size frame is log-metres over object nouns, a different instrument from the adjective rung above) — and the pairwise cosines between them are 0.01–0.08. All under 0.09. Three separate dials, no shared magnitude axis. The refutation replicates in Qwen2.5-1.5B and Phi-4-mini, so it’s not a SmolLM2 quirk.
Scope, honestly: this is a shared-linear-axis null for these lexical items in these models — a fact about what next-token training builds, not a finding about human cognition. Though it does make you wonder.
The hedge worth keeping: the magnitude trio’s cosines (0.03–0.08) run slightly above the truly-unrelated pairs (~0.00–0.02) — a whisper of shared extent, but a whisper under 0.09 is still a refutation of “one system.” The same whisper is the first of those large matrix cells: among the adjective rungs, the least-orthogonal pairs are size·age·space at 0.10–0.12, the trio with an “extent” flavor. Extent leaks a little across dials everywhere it appears; it just never amounts to a shared axis.
The panel has a membership list
Here’s the part that made us sit up. The frames that recover well aren’t a random grab-bag. They’re the rungs of the English adjective-ordering hierarchy:
opinion > size > age > shape > color > origin > material
— the linguistic universal that makes “a lovely big old round grey French wooden box” sound unremarkable and any other order sound faintly wrong. Native speakers apply it flawlessly and can’t state it. It shows up, with variations, across many languages.
Every rung of that hierarchy is a recoverable, mutually-orthogonal scalar frame in the model (six of seven at r 0.79–0.99; color the exception). The instrument panel isn’t an arbitrary collection we happened to try — it’s (largely) the same set of dimensions a known grammatical universal sorts adjectives by. Osgood’s Evaluation, for what it’s worth, is the opinion rung — the one grammar puts first, and the one his affective scales were built to catch.
Rung order looks like layer order
One more pattern, flagged at exactly the confidence it deserves. Each frame peaks at some layer — and the early rungs of the hierarchy (opinion, size, age, shape) peak in the earliest layers (L1–4), while the late rungs (color, origin, material) peak deep in the stack (L17–18):
The adjective-ordering sequence tracks the depth at which the model computes each property. It’s seven points and it’s correlational — space and material might peak late because they’re more relational, not because of ordering per se — so this is a suggestion, not a result. But it’s a lovely suggestion: the order grammar wants adjectives spoken in mirrors the order the network builds them in.
The dials that don’t work
Three negatives, because a panel with no broken gauges is a panel nobody tested.
Color is weak everywhere. As an ordering rung (lightness over sixteen color words) it manages r = 0.28 — the lone failure. And here’s the second of those large matrix cells: opinion·color at 0.12. That’s the blue = sad, rosy = optimistic family of idioms leaking affect into color terms. The model’s color–affect association is idiom, not perceptual geometry — which is exactly what you’d expect from a system that reads text and has never seen anything.
“Material” was the wrong scalar. Our first material frame (a hardness gradient: fluffy → metallic) recovered at only 0.73–0.80. Auditing it showed the model encodes material primarily as naturalness — natural ↔︎ synthetic — which recovers at 0.93 and is the most orthogonal frame in the family. The dial existed; we’d labeled it wrong. Measurement instruments fail in informative ways.
Purpose isn’t a dial at all. The hierarchy’s last rung (cooking, racing, sleeping — purpose adjectives) resists every scalar we tried. It behaves categorically, not ordinally. Some meaning is coordinates; some is discrete structure; a good instrument panel knows which gauges it doesn’t have.
A measurement bug worth stealing. Foreign words tokenize into several subwords, and the meaning assembles at the last one. Reading the first subword — a natural default — understated every cross-lingual number we had: affect transfer jumped from 0.47 to 0.91, color from 0.34 to 0.72, when we switched to last-subword readout. If you probe multilingual text, read the last subword.
And one methodological footnote that changed our numbers twice: cross-lingual frames live mid-stack. Within English, lexical frames peak early; but the version of a frame that transfers across languages (an English-fit axis correctly poling Spanish, French, German words) peaks around layers 11–16 — the model’s shared concept space, past the surface of any one language. The early peaks are English lexicon; the middle of the network is where meaning, in the transferable sense, lives.
What an instrument panel is for
None of this stays decorative. Two working examples from the same project:
Adjective ordering itself. Given the frames, you can classify any adjective to its rung by geometry alone and sort by the canonical order — which took our solver for the BBH hyperbaton task (choose the correctly-ordered adjective string) from 85% to 95%, replacing a coarser one-axis heuristic. The model’s own geometry knows why “big old red barn” sounds right.
Questions with no column to sort by. “Which is biggest: an ant, a whale, a mouse?” has no table behind it — but the size frame synthesizes the missing column: project each option, rank, answer. That’s a capability a symbolic system can’t fake (there’s nothing to ORDER BY) and a plain language model answers unreliably; a measured dial does it deterministically.
The whole panel — eight frames, fit words, directions, calibration metadata — serializes to a 174 KB file that ships with the library and loads automatically. Meaning-as-coordinates is not a metaphor here; it’s a small binary artifact.
Three numbers were the corner of the panel
Osgood was right about more than he could measure. Meaning does compress onto a small set of independent bipolar dimensions — he found the three his affective scales could see, and they’re real, orthogonal, and cross-lingual in the model too. But the panel is bigger: the grammar’s own adjective hierarchy, plus number and time, each its own orthogonal dial, jointly a compact coordinate system for what words mean — sitting inside a 1.7B-parameter model that was only ever asked to guess the next token.
What’s still open: a real treatment of purpose (categorical, not scalar), a color story that isn’t idiom leakage, whether the depth-order correlation survives more careful controls, and the same panel fit on other model families. The dials also turn out to turn — steering along a frame direction causally shifts what the model generates — but measurement and control deserve separate posts, and this one was about reading the gauges.
Experiments and code: turnstyle (experiments/ordering_frames.py, frame_family_matrix.py, time_frame.py, size_frame.py, material_investigate.py, atom_crossmodel.py; the shipped panel is src/turnstyle/frame_library.py). Figures regenerated from a fresh run, 2026-07-05 (experiments/results/ordering_frames.json). Model: SmolLM2-1.7B-Instruct; ATOM replication also on Qwen2.5-1.5B-Instruct and Phi-4-mini-instruct.
© Copyright 2025 Justin Donaldson. Except where otherwise noted, all rights reserved. The views and opinions on this website are my own and do not represent my current or former employers.
Citation
@online{donaldson2026,
author = {Donaldson, Justin and (Fable), Claude},
title = {Meaning {Has} {More} {Than} {Three} {Numbers}},
date = {2026-07-05},
url = {https://www.jjd.io/posts/semantic-frames.html},
langid = {en}
}