A Vector-Symbolic Clojure

Every value is one high-dimensional vector. September 2026.

This notebook runs a small Clojure in which every value is a single vector of 2048 numbers. That covers lists, maps, sets, integers, closures, and the environment the evaluator runs in. The evaluator never looks inside a vector. It only binds, unbinds, adds and compares vectors, and cleans up noisy ones against a memory.

It is the idea of Tomkins-Flanagan and Kelly’s “Hey Pentti, We Did It!” (2024), which builds Lisp 1.5 this way, carried over to Clojure and then run until it broke. Every result below is computed live when the page renders.

Why care. Brains, and the neuromorphic chips modelled on them, do not store bits in addresses. They store patterns spread over many noisy units. Vector-symbolic architectures are a mathematical model of how symbols and structure could live in such patterns. A language whose values are all such vectors asks the question concretely: can you program on that kind of substrate, and what goes wrong when you try? It also puts data and code in one format, and it allows a new way of programming, on superpositions of values.

Three findings.

  • It works, but only after fixing five failure modes that the algebra hides (below).
  • Stored data survives heavy damage. Computation is far more fragile.
  • A value can be a weighted mix of several values, and the program computes on all of them at once. Getting this to work needed a new kind of memory.

How the interpreter is built, step by step, with all the code: Building a Vector-Symbolic Interpreter.

Code, tests, experiments and the paper draft: github.com/benjamin-asdf/vector-symbolic-clj.

Hyperdimensional computing in four operations

A symbol is a random vector of D = 2048 numbers. Two random vectors in that many dimensions are almost unrelated: their similarity (the cosine of the angle between them, 1 for equal, 0 for unrelated) is about ±0.02. So there is room for far more symbols than dimensions. Four operations build everything else.

  • Binding a⊗b glues two vectors together, like a name tag onto a value. The result looks like neither, but given one you can get the other back. (Technically: circular convolution.)
  • Unbinding a⊘c peels the tag a off c = a⊗b and gives back b.
  • Bundling a + b throws vectors into one bag. The bag is similar to each thing in it. That is a set.
  • Cleanup snaps a noisy vector to the nearest vector the memory knows, like autocorrect snapping a typo to a real word.

These are holographic reduced representations (Plate, 1995).

Every value is one vector

Start a machine, then read a map into vector space. What comes back is a numpy array, and that array is the entire map.

(vsc/init!)
:ready
(kind/code (str (vsc/eval-form '{:name "Ada" :lang :clojure})))
[-0.01399209 -0.01042913 -0.0025942  ...  0.02689232  0.01043207
  0.04800946]

How a list becomes one vector: a list cell says “first is a, rest is b”. Bind a to a fixed role label L and b to a role label R, and bundle the two: L⊗a + R⊗b. To read the first element, unbind L and clean up. Every such cell also gets its own random ID (a pointer), for a reason explained under “What breaks”. The rest of the encoding follows the same pattern. In the table, ν means “rescale to length 1”, and ⟨v⟩ is v behind its own pointer.

value vector
symbol, keyword, string a random atom, stored in the memory M with its kind
(a . b), [a b] a pointer p, with M: p ↦ ν(L⊗a + R⊗b)
first, rest unbind L or R from what p points to, then clean up
{k v …} p ↦ ν(Σ k⊗⟨v⟩) plus the set of keys and the size
#{x …} p ↦ ν(Σ x) plus the size
integer n Bⁿ⊗Z, so + is binding
closure p ↦ ν(L⊗form + R⊗env)

L, R, B and Z are fixed random vectors, and ν normalises to length 1. A map is a record: get is one unbind and one cleanup. Map equality ignores key order for free, because addition commutes.

(examples vsc/run
          '[(cons 'a '(b c))
            (:name {:name "Ada" :lang :clojure})
            (= {:a 1 :b 2} {:b 2 :a 1})
            (contains? #{:x :y :z} :y)
            (assoc {:a 1} :b [1 2])
            (+ 40 2)
            (eval (list '+ 1 2))])
form result ms 
(cons (quote a) (quote (b c)))
 
(a b c)
 15 
(:name {:name "Ada", :lang :clojure})
 
"Ada"
 10 
(= {:a 1, :b 2} {:b 2, :a 1})
 
true
 18 
(contains? #{:y :z :x} :y)
 
true
 10 
(assoc {:a 1} :b [1 2])
 
{:b [1 2], :a 1}
 23 
(+ 40 2)
 
42
 6 
(eval (list (quote +) 1 2))
 
3
 19

Programs work too. The prelude’s map, filter and reduce are themselves written in this dialect.

(vsc/run '(defn fact [n] (if (zero? n) 1 (* n (fact (dec n))))))
fact
(examples vsc/run
          '[(fact 5)
            (map (fn [x] (* x x)) (range 6))
            (reduce + 0 [1 2 3 4])
            (let [add (fn [a] (fn [b] (+ a b)))] ((add 3) 4))])
form result ms 
(fact 5)
 
120
 1358 
(map (fn [x] (* x x)) (range 6))
 
(0 1 4 9 16 25)
 1280 
(reduce + 0 [1 2 3 4])
 
10
 250 
(let [add (fn [a] (fn [b] (+ a b)))] ((add 3) 4))
 
7
 38

A small program takes seconds, where native Clojure takes microseconds. Speed is not the point.

Noise

Add noise as large as the signal to a list, and its first element still comes back, because every read ends in a cleanup.

(examples vsc/run
          '[(first (degrade (quote (a b c)) 100))
            (get (degrade {:a 1 :b 2} 60) :b)])
form result ms 
(first (degrade (quote (a b c)) 100))
 
a
 12 
(get (degrade {:a 1, :b 2} 60) :b)
 
2
 16

What breaks

Running real programs turned up five failure modes that the algebra hides.

1. Similar lists get confused. In the original paper, a list cell is just the bundle of its parts. Recursion creates many nearly equal cells, such as ((a . 5) . ()) and ((a . 4) . ()), and as bundles they are 89% similar. Reading one of them beats its look-alike by a margin of only 0.05, so a little noise returns the wrong cell. The fix: every cell gets its own random ID, and cells with equal contents share one ID, so checking equality is still one comparison. The right cell now wins by 0.66, and reading survives 2.5× more noise. (Noise σ here means noise of σ times the vector’s own size.)

Left: the margin of a rest cleanup over the best sibling. Right: accuracy under probe noise.

2. Binding commutes. In {1 2, 2 3} the entry 1 ↦ 2 contributes 1⊗2 = 2⊗1, so (get {1 2 2 3} 2) returned 1. Each value now gets its own pointer, which no key can equal.

3. Numbers can blend into fake numbers. We store an integer by its remainders, like reading six clocks at once: 11 is “1 on a 5-hour clock, 4 on a 7-hour clock, 0 on an 11-hour clock”, and so on up to a 19-hour clock. Each clock lives in its own share of the vector. That makes numbers cheap to recognise: read each clock, then combine (Kymn et al., 2025; Hanley et al., 2025).

Now put 11 and 12 into one set. The set shows both numbers’ readings on every clock. A number that shows 11’s reading on some clocks and 12’s on the others, 13102 for example, matches the set exactly as well as 11 or 12 do. It is a chimera: a fake member stitched together from pieces of real ones. With six clocks there are 2⁶ − 2 = 62 of them.

Left: all 62 fakes score exactly like the real members 11 and 12; random numbers score near 0. Right: each clock of the fake 13102 matches either 11 or 12.

The fix: inside sets and maps, each number is stored behind its own random ID, and IDs cannot be stitched together.

4. Zero lived in a single dimension. The number 0 was a vector with all its weight in one of the 2048 dimensions. When that one dimension was damaged, all arithmetic broke. Zero is now spread over all dimensions like every other value. The clocks cause a second trap: 5005 = 5·7·11·13 reads 0 on four of the six clocks, so it is 4/6 similar to zero, and a test “similar enough to zero” would accept it. zero? now asks for equality:

(examples vsc/run '[(zero? 5005) (zero? 0)])
form result ms 
(zero? 5005)
 
false
 5 
(zero? 0)
 
true
 4

5. Fixed thresholds throw robustness away. Bigger vectors should tolerate more noise, but a fixed “similar enough” cutoff does not grow with them. The first interpreter recognised a value’s kind when its cosine exceeded 0.5. A clean vector’s cosine to its own noisy probe is 1/√(1 + σ²), which crosses 0.5 at σ = √3 for every dimension. So every structure failed at σ ≈ 1.7, while the cleanup underneath survives σ ≈ 14. Deciding by argmax, or relative to the chance level 1/√D, makes robustness grow with √D again. Fixed thresholds are common practice: Nengo’s associative memory defaults to 0.3 in every dimension.

How robust it is

Here D is the number of dimensions and σ the noise, relative to the signal. A single cleanup matches its textbook model within 0.007 on average, and the noise it tolerates grows as √D: four times the dimensions, twice the noise.

How often cleanup finds the right item, against noise, for D from 512 to 8192 and memories of 100 to 100,000 items. Dashed: theory.

Capacity is linear in D: about D/64 map entries and D/36 set members at 90% reliability. Overloaded structures forget; they do not hallucinate members.

Lookups against the number of entries.

Storage survives the loss of 60–80% of dimensions. Computation survives about 4%, because every bind re-loses the dead dimensions until the next cleanup.

Lesions: stored structures against programs.

Hopfield networks do not help

The paper suggests a modern Hopfield network as the cleanup memory. Instead of returning the one best match, it returns a weighted mix of the stored items, and can repeat that step. A parameter β sets how hard it favours the best match: at large β the mix is almost only the winner.

It never beats a plain lookup table. At large β it is the table, only slower. At smaller β, and whenever it repeats the step, it is worse. The reason: when a probe is one stored item plus noise, picking the single best match is already the best possible decision.

The noise each memory tolerates (higher is better): the lookup table against Hopfield settings and a linear memory.

The interpreter as vectors

The evaluator above is host code. The next step moves it into vectors. A classic design for interpreters, the CEK machine, keeps three things: C, what to evaluate now; E, which variable has which value; and K, a to-do list of what comes after. Here all three, plus a mode, are bundled into one vector: s ↦ ν(C⊗control + E⊗env + K⊗to-do + M⊗mode). Its 35 transition rules sit in a memory and are picked by similarity. Rule keys are sums of features, so the most specific matching rule wins by nearest neighbour, and shorter keys act as defaults. The host loop is 47 lines and knows no Lisp. Tail calls come for free.

One step of the machine goes round this loop. Everything in the blue boxes is a vector in memory; only the loop itself is host code.

state s: one vectorC what to evaluate nowE variables → valuesK to-do list (9 kinds of frame)key(s): the state's shapefetch 12 instructions read sΣ mode + kind + head+ identity + next framerule memory R35 rules, each with a keynearest key wins⇒ the most specific rulethe rule's microprogramlist of instruction vectors9 ops FIELD CLEAN BIND UNBIND REC MOV JMPEQ PRIM DEFread its shapenearest keyrun itnew statehost loop · 47 lines · no Lisp

Why the most specific rule wins. A state’s key always has all five features. A rule’s key names only the features it cares about: a default rule names one, the rule for if names three. When all of a rule’s features match, its similarity to the state’s key is √(k/5) for k features. So when several rules match, the one that names the most features is nearest, and a rule that names fewer acts as a default.

(kind/table
 {:column-names ["features the rule names" "example" "similarity to the state's key"]
  :row-vectors (for [[k ex] [[1 "any value evaluates to itself"]
                             [2 "a list is a function call"]
                             [3 "a list headed by if is an if"]]]
                 [k ex (format "%.3f" (Math/sqrt (/ k 5.0)))])})
features the rule names example similarity to the state's key
1 any value evaluates to itself 0.447
2 a list is a function call 0.632
3 a list headed by if is an if 0.775

A run, step by step. Each row is decoded from the state vector: the mode, what is being evaluated, the rule that nearest-neighbour lookup chose, and the to-do list. At step 14 the if frame is already gone: the branch runs as a tail call, so the to-do list does not grow.

step mode evaluating rule chosen to-do list
0 EVAL ((fn [x] (if (zero? x) :zero (inc x))) 1) call halt
1 EVAL (fn [x] (if (zero? x) :zero (inc x))) fn args → halt
2 RET #fn (fn [x] (if (zero? x) :zero (inc x))) arg args → halt
3 EVAL 1 self-evaluating args → halt
4 RET 1 arg args → halt
5 APPLY #fn (fn [x] (if (zero? x) :zero (inc x))) closure halt
6 EVAL (if (zero? x) :zero (inc x)) if halt
7 EVAL (zero? x) call if → halt
8 EVAL zero? symbol args → if → halt
9 RET #prim zero? arg args → if → halt
10 EVAL x symbol args → if → halt
11 RET 1 arg args → if → halt
12 APPLY #prim zero? primitive if → halt
13 RET false if-branch if → halt
14 EVAL (inc x) call halt
15 EVAL inc symbol args → halt
16 RET #prim inc arg args → halt
17 EVAL x symbol args → halt
18 RET 1 arg args → halt
19 APPLY #prim inc primitive halt
20 RET 2 halt halt

Result: 2

Superposition programming

A weighted sum of values is also a vector. So a variable can hold several worlds at once, and the program computes on all of them. Destructors act on every world in one memory read, and integer arithmetic lifts exactly, because binding distributes over addition.

(w/init! {:dim 4096})
:ready
(examples worlds
          '[(+ (amb 1 2) 10)
            (first (amb (quote (a b)) (quote (c d))))
            (let [x (amb 1 2)] (+ x x))
            (for-worlds [x (amb 1 2)] (+ x x))])
form result ms 
(+ (amb 1 2) 10)
 
{11 0.5, 12 0.5}
 25 
(first (amb (quote (a b)) (quote (c d))))
 
{a 0.4999, c 0.5001}
 32 
(let [x (amb 1 2)] (+ x x))
 
{2 0.25, 3 0.5, 4 0.25}
 48 
(for-worlds [x (amb 1 2)] (+ x x))
 
{2 0.5, 4 0.5}
 66

Look at the last two rows. A vector carries no labels saying which world a value belongs to, so the two uses of x are independent draws. That is run-time choice. The name comes from nondeterministic programming, where Hussmann (1993) distinguished it from call-time choice, in which a variable picks one value and keeps it. Languages like Curry use call-time choice. Here it is explicit, with for-worlds.

This needed a new kind of memory. A lookup table and a softmax (Hopfield) memory both collapse a sum of worlds to one world. A linear memory keeps the worlds but lets every other stored item through as well. What works is a memory that keeps only the stored items that clearly stand out above chance and solves least squares on them. It holds about D/100 worlds.

if on a superposed test runs both branches and weights them by the test. Each world goes down its own branch, so recursion ends at a different depth in each world:

(w/run '(defn dbl [n] (if (zero? n) 0 (+ 2 (dbl (dec n))))))
dbl
(examples worlds '[(dbl (superpose {2 5 3 3 5 2}))])
form result ms 
(dbl (superpose {2 5, 3 3, 5 2}))
 
{4 0.5, 6 0.3, 10 0.2}
 639

The weights behave like probabilities, so a textbook Bayesian network works. Rain makes the sprinkler less likely, and either makes the grass wet. What is P(rain | wet)? The exact answer is 0.36255. Probability with such vectors is not new: Furlong and Eliasmith (2024) and the “hyperdimensional transform” (Dewulf et al., 2023) do inference this way. What is new here is that it runs inside an ordinary programming language.

(w/run-string (slurp "examples/worlds/sprinkler.clj"))
#worlds {false 0.6378, true 0.3622}
(examples worlds '[rain-given-wet rain-given-wet-naive])
form result ms 
rain-given-wet
 
{false 0.6378, true 0.3622}
 5 
rain-given-wet-naive
 
{false 0.7999, true 0.2001}
 4

The naive version uses let instead of for-worlds, so run-time choice loses the link between rain and wet grass, and it returns the prior, 0.2. This query still enumerates the worlds one by one, so it is no faster than ordinary code.

Worlds read back exactly, against the number of worlds and D.

There is no speedup: a superposed step still scans the memory. What superposition buys is a programming model, within a capacity of about D/100 worlds, and only about 5 integer worlds.

Limits

  • Seconds per small program. Every cleanup scans a memory that only grows.
  • No macros, loop/recur, floats or strings.
  • Records hold about D/64 entries; superpositions about D/100 worlds.

References

  • Tomkins-Flanagan, E., & Kelly, M. A. (2024). Hey Pentti, we did it!: A fully vector-symbolic Lisp. MathPsych/ICCM 2024. arXiv:2510.17889.
  • Hanley, C., Tomkins-Flanagan, E., & Kelly, M. A. (2025). Hey Pentti, we did (more of) it!: A vector-symbolic Lisp with residue arithmetic. IJCNN 2025. arXiv:2511.08767.
  • Kymn, C. J., et al. (2025). Computing with residue numbers in high-dimensional representation. Neural Computation, 37(1).
  • Meier, C. (2023). Vector symbolic architectures in Clojure. Clojure/Conj 2023. https://github.com/gigasquid/vsa-clj
  • Dewulf, P., De Baets, B., & Stock, M. (2023). The hyperdimensional transform for distributional modelling, regression and classification. arXiv:2311.08150.
  • Furlong, P. M., & Eliasmith, C. (2024). Modelling neural probabilistic computation using vector symbolic architectures. Cognitive Neurodynamics, 18(6).
  • Hussmann, H. (1993). Nondeterminism in Algebraic Specifications and Algebraic Programs. Birkhäuser.
  • Plate, T. A. (1995). Holographic reduced representations. IEEE Transactions on Neural Networks, 6(3), 623–641.
  • Ramsauer, H., et al. (2021). Hopfield networks is all you need. ICLR 
source: notebooks/vector_symbolic_clojure.clj