HDC Overview

• High-Dimensional Computing With Sparse Vectors Using Clojure: the big one. Binary sparse block codes in Clojure, a sparse distributed memory (SDM) with a multi-value mode, a k-fold sequence memory, and my argument that Kanerva's address decoder is a reader-centric cell assembly seen from the reader: the decoder is the reader, the input patterns within its radius are its assemblies.
• Machine Learning Diary: classifying Iris with a sparse distributed memory, in exploration mode.
• Neuronal Ensemble Memetics: the very large pile of notes on cell assemblies where HDC first showed up, as the more principled take on some of the same ideas.
• Neuronal Memetics Overview: the short version. Ensembles as active data structures, a hyperspace as concept space.
• Current Stuff | November 2024: a snapshot, with a section on HDC, SDM and associative models.

A small Clojure in which every value is one vector of 2048 numbers: lists, maps, sets, integers, closures, and the environment the evaluator runs in. It carries Tomkins-Flanagan and Kelly's Hey Pentti, We Did It!, a Lisp 1.5 built from holographic reduced representations, over to Clojure,³ and then runs it until it breaks. The overview and the build-up are notebooks whose numbers the code computes live. Code.

• A Vector-Symbolic Clojure: the overview. The design, how robust it is, superposition programming, and the five traps I fell into.
• Building a Vector-Symbolic Interpreter: from random vectors to (fact 5), step by step.
• How the Vector-Symbolic Clojure Got Here: what broke, roughly in the order I found it, and what changed.
• Reproducing the Traps in the Original Code: the five traps, run against the original authors' own code, which shows which of them apply there, with errata for the authors.⁴

HDC-MiniROCKET is the work of Kenny Schlegel, Peer Neubert and Peter Protzel, all at TU Chemnitz at the time. Schlegel is still there; Neubert is now at the University of Koblenz.⁵ MiniROCKET turns a time series into about 10,000 features that ignore when something happened. HDC-MiniROCKET binds the features at each time step to a vector that encodes that time, and one scale parameter s sets how fine the time resolution is (s = 0 is plain MiniROCKET).⁶ A later paper by Schlegel, Rachkovskij, Kleyko and coauthors gives three real-valued versions of the position encoding; one of them, Sinusoid, has an exact shift operator.⁷ The ideas are theirs. This series reproduces their work and pulls on threads they left open. Four parts, with negative results stated as negative:

1. Reproducing HDC-MiniROCKET: a clean-room reimplementation, run on all 128 UCR datasets. The authors' published per-dataset accuracies reproduce exactly from their code as it was before a 2022 bug fix. The three real-valued encodings are not significantly different. The shift operator moves positions; it does not predict the vector of a shifted signal.⁸
2. Choosing the Time Scale: how to pick s without peeking at the test set. A development run on 20 datasets with three seeds, in my encoding and also in the authors' own encoding and pipeline: no method I tried beats the automatic choice from their 2025 follow-up paper, and that choice does not significantly beat leaving the time encoding out either. More than half of the oracle's gain is selection on test-set noise. The 20 datasets lean towards time encoding, so these are development results, not conclusions.⁹
3. Streaming HDC-MiniROCKET: keep the sequence vector up to date in O(D) per sample with the shift operator. It equals the batch vector of the prefix once the edge is flushed, and at windows of 256 to 4096 samples a push is tens to hundreds of times cheaper than re-encoding, if you predict on every sample. Negative: on prefixes of a series, HDC does no better than plain MiniROCKET features, and a classifier trained on whole series cannot read prefixes.¹⁰
4. Finding the Shift: align two series by sweeping the shift between their vectors, the whole similarity curve in one transform. Negative: for locating a planted copy in a stream it finds the copy less often than the plain MASS algorithm, even with far more memory than the raw stream, and with no more memory than the stream it finds it in 10 % of cases against 93 % for MASS. Retrieving series of the same class comes close to MASS on average, but not on every dataset.¹¹

All of these are public on GitHub.

• vector-symbolic-clj: the code behind the Vector-Symbolic Clojure series.
• vsa-binary-sparse-distributed-segments-clj: binary sparse segmented vectors after Laiho et al.,¹² the SDM, and small examples of programming in superposition: What is the dollar in Mexico?, nondeterministic automata, trees, analogies.
• hdc-anomaly-skab: anomaly detection on the SKAB sensor benchmark. F1 0.71, against 0.78 for the best leaderboard methods, with one-pass training and a 1.2 KB model. My threshold is tuned on the test split, while the leaderboard notebooks set theirs without the test labels, so the 0.71 is optimistic.¹³
• clj-hdc-push: a Clojure HDC exploration library on PyTorch: MAP, HRR, FHRR and block codes, resonator networks. Meant as the HDC layer for Push-style genetic programming.
• hdc-wolfram: the beginnings of an HD/VSA framework for Wolfram, driven from Clojure via Wolframite, with notebooks.
• hyper-substrates: visualizing ensemble-like neuronal nets (blerps), on the way to putting a biological substrate together with HDC and associative memory.

• Time scale on held-out data. Freeze the methods from the development run, then run the 108 held-out UCR datasets with three seeds. Only then does any claim about s count.
• Streaming on a real stream. Human activity data instead of synthetic streams, and a real early-classification policy that stops when it is confident, with its earliness/accuracy trade-off.
• Shift-invariant similarity. The maximum over shifts as a similarity between series, for 1-nearest-neighbour classification, against Euclidean distance and DTW.
• Better location. Fix the tail of the shift estimator, which is right at the median and grossly wrong too often.

• Is a neuronal ensemble a hypervector, or does it only look like one from far away?
• The shift search says the vectors carry what well and where poorly. Is that a limit of the dimension, or of smooth signals that put their energy at a few frequencies?
• Is a fixed s = 7 really that good, or is that the small development set talking?
• Does early classification need one classifier per prefix length, or is there an encoding in which a prefix is just a noisy whole?
• Where does programming in superposition stop being cheaper than plain search?
• What plays the role of the cleanup memory in a brain?