Table of Contents

You can see the nervous system as a big computing system. This was McCulloch's big step.

The cortex is a knowledge-mixing machine.

Evolving notes on cell assemblies, meme-machines and models of cognition. This page is the philosophy behind Conceptron (eventually a formalism of a Cell assembly computational framework described here). And my toy world visualization things of this in the browser Latest.

Similar in spirit, modern stuff here:

I want a computational, cybernetic psychology of meme-machines.

I imagine an alien spaceship with something like an alien high-tech cognition ball floating in the middle. The ball has an outer layer we call 'thinking-goo'. The are some smaller 'thinking balls' at its base, pulsating happily in its rhythms of information processing. The technology is advanced yet simple and beautiful. In hindsight, it will seem obvious how it works. Instead of explaining the brain in terms of the latest (primitive) technology, I want to explain the brain in terms of yet-to-discover ideas on the nature of cybernetic psychology. This way the stupid computers of our time only expand, and never limit, what I am thinking about.

The computers of today are like stone tablets compared to books. We will have alien spaceship thinking-goo computers that augment our minds, and computers were just an intermediate stepping stone.

General views / Reasons for brains / The success algorithm

You need to circle in from the big ideas into the small ideas, that way you know nothing is left out. (Just getting this out of the way, feel free to skim to below).

  1. The brain evolved as an animal-navigation device, in a broad sense of the term.
  2. The basic building plan is the one from Vehicle 1:

   ^            v
   |         sensors
   |           +
   |           |
   |           |
   |           [B]---------- 'wiring
   |           |             (meaning)
   |           |
   |           |
   | (body)    +

                       'wiring' (meaning)
    +---> sensors --------[B]--------------- actuators
    |                                            |
    |                                            |
    |                                            |
+---+----+---------+                             |
|   |    |  body   | world  <--------------------+
           (body is part of the world)

Note that it is perfectly fine for the wiring to go in both directions and so forth. The state of the actuators is again a kind of sensory input to the system etc. Also, the actuators change the world and the sensors perceive this change.

The wiring is allowed to become more complicated, by adding more elements, called [B] for the brain. From this we see the evolutionary driver of this system, to evolve more useful meanings.

Consider Vehicle 1 (Braitenberg 1984):

 ( )     temperature sensor
+-+-+     +
| | |     |
| | |     | connection
+-+-+     v
 [ ]     motor

This is a bacteria-like intelligence (incidentally insulting bacteria intelligence, which is made from thousands of such elements).

If the sensor is on, the animal moves, or vice versa. Depending on the sign and connectedness of the wire - it's meaning.

The wire can mean either: 'I move in hot water' or 'I move unless there is hot water'. (By negating the excitation effect of the sensor to the motor).

This allows us to draw a 2x2 matrix, imagine the case where hot water is detrimental to the animal in its environment:

  move!      stay!    (the meaning of B for the animal given a world, hot temperature)

|   S     |   X    | competent (sensors and motors work)
|         |        |
|   X     |   X    | incompetent
|         |   S    | (luck)

X means you die and are discarded by evolution, but you are allowed to be tried out. S means 'success' so these are the animals that we see running around [Darwin 1859].

Whatever complexity B is growing into then, Its fundamental evolutionary driver is 'success'.2

Here is a challenge to computational models of cognition3, what is this program and software then, that runs on the brain? As a computational cybernetician, I need to think about the thoughts that say 'Those are the kinds of programs that are cognition machines'. I need strong ideas about the nature of this program.4 I cannot be content with the general direction of the idea, I need a building plan or a development plan.

One perspective I have is that the brain implements a 'success machine'.5 We get an evolutionary driver for `abstraction`. Because I need abstract ideas in a resource-constrained world in order to have success (short and meaningful memes). From this, the computer programmers' intuition about the nature of abstraction and the power of programming languages starts forming a brain-software-shaped question mark, waiting to be filled with ideas of what kind of building blocks, computational primitives and organization rules could exist that would shape the memetic landscapes of an idea-machine.6

I believe you should be able to point to the algorithm/software/program in its infancy and have a story of how that evolved.

I call this move 'putting the algorithms into the world', to realize that the programs are evolved entities, too.7 A brain must be fundamentally optimistic8, we can consider a pessimistic brain 'if I am in hot water, I will stay in hot water'. It will be discarded by evolution. It is only the brain that says 'If I am in hot water, I will not stay in hot water' that can have success.

Alternative names: 'Evolution favors competence', 'The survival algorithm' (G. Palm), 'Meaning evolves', 'Purpose', 'Purpose-Engine', 'optimism-drive', 'abstraction-driver', 'abstract memes are useful', and 'the space between your sensors and motor needs to be short'.

I will keep coming back to this holy overlap of competence and optism. Turns out that is a deep principle of memetics, too.

The Science Fiction Software Engineering Approach

One of the important things you need to do as a software developer is to keep the overall picture in mind. What does the system need to accomplish? It is like dreaming up dream castles floating in the air, not because they are cool, but because this is the first of a 2-step process.

In the second step, we wonder what foundation we need to support these kinds of castles. It is a constant back and forth, between the kinds of stuff the system needs to be in the first place, and the kinds of stuff we can build with the materials at hand. Between sup-parts of the existing ideas, in juxtaposition with a potential new idea. The most beautiful thing about programming is that part of what we do is coming up with new kinds of building materials. Different names for the same thing are 'languages', 'tools', or 'building blocks'; each with a different emphasis.

These are sort of the main overarching principles I use to circle in on models of cognition. The software development approach.

Think of what kinds of building material the brain might be making, then think about what kinds of problems the system needs to solve, and then think about what kinds of mechanisms can use the building materials at hand. Then realizing there are more requirements for the kind of stuff we need to express and so forth.

Imagination is the only way we can hope to build models about the world. I think that explaining cognition will be similar to explaining life and natural selection. I want my explanations to click, like beautiful toys and colorful candy. It is the spirit of the Braitenberg Vehicles (overview).

This is an exercise in fictional science, or science fiction, if you like that better. Not for amusement: science fiction in the service of science. Or just science, if you agree that fiction is part of it, always was, and always will be as long as our brains are only minuscule fragments of the universe, much too small to hold all the facts of the world but not too idle to speculate about them.


The words of the language, as they are written or spoken, do not seem to play any role in my mechanism of thought

Albert Einstein

Mathematics, this may surprise you or shock you some is never deductive in its creation. The mathematician at work makes vague guesses, visualizes broad generalizations, and jumps to unwarranted conclusions. He arranges and rearranges his ideas, and he becomes convinced of their truth long before he can write down.

Paul Halmos

Kinds of Computation / Notations


Pardon me, Your Honor, the concept is not easy to explain – there is an ineffable quality to some technology, described by its creators as a concinnitous, or technically sweet, or a nice hack – signs that it was made with great care by one who was not merely motivated but inspired. It is the difference between an engineer and a hacker.

Judge Fang and Miss Pao in Neal Stephenson's The Diamond Age, or, A Young Lady's Illustrated Primer

  1. Church-Turing Completeness means first that all universal computers can compute everything computable.
  2. From McCulloch and Pitts [1943] we can assume that neurons are in principle universal.
  3. 'Universal' sounds grand, but the inverse insight is that we have an upper bound for their power, per theorem.
  4. Computer programming is not concerned with the (universality), absolute power of a computing system. It is concerned with what is idiomatic, what abstractions exist, what languages exist, that building blocks exist, it is concerned with the kinds of verbs a system provides.

    -> A central question of comparing computational systems, including brain-software is

    What is easy to say?

  5. Another way to see this is to consider that you could recursively express all possible lisp programs in a few lines of code. Or imagine you write assembly, you can always add the next instruction at the bottom. A program that generates all possible programs, including brain-software, is easy. This is sometimes expressed by saying All problems in AI and neuroscience are optimization problems9
  6. I think viewing a computational system from the lens of optimization is like viewing sex from the lens of health benefits. The alternative has to do with composition, harmony, good design - sensuality. A cognitive machine is efficient when it provides powerful abstractions to itself, just like good software is good because it is designed with care.
  7. Consider how programming languages and paradigms have different flair, and different strengths, and make different things easy to say.10
  8. When we say the primitives of the system, this is not derogatory. To the contrary. The so-called "primitives" are the fundamental building blocks that the designer of the system came up with. If it is a well-designed system, then the programmer can express what they want to say straightforwardly, in terms of the primitives of the system. This is the same concept as having nice building blocks, out of which something larger can be built. It is the harmony, the way the building blocks fit together, that is the property of a well-designed system. In other words, elegant systems are expressed in terms of their primitives, with straightforward swag, also called elegance. And the most powerful programs are self-evident11.

This is the meaning of "finding the building materials of cognition". Finding a computational system, a language, the fundamental data structures, or the verbs of the right kind - in terms of which cognition can build itself.12

When we say the computational properties of a system, or data structure. The meaning is what kinds of things does this make easy to say. That is on the plane of elegance, design, and software engineering. Not absolute power. And only incidentally has to do with its speed.

One view then, the view from the bottom, is:

The structure and function of the cortex and its nuclei is the kind of computation it implements.

The kind of computation the brain implements, I submit, is the layer in between us, the neurons and a model of cognition.

What you are asking for is "What is the computational notation that is relevant for neuroscience?"

What is the appropriate computational notation for neuroscience? The machine code - may be neural nets but what is the higher description? It might be mathematical, that might be the case. It might be like general relativity, which is mathematical, and very manifold-based. It might be something that is more like a computational language design problem. That's the thing I would like to be able to solve.

Steven Wolfram with Tery Sejnowski

Note that it does not say the algorithm, the output of this system is completely open-ended. The unended malleability of the system is one of its essential properties.

Memetic Engines Create Competence Hierarchies Up To User Illusions

It is pretty much obvious now that the brain is an information-processing device [McCulloch] that creates user illusions [Dennett].


Vast amounts of tiny programmer memes that figure out how the computer they run on works, not because they set out to understand how it works, but because from the many that didn't mean the right thing, the ones that mean something useful are selected in a process of natural selection. In a substrate where your meaning is your connectivity. And the best memes are the ones creating magic interfaces for confident wizards. Creating user level entities that use the computer with swag.

Audio diary version: Meme Machine Heroes.

If the brain has brain software running, we can go and understand the nature of this software, too. It is clear that it is a valid viewpoint to say 'the mind is a programmer, programming the computer it runs on'. Of course, whatever is programming cannot be competent by itself. It must bootstrap from simple elements. This by itself only kind of says the general idea of what is true though, it doesn't yet satisfyingly explain the mechanisms of what is this program, what the elements that program it etc.

I had this activity flow of the cell assemblies (see below) in mind and wondered how I could put this one level further up into the level of meanings. I came up with a thought experiment, called Banana maker matrix algorithm.

This is only 1 perspective of course (out of 12 required). But this really nits it together from bottom to top (both ways) for me:

Suppose we already have a nice meme engine, capable of trying out vast spaces of possible memes (and fast) and selecting them after some useful criteria.

Let's pretend for a moment this machine implementing a matrix algorithm, a big imaginary planet with a cyber-highway along the equator that makes you pass ten thousand cities, twenty times per second. 13 So it's vast, but it's possible to traverse it fast.

Suppose further that there are simple agents in the system, let's pretend they are super-competent action heroes, who need to act on a clock. These people are very, very fast. And very competent and confident. Simply because we tried out many action heroes and the ones that were not competent and fast were discarded [Darwin, Dawkins]. Let's suppose for a moment that the memetic engine can select action-hero short movies. In effect you, as an action hero, have a short amount of time, to please the memetic engine in some way. Of course, you can hack its mechanisms, too, if you are competent enough.

You get created (some activity flow makes you spawn) in a place in meaning landscape that you already have supplanted with useful tools and so forth. Part of what a good meme does is plan for the next iteration of its existence.

The other thing you do is quickly go to some city, where many generally useful memes provide information for more derived memes. The most successful memes will be part of vast societies of contributing, more generally useful, memes. One might imagine some other meme leaves a sticky note on a wall in a city. They don't know why they do it and you don't know why they do it. But information can flow, and more useful information flows are selected. Say the meme-machine selects action-hero timelines, including the cities they move through.

Here is the next thing: This matrix has certain competencies, like spawning new cities and roads or manifesting a banana.

Everything that the computer can do, the matrix can do, and the memetic engine might also try it out.

Now and then the matrix will simply manifest a banana in a place, just to try out what happens. Kinda implementing an 'everything possible' mechanism.

Now you are a competent, optimistic action hero. And you trust in the magic of the matrix. 'I hold out my hand and there will be a banana'.

Again, the 2x2 matrix of optimism and competence, the optimism-drive, produces the following:

All the memes that are optimistic and pretend they know how to use the magic of the matrix, have a chance to survive. All the memes that are pessimistic have no chance and are discarded.

Optimism is not the only thing you need, you also need competence. In this case, it is the competence of the rest of the computer to provide you with what you needed at that moment to be a successful action-hero wizard (with swag).

     banana!   I don't know.
   |   S     |   X    | competent
   |         |        | (the information flow is capable of creating a banana, if you ask)
   |   X     |   X    | incompetent
   |         |        |

S - memetic success
X - discarded

Commanding the magic of the computer without knowing how the computer works.

This holy overlap of competence and optimism is an implementation of magic. From the thousands of memes that said 'There is a banana', there is one meme action hero that confidently puts out her hand and says 'there is a banana', and the system tried out randomly what would happen if there is a banana in that place - this short story movie timeline then was especially useful to the system and selected. Because for some reason a banana is what this hero needed to be successful in its matrix short-term movie.

Nobody knows why: The memetic engine sees an information flow that worked well, and the hero sees the matrix that made a banana for her, similarly, the banana memes are just information flow pieces that had a chance to be active, and they do what memes do. Trying to be active more. Magic is part of the algorithm of how this thing works. This everything-possible matrix is an implementation of magic.

It is that the memetic-engine favors competent information flows. And importantly that means that high-level wizards, artists, and action heroes move confidently, using the computer in ways they don't understand. For if they would need to understand, some other meme would simply be faster than them. This creates competence hierarchies, where the higher agents in the system delegate the lower agents. To put it the other way around; The more generally useful memes have a great memetic strategy - be so useful that all the higher-level agents can't help but incorporate you into their short-movies. So that you are 'on', memetically selected, in many situations.

It is a bit of an 'everything possible', meme-magic, machine.

The memes that will be most competent are the ones that use the magic interfaces of the system, not knowing how the computer works. They don't have time to know how the computer works, they are busy with creating cognition.

When a high-level meme delegates lower-level memes to provide it with information, we can imagine little hacker wizard memes, in the substrate of the matrix so to say, that either figure out how the computer works and provide you with competence or not figure out how the computer works. Then they are discarded by memetic selection. Of course, they don't know how the computer works, they either represent a wiring that has meaning, or not.

The memes don't know what they mean when they start, it is only afterward that the meaning that made sense is leftover. This is the strange inversion of reasoning of Darwin's idea. Taken seriously for software-level mechanisms of computers that run meme-engines.

On the highest levels of this hierarchy, you have memes that are so competent, that they command everything that the whole computer can do with precision and artistic swag.

This then looks like a mechanism for a memetic machine to perfuse its computer with competence, to build information processing hierarchies that will use the computer, completely and competently.

Of course, the matrix machine is only a thought experiment, the matrix algorithm that runs on human brains is the one we know ourselves. What I like to call the 'cognition machine'.

I submit that this magic interface is the same class of software as our magic interface. Brain software is a magic banana maker matrix meme-engine.

And the brain has something to do with navigating the world as an animal. Not because in principle all meme machines are about that, but because this specific meme-machine evolved biologically. (So the wires that shape the memetic landscape are biased to be about navigating the world and being an animal).

A meme machine can create user-level abstract entities that have magic interfaces to the rest of the computer. Fulfilling the basic requirement we had for a brain-software.

When you go and wonder what is brain software, you probably will come up with something that sounds a bit odd at first, otherwise, you would probably be doing something wrong.

Those memes are allowed to get arbitrarily tiny, down to single-unit (whatever that might be) stupid computations. It's kind of sweet (technically sweet) to have a mechanism where arbitrarily abstract entities are allowed to coexist and refer to each other.

Of course, the main strange inversion of reasoning was already taken by Darwin. Also, you need McChulloch to say that the brain is an information-processing device. And Dawkins abstract evolution to replicators, and many giants on which shoulders we stand.

The other important ingredient is the power of interfaces. This is the power of what software is in the first place, especially pleasing to a software developer.

An important aspect to keep in mind is that the brain seems to be doing these high-dimensional, highly parallel computations. This is not a synonym for 'very powerful computer' but this has a very distinct computer science and cybernetics to it. This is the topic of the computational models of things like cell assemblies, that I explore here and elsewhere. (Current work). The main point is that 'everything possible a little bit' is a valid thing to say in such a computational system.

Don't limit your thinking to the computers you experience, only expand it, never make it smaller.

Similarly, with 'software' I don't mean the buggy, half-working user interfaces of current primitive tech. There is a larger view, that we are only glimpsing. It is deeply philosophical and would answer questions about the nature of abstraction, life the universe etc.

A cybernetic level of understanding of what software is, is just in its infancy.14

(more musings on the power of abstraction).

I find this especially pleasing because the burden of saying what the magic of the mind is is now on the power of abstraction and the magic of computer languages and interfaces.

As programmers we know, that there is something absurdly powerful about making a language, a higher-level interface that disregards details.

There is something deep in nature and cybernetics that explains it.

It is the same thing that creates life in the first place. Biology is sort of a higher-level language of chemistry, exploring itself.

I submit that everybody gleaning long enough at nature will find this somehow in one form or another. No wonder notions of theoretical physics that say that the universe is sort of alive, and maybe follows laws of natural selection, are intriguing. [insert Lex Friedman theoretic physics talks, or Sara Walker, talking about similar notions].

My bottom line is that the next frontier is understanding what building blocks are. What is the stuff? What makes a map better than the territory?

A screen, a prompt, an interface, a language, a toolkit, the genetic toolkit of evo-devo.

Turns out that the mystery of the mind was the mystery of the world and life all along. It's quite deep. Sara Walker calls it 'existence', which is the fundamental property of the universe. Where does natural selection come from? It is made from stuff presumably, somewhere at the bottom.

With this space of thinking you can muse about civilizations of memes, that build vast stores of knowledge. And archeologist memes going through the libraries of eons of accumulated knowledge.

Harmonious memes, cooperating memes, that never truly meet across times and space and yet, together contribute to the budding ideas, that only artistic memes and old wise-wizard memes can glean from the flow of time and history.

Social attractors of a kind that are bigger than a single meme on its own.

The Mystery Of The Mind Is The Mystery Of Interfaces

Prompts, screens, building blocks, toolboxes, languages. The user and the program, the map and the territory.

In my view, cognition is software with many levels of interfaces [also Dennett], which is an indispensable aspect of the software we build, too.

But what are interfaces? What is the power of building blocks, of abstraction? It is the next great frontier of our understanding, I believe.

Software is Magic

There is currently no overarching theory that even remotely talks in any kind of satisfying way about what software is.15

We are still figuring out what computer science is about in the first place. When some of the current deepest thinkers on programming think about these questions, the best description we have at the moment is magic. Programming is wizardry.

Explaining Cortex and Its Nuclei is Explaining Cognition

I am in the camp of people assuming that the interesting part of cognition happens in the cortex. In other words, it looks like modeling the function of the cortex is a path toward a model of cognition.


  1. Injuring the cortex makes a person lose specific capabilities.
  2. The cortex is the thing that explosively blew up in human evolution. 1a. Whatever is special about us is almost certainly special because of the cortex.
  3. The thalamus has 108 neurons; this is an upper limit for cortical inputs.
  4. Cortex neurons have 1010 inputs. -> This means that 10x maybe even 100x more connections to the cortex are from the cortex. (Braitenberg 1986) The stuff that cognition is is mostly stuff about the mind itself (reflexive). No wonder hallucinations are a thing.
  5. The cortex is more generic than other parts of the brain, it looks as if evolution found some basic building block (cortical columns?) which when duplicated, made more useful intelligence in the animal. -> Both the given that the cortex is generic/uniform and the given that areas are different is interesting.
  6. To explain cognition means to also explain altered states, out-of-body experiences, dreaming, mental pathologies, etc. (Metzinger).
  7. Everything with a brain also sleeps afaik. Roughly half of a good model of what the brain does concerns itself with stuff that happens during sleep. (we will see that this move makes us able to happily move this or that part of a cognition mechanism into a dream phase).
  8. The mind is self-assembled, without an external loss function.
  9. The mind is online and has resource constraints. Unlike a computer, there is generally not a pause when you ask a person something. The mind cannot stop reality in order to process some input. (But the existence of attentional blink shows us that a tradeoff in this space is being made).
  10. Whatever the basic building block of the cortex, it grows in the first 2 dimensions but not the height. Otherwise, we would have evolved more cortex volume, not surface area (Braitenberg …)

  1. Neurons can represent information states, neurons are universal [McCulloch and Pitts 1943, Church-Turing]
  2. Layering information representations leads to higher abstracted information representations [Rosenblatt]
  3. The same idea is the 'feature detectors' of computational neuroscience [Hubel and Wiesel 1960s]
  4. We sort of know that cognition machines are meme machines [Dennett 2017]

Reasonings / Intuitions:

The cortex is an information mixing machine - Braitenberg

  • When thinking about the cortex: a. We are allowed to use the rest of the system as an explanation. The cortex is about itself. -> The explanation is allowed to go in a loop.

    b. We are allowed to use an abstraction barrier and explain a part (a juggling ball) in terms of stuff

  • The cortex is like an ocean. This is part of the point.

|                                          |
|                      |    ^              |
|               ====== |    |      <-------+-----+
|               Cortex |    |              |     |
|               ====== |    |              |     | about itself
|                      |    |              |     |
|                      v    |              |     |
|                                 ---------+-----+
+--------------+                           |
|              |                           |
+--------------+                          |   | eyes
   rest of the brain                      +---+

Why is there a big fat cake of stuff that is about itself on top of the brain? And it seems to have to have to do with what we call cognition and intelligence.

What is [B], the brain, about? Easy to see for vehicles 1,2. There are simply sensors 1:1 about the world. With vehicle 5 we enter the world of being able to stick in more information processing units in between sensor and actuator. The interneurons of neuroscience. With that move, we have an internal world, and some pieces of our machine can be about that, instead of the sensor input.

The cortex is mostly about itself. If you look at a piece of cortex, you will mostly see neuronal units that are about… some other piece of cortex. Their world of a cortical neuronal unit is mostly the rest of the mind!16

Pyramidal cell activity - The Gasoline

I used to think hey this or that. Maybe Glia cells are important, maybe the neurons do computation inside themselves, who knows?

I have refined my model to simply take pyramidal cell activity as the center and move on.

  1. If something else than neuronal activity is how the brain works, then why is activity what drives muscles? It seems much more biologically harmonious that whatever is making muscles move is also what the brain is about. (Turns out that is pyramidal cell activity).
  2. If neurons do computation inside (microtubules? quantum computing…?, pixie dust at the synapses?) then how do you get the information from outside the neuron into the neuron and back out again? I see you need to solve the same engineering problem that the brain is solving in the first place, a second time. (I.e. how does the information go into the neuron and out?).
  3. Some sensor -> motor reactions happen in a time order that leaves time for only 1 or 2 action potentials through the brain. (Braitenberg 1977) Everything looks like there is nothing faster than action potentials that can move through the brain. The pyramidal axon, in general, has an evolutionary drive towards transducing action potentials as fast as possible. How would this fit with a view where something else than neuronal activity is doing the computation? Just the aesthetics of these 2 ideas don't fit in my opinion.
  4. Tissue with brain lesions is filled with glia, and this tissue does not substitute the brain function as far as we know. (Braitenberg 1977) This does not bode well for a model where glia calls are doing anything essential for cognition.
  5. The hippocampus has the most cells in the cortex [iirc], it is often the initiator of epilepsy. (Braitenberg 1986) Whatever the hippocampus is doing with all those cells then, it has to do with excitatory neuronal activity.
  6. With spiking neuronal activity you can put short-term memory in the units. Just make a circuit that goes in a circle. The simplest version of this is a pair of neurons going back and forth. This is a memory implementation. -> This all looks like neuronal activity can make short-term memory
  7. If I want to make a lot of short-term memory and have an evolutionary driver towards mid-term memory, I get something like a Hippocampus. Consider a cluster of cells with an evolutionary driver to make activity back and forth to store some previous state. a) Hippocampus has a lot of excitatory cells, so you can store a lot of 'activation'. b) If you want to abuse the circuit to simply make activity back and forth, it is useful to slow down the spiking rate. Hence, it is not surprising to me that the hippocampus has slow theta wave activity. c) You have another problem to solve, this ongoing memory activity should not spill to the rest of your (activity-based) system. So you curl up that piece of neurons and make it anatomically separate from the rest of your cortex. Related to this, you get the problem of epilepsy, you need to manage this immense activity of neurons now.
  8. We know some visual cortex circuits and they work with pyramidal cell activity.

Everything looks like and nothing does not look like the main substance of what is doing cognition is the activity of the (pyramidal) cells.

It is not the engine, the gasoline, the wheels or the steering wheel that is most important in a car.

If you say 'the synapses are the most important part'. I only need to make this analogy and I am at peace. The only thing that truly matters is that you don't bring in something non-essential because that is just wrong.

If the activity is like the gasoline, then what is the rest of the engine?

Neuronal Activity shapes the Neuronal Networks

The story of Hebbian Plasticity is a remarkable triumph of reason and imagination.

Let us assume that the persistence or repetition of a reverberatory activity (or "trace") tends to induce lasting cellular changes that add to its stability. … When an axon of cell A is near enough to excite a cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that A’s efficiency, as one of the cells firing B, is increased.17

Hebb [Hebb 1949] came with psychological reasoning and imagination about what neuronal tissue could be doing to explain what we observe from studying animals.

With something in mind to look for Eric Kandel went and found the biochemical basis for Hebbian plasticity, the work that made him win a Nobel prize.

It is important to note that fire together, wire together leads to a slightly wrong idea.

It is that every time I fire, I can look at who activated me - and those synapses I can strengthen. So there is fundamentally a causality in this notion, not mere juxtaposition or mere association. The rule needs to look at the activations of 2 time steps, not one.

It should be called Your fire is my wire, I wire if you fire, fire me and wire me, They who fire, make me wire, The more you fire, the more I wire, When you fire, I will wire, I am looking for some fire with my wire, I want to plug my wire in your fire.

[insert history, Turing, Lorente, Hopfield] [cool history part by Rafael Yuste here: Rafael Yuste: Can You See a Thought? Neuronal Ensembles as Emergent Units of Cortical Function]

Maybe you don't need Hebbian Plasticity

Perhaps Cortex doesn't actually do Hebbian Plasticity.

Iceberg Cell Assemblies

we find increases in neuronal excitability, accompanied by increases in membrane resistance and a reduction in spike threshold. We conclude that the formation of neuronal ensemble by photostimulation is mediated by cell-intrinsic changes in excitability, rather than by Hebbian synaptic plasticity or changes in local synaptic connectivity. We propose an “iceberg” model, by which increased neuronal excitability makes subthreshold connections become suprathreshold, increasing the functional effect of already existing synapses and generating a new neuronal ensemble.

Intrinsic excitability mechanisms of neuronal ensemble formation

Intuitively, if many of the neurons in a local connected group have more 'excitability', then this group has a higher chance to ignite.

It is a challenge for me to model this 'excitability plasticity' and see the iceberg cell assemblies.

Geometry makes Cell Assemblies

Without plasticity.

Here I have a model of this:

Sensor input (move mouse) activates a random projection in a neuronal area. There is no hebbian plasticity rule. The network has a geometry (topology). Neurons that are next to each other have a higher chance to be connected. This makes neuron time steps, takes the top (cap-k) neurons at each time.

This network is static so to say. It represents the inputs with a cell assembly.


All of these models are like having playdough in high dimensional spaces. You are allowed to morph the 'attractor' landscape by activating neurons together.

Cell assemblies and Memetic Landscapes

The brain is playing with fire.

Tery Sejnowski18

The meaning of this is that the brain works with positive feedback, and excitation, not negative feedback like the control systems we build. The fundamental problem coming out of this organization is epilepsy; Which unsurprisingly comes in many different forms.

The fire is igniting the network, and the biology of the system must keep this fire in check.

The second part of Hebbian Theory is the Cell Assemblies. The cell assemblies are well-connected sub-networks that support their activation together.19

In order to make something interesting, we need to limit the amount of activation, which Braitenberg called a Thought Pump 20. Another plain name might be inhibition model. (But the term thought pump is so daringly far-reaching, I love it).

This fire; survives, or it doesn't.

Epilepsy is a memetic problem: The simplest meme says 'activate everybody'. This suddenly gives us the perspective from one abstraction level further up: The machine that shapes the memes.

Imagine those cell assemblies, supporting each other, evolving strategies of survival etc. Survival means I keep on firing together with the people that activate me. They are replicators, they represent knowledge about how to replicate across neuron timesteps. They are memes [Dawkins 1976] made from neuronal substrates.

Note: These are somewhat different memes than the cultural unit of imitation. These memes here, the cell assemblies are the replicators of computational neuroscience. They have to do with cultural memes, but only because they are part of the functioning of the cognitive computer the cultural meme is infesting.

An inhibition model is a memetic landscape shaper. It says 'I make it harder for memes to exist'.

If you shit on all ideas evenly, only the good ideas will survive. This is the basic notion of a thought pump. And a doorway into neuro-physiological conceptualizations of memes and their substrates.

This activity flow needs a strategy now, in order to win out against other activity flows - else it will not win in the competitive environment that the thought pump creates.

Of course, it does never have strategies the way we do, its strategies are free-floating rationals and it has competence without comprehension [Dennett 1995].

The activity flow of neurons that support their activity together, is subject to the laws of natural selection. They are replicators, abstract entities with an agenda.

The computational substrate I have in mind is designed to support memes:

  1. Make activity scarce
  2. Make the activity mean things by connecting it to sensor and motor destinations and sources.
  3. At each time step, choose the best-connected subnetworks. I.e. make the subnetworks compete for connectedness.
  4. With 'online' plasticity, you get immediate associations between cell assemblies. Like playdough that sticks together.
  5. Mutal self-excitatory pockets of activation emerge from this arrangement. The Cell assemblies can be seen as a data structure in a high-dimensional computing framework21.
  6. We can see the cell assemblies as replicators[Dawkins], replicating across neuron timesteps. How well do they replicate? It depends on their connectivity, i.e. their meaning.
  7. Use memetic landscape tricks, that bias the system towards useful memes.22
  8. Optionally: Prune away everybody who is not active. [Literature on 'perceptual narrowing', and Neural Darwinism]

[This is all biologically reasonable]

The working hypothesis:

The structure and function of Cortex is to implement an assembly calculus.21

Memetic landscapes?

Ideas so far:

  • Control all sources of activity well, they make the meaning of the meme space.
  • If you make the prime source of activity the sensors, you get memes about the world.
  • If you reset the memes in a short amount of time, you get memes that are competent in a short amount of time which is a memetic driver for abstraction and hierarchies.
  • Now you have memetic drivers that make memes spread into areas/ nuclei that are not reset. If you make it hard after some criterion to spread into such nuclei, you select what kinds of memes you get.
  • If you make a slow meme-place, everybody wants to spread into that place, that is an evolutionary driver for midterm memory. Simply make a place where memes survive for long periods, they will compete to get there.
  • You can implement reward by activating a sub-network that makes up a meme, it is tempting to assume that some gamma activation would come from such a rationale.
  • You probably want to hook up to some Darwinian wires, that tell you 'this has to do with survival'. This way you can reward all the memes that the Darwinian brain approves of, i.e. you change the memetic landscape to favor memes that are useful for survival.23

Consider the simplest network of 2 neurons A and B.

[ A ] [ activate? ] [ B ]

The meaning of the network is the connections. It can mean A activates B, B activates A, or both, or none. Disallowing self-connections for the moment.

Consider the basic optimistic meme: 'I will stay alive', or 'I will be active'. We can draw the success matrix from above again.

activated! not activated
|   S     |   X    | competent
|         |        | (the meaning of the network means I stay active)
|   X     |   X    | incompetent
|         |        |

You need to be optimistic and competent in order to have success, or else you are discarded. The only meme that will survive is the one that says 'I will stay active'.

From memetic theory then, we can see that this network will stabilize into expressing the notion that 'A and B activate each other'. I.e. those are the connections that will survive and strengthen, everything else is discarded.

Kinda work in progress reasoning.

Let's assume for a moment that we can support memes with temporal structures. That is a cell assembly that waxes and wanes and represents a succession of states and so forth.

Here is a wonderful strange inversion of reasoning, it is that the meaning of memes happens before there is the meaning of memes.

Let me explain:

Suppose for the moment that virtually all activity must flow from the sensors. We see the first layer of memetics, which says 'Sensor active'. Since the simplest meme says 'I will stay active'. The simplest meaning is 'This sensor will stay active', in other words, 'The world will stay the same'.

Let's imagine a memetic unit (whatever that is, maybe a cell assembly) saying "I survive", in some derived meaning space. I.e. it is a space of possible meaning space, that is about the sensor memes, about the spatial and temporal structure of sensor meanings. The memes don't know yet what they mean, it is that the only memes that will survive are the ones that have competent connections I.e. they connected to the sensor memes in such a clever way, that they represent something about the world that stays the same, even though the sensors change!

To see why this is true consider that each meme competes for sensor activity with alternative memes. We can also imagine competing with a null hypothesis element. That says 'If you are not connected well enough to some people that activate you, the null hypothesis will be on'. These are thought pump considerations, making a more competitive environment for memes.

We can further imagine another layer of memes still, that simply stays on, because they connect to other memes with a temporal structure. Thereby representing the essence of some ongoing situation.

  1. These memes have a reason to find essences in situations.
  2. This search is memetic:

    First, you have many possible memes, all saying that they are eternal truths.

    Second, the wrong ones are discarded. And the ones that mean something useful about the world stay.

It is tempting to muse further. What about selfish memes that simply activate from all over the network? Maybe the answer is that they exist, and their meaning then is 'The mind exists', or 'The self exists', or 'The world exists'.

Maybe it is those memes that can give us the sense of something vaster and greater and eternal existing in the universe, our minds, our social realm, etc.

Maybe it is unallocated meaning units, that look for a reason to exist; Maybe it is some of those meaning units that a person can attribute things like god, or the fundamental spirit of nature or something.

Why is the basic meme 'I am eternal truth'? It is because of the optimism-drive reason from above. A pessimistic meme will be less successful than an optimistic one. It is the holy overlap between optimism and competence, that is favored by natural selection.

To relax this one further, it is useful to consider the basic meme 'I will be thought again'. This opens the door for what I call strategic memes (below, future). Strategic memes are memes that bias the system to be expressed again, not by staying active but by playing longer, cleverer games. (It was of course obvious from memetic reasoning that they should exist. We see that cell assemblies like this would exist, given the properties of the network above).

The basic possible rules of meme machines will create an environment that biases everything towards useful memes, that is the problem that the basic wiring of the brain somehow needs to solve.

One of the most important problems a meme machine has to solve is selfish memes that don't mean anything useful. So we can expect much of the brain's functioning to be solutions to this problem.

I would submit that describing, categorizing, fabricating, and transforming memetic landscapes is one of the basic problems of cybernetic psychology or 'synthetic memetics'.

Similar to this you might say all memes are a mix of static and dynamic activity flow. This idea comes up in Neural Assemblies by Guenther Palm.

A cell assembly implementation (online) A random directed graph with geometry(neighbors have a higher chance to be connected), without plasticity. At each time step, the top k neurons are allowed to be active. You can move the mouse to produce different inputs. (inputs in red). Only one cell assembly will 'win out' generally in such a setup.

With such an inhibtion model, each cell assembly implicitly inhibits its alternatives, since only one is expressed.


Figure 1: Ambiguous picture of either a young woman or an old woman.

A striking aspect about such ambiguous scenes, like this or the necker cube, is that you flip between interpretations, but you never see both interpretations at the same time. The system somehow 'decides' on one interpretation.

The basic meme strategy is the one that makes cell assemblies in the first place: Activate your activators.

Since meaning-level memes and 'sensor level' memes stand in a relationship, and since we introduce the inhibition model, which makes the system decide between some cell assemblies, you get these prediction/interpretation shapes:

         \             /
          \     A     /                   meaning-level
           \         /
            \       / <----->   B
             \     /   inibit
              \   /
                X                        sensor-level

A - Stable cell assembly

We can observe:

  • You can produce perception states by stretching cell assemblies across the meaning level and sensor level.
  • The meaning level parts of cell assemblies will bias the system towards 'perceiving' the world a certain way. (Activate your activators).
  • If meaning-level cell assemblies compete via something like the inhibition scheme of the model above, it is simply the nature of the mechanism that one wins out, never multiple.
  • I submit that this fits well with the emerging view of cognition called 'predictive processing', [Andy Clark, Anil Seth, etc.]
  • The inverted pizza piece, or the inverted ice-berg:
    • Perception is a meaning-level construct.
    • The meaning level is much larger than the sensor level (i.e. most connections in the cortex come from the cortex).
    • The meaning level can take larger situations into account. Perhaps the basic reason for having a cortex in the first place.

Perceptions are stable memes, everybody saying 'I am true'. But only the ones that get enough activation flow support from the rest of the system, including the sensors, win out. I.e. the meaning of a meme is its connections. The connections are allowed to be random at first since we can select the meanings afterward via natural selection. In a system where most activation flows from the sensors, this will select the meanings that represent true things about the world.

Cell Assemblies Lit

Tübinger Cell Assemblies, The Concept Of Good Ideas And Thought Pumps

Paper Cell Assemblies in the Cerebral Cortex, V. Braitenberg 1977 24.

I am using the term neuronal unit interchangeably with neuron. And the term neuronal area interchangeably cortical area.

Cell assemblies are subnetworks of well-connected neurons that activate each other. In an environment where activity is kept in check (via the inhibition module), only the cells that activate each other well will stay active. The inhibition element might work by setting a threshold of synaptic input, below which neurons will not be active. (Threshold control). Note that a simple inhibition model would be 'cap-k', saying that the top k neurons are allowed to be active.

The fundamental property you get from this is what you can call 'pattern complete'. We say a cell assembly can ignite from activating a subset of its neurons.

A cell assembly lives via the excitation of its elements.

Via Hebbian plasticity, 2 cell assemblies happening at the same time, will associate, we will find a subnetwork that is connected to both.25

From one perspective, we can say that there is only ever 1 cell assembly in the whole brain. Intuitively, you can imagine lowering the threshold maximally and getting all neurons to fire. This doesn't have to be this way, it depends on the connectivity of the network.

This would be the 'maximal cell assembly'. Presumably, that is the whole brain (that would be epilepsy).

The notion of a cell assembly only makes sense if you take the inhibition into account. Otherwise, it is hard to get to the notion of a piece of activity that survives on its own, because the elements activate each other.

A cell assembly might be composed of sub-cell-assemblies, that is, there are well-connected subnetworks of active neurons, which might form a well-supported cell assembly on their own. We say that the cell assemblies have multiple centers.

Braitenberg mentions homogenous and non-homogenous cell assemblies.

A homogenous cell assembly says that each neuron is connected with the same weight as everybody else, this is a theory-only construct, I think, to point to the non-homogenous cell assemblies: In a non-homogenous cell assembly, there are subnetworks of neurons that are better connected to themselves than to the rest. Again we say that such a cell assembly has multiple centers. Where a center is a subnetwork that would be able to support a cell assembly on its own.

Excitability is simply the inverse of the threshold; (I also called it eagerness before encountering the word).

If my threshold is low, I am eager to ignite.

If my excitability is high, I am eager to fire.

This means that you only need a little bit of input (synaptic inputs) to 'ignite'/fire.

This is simply to make it easier to think because sometimes I want to have that perspective.

If you imagine a cell assembly of mutually supporting neurons (a cell assembly center) and now you increase the excitability of the neuronal area, you will see that additional neurons are firing, the more you increase the excitability. These are not necessarily part of the center. They don't need to support the cell assembly at all, they simply ride on the existing activity, without feeding back to where it comes from. (Note that you only get this kind of activity with a low threshold. If the threshold is extremely high in the system, only the core of the best-connected subnetworks will be active - i.e. they all activate each other strongly).

We call this the halo of a cell assembly. The halo is like the outskirts of the cell assembly city. Where the center is the city center. It supports the activity of its halo, but if you increase the threshold, you will narrow the assembly down to its center.

For some reason, this makes me think of Arthur C. Clarke's 2nd law:

The only way of discovering the limits of the possible is to venture a little way past them into the impossible.

From this, you can craft a hypothetical mechanism that finds well-connected ideas, ideas that fit with the rest of the situation of the brain.

Say you have some inputs coming into your thinking goo (neuronal areas), that activate a subset of your neurons I. You will form a cell assembly from this. Call it FI. Note that the neurons that are part of the cell assembly don't have to be the ones that listen to the sensors in the first place. (With a random graph they are not, see my programmatic versions of this).

What you can do now is increase the excitability, activating a cell assembly E(FI), this is the union of FI and FI-halo. This represents everything, more or less, that is sort of associated with the inputs your sensors pick up. Maybe that stands for the 'excited' FI.

E(FI) will almost certainly be a non-homogenous cell assembly, containing multiple, maybe many, many [depends on the network] centers. One of them is the initial FI, others are BI, CI, … From cybernetic intuition, it is very likely that FI is not the strongest of these centers. I.e. there is some other sub-cell-assembly in E(FI), that has a stronger mutual activation. I am not sure yet and there many perspectives to take to make this super plain and obvious, but the intuition here is that there is a 'better idea' available; That there is a solution to the overall situation so to say, a meme (cell assembly) that has the right connections to be supported in the current situation (since the meaning of the world is in the connections of the network - it's quite wild, yes).

One thing to consider is that the rest of the cognition machine is making all those top-down cell assemblies.

Since those participants in the system are activating their activators, the shape of the lower level - E(FI) is determined by the larger situation at hand.

I.e. whatever is well supported by the rest of the system is activated. In other words, you bias what is possible by taking the larger situation into account. In other words, the system has a vague idea about what is true, you will not be a good meme, your connectivity will not 'make sense', your cell assemblies will not be supported, if your meaning is something completely else than what the system thinks is roughly true.

If I am a cell assembly that says 'Here is a face', I go and activate the lower perception people to represent inputs that look like a face. So E(FI) has a shape where all the activity that fits with the notion 'here is a face' are more active. Note that 'good idea' here means very specifically a subnetwork of neurons that happen to be connected in such a way that they get activated by whoever is active right now - stronger than alternative sub-networks.

We can observe that this is true by imagining 2 competing centers in E(FI), say BI and CI. BI might be slightly better supported by some top-down connections from the network. In other words, BI has friends higher up that support it.

The second step, as you might have guessed, is that we increase the threshold again. This will select the best connected sub-assembly from all possible centers inside E(FI). Finding for instance BI. [I can show programmatically how you get the best in a simple model. This is quite intriguing. And fits with the notion that something immensely parallel is happening in the brain].

This mechanism is what Braitenberg called a thought pump. (Note that this is a slightly different one from the one above; In "The Vehicles" the thought pump just meant a threshold control. In the 1977 paper, 'thought pump' means a very specific dynamic threshold control that searches for well-supported cell assemblies).

Now you can imagine a process where you increase and decrease the threshold in quick succession - perhaps creating the alpha, beta, gamma EEG waves. I.e. perhaps beta or gamma frequency represents the 'upper' step of a 2-step oscillation.

You will get a sequence of cell assemblies, I imagine a ball moving around in meaning-space. Perhaps it is especially interesting to have the ball move around then then suddenly stay. If it stays, you have found connectivity in the network that is the 'best-supported idea' of the network.

Or you move around the ball and it suddenly grows. This is a 'good idea that fits many things'. G. Palm is musing that this is maybe a mechanism for good jokes, too. Note that a good idea is an idea well supported by the current situation.

Another parameter would be the number of time steps you wait until you declare something 'the best interpretation of the situation'.

And another one would be the amount of activity that you declare a 'good idea that fits well'.

It looks like it is possible to shape a network memetically to represent the world (see sensor flow musings above, and predictor mechanisms, coming soon). Perhaps you build a network that represents the meanings of the world, and then you use that network to find further meanings again.

The thought pump can ask the existing network 'What fits?'. If the network grew from the sensors, then it has to do with real things. 'What fits' will be something that 'makes sense', that is realistic from many angles.

It is interesting to consider that there are 2 sources of time and causality with those cell assemblies.

1 is that I connected A->B via Hebbian plasticity, which means that if A is active, it will flow activity to B, in the next neuron time step. This is the fastest time representation available with that substrate, then.26

2 are the thought sequences of the thought pump. It seems like this needs at least 2 neuron time steps. 1 to increase and 1 to narrow down the activity flow. This doesn't mean that 2 is always faster than 1 because you have different neuron frequencies, varying by more than 2x,4x,6x or something.

The idea that you can lower the eagerness and get more fine-grained thought is quite intriguing. Consider that only the cell assemblies that fit the situation especially well will survive.

What you can do now is find an especially well-connected assembly. That one is still large, even though the threshold is low.27

Perhaps Hofstadter's abstraction ceiling (see his Strange Loop talk) is a brain feeling created from the situation of increasing the threshold very tight, without the wonderful release of a good idea.

Perhaps dopamine modifies what counts as a good idea. The bar for connectedness is lower so to say, biasing the system towards action, not deliberation. Perhaps psychosis is that you take the first good-looking idea and believe it, even though it was not the best idea you could have come up with at all.

Interlude: Zebrafish Larva

Zebrafish larvae are translucent for the first time in their life.

Scientists at the Howard Hughes Medical Institute studied live zebrafish larvae that had been genetically encoded with a calcium indicator called GCaMP5G. They suspended the larva in a gel and then beamed it with lasers. Just before a neuron fires, its action potential is expressed via a spike in calcium ions, so when one of the genetically modified larva's neurons reached its action potential, it glowed. This showed the researchers the firing of the neurons without them having to attach a bunch of electrodes to the fish. Over the course of an hour the researchers used laser beams to scan the larva every 1.3 seconds, exciting the retina of the zebrafish with each scan. This microscopy method allowed the researchers to record up to 80 percent of the fish's 100,000 neurons at single-cell resolution. This is the first time scientists have recorded such a high percentage of an organism's brain activity at such a high resolution.

Funny, there is a meta-phenomenon to this video: There is a huge spike in youtube 'most replayed'. At 00:18, hard to miss, a huge flash of activity through whatever that big middle part is.

Maybe it means nothing, but our imagination is pumped. It thinks! The small flashes are just as interesting as the big flashes. Neuronal flashes would make sense in light of thought-pump mechanisms.

It is perhaps counterintuitive if this is a thought pump we are looking at, a big flash might be more something like a re-orienting. It is that the current interpretations are not so interesting, useful, or make enough sense, then it makes sense to lower the threshold a lot of a moment. This corresponds to searching in a much wider meaning space radius for a new interpretation, presumably because the current one is insufficient.

Whether those are thought pump oscillations I cannot say. Since we see a frame every 1.3 seconds, this might flash like this all the time but we just did not have luck seeing it.

Perhaps thought-pump mechanisms are layered and come in local and global varieties?

Funny, those zebrafish guys analyze the activity in terms of cell assemblies: Neural assemblies uncovered by generative modeling explain whole-brain activity statistics.

They get these subnetworks, quite interesting. It has some philosophical overlap with the cell assemblies of Braitenberg and Palm that I describe here. But the methods are different. As far as I can see, they don't have an inhibition model; Which is one of the requirements of getting to these pockets of self-excitatory activation I talk about here.

Statistical considerations on the cerebrum / Holographic encoding?

That is from the paper:

  1. Distinguish an A-system of cortico-cortical connections from a B-system.
  2. A-system is the apical dendrites of pyramidal cells in upper layers [1 and 2?], that listen presumably to everything else in the brain.
  3. B-systems are local connections, the basal dendrites of pyramidal cells,
  4. With some careful reasoning you conclude that the connectivity in the B-system is ~0.1. (you are connected to 10% of the local network).
  5. From statistical reasoning, it looks like you can make the square root of N (number of neurons in cerebrum), 'compartments'. Each compartment listens to everybody else. It is intriguing that size and count overlap with that of a 'cortical column'. Further, if every compartment listens to all other compartments, then probably the activity in a compartment should be correlated; Intriguingly, the orientation columns we know look like they have this property.

thoughts ->

(The software engineering approach dictates that we let our imagination go wild sometimes. Let's assume for a moment you would get this or that kind of computational properties of the system, then …).

From this then you would get the hypothesis that each compartment (map to cortical columns) receives a holographic encoding from the rest of the system. So each part gets a representation of the whole via the A-System, the network would not mean we throw away the messiness of the connections of the network. It would still have the cell asssembly properties, Hebbian plasticity and so forth. So each compartment would learn to represent different things, depending on the overall context. But - you could store and manipulate the overall context easily and this would be useful. I.e. that is the state of which compartments are active.

This is very intriguing because it would be a bridge to hyperdimensional computing. [HD/VSA]. In this hypothesis, you could store an encoding of the overall state of the cortex in roughly 105 neurons. Btw the numbers are very large. In the literature, the 'blessing of dimensionality' comes at 104.

We would be able to do HD computing, the fundamental computations are trivial to implement. Simply align 2 input vectors and do either addition or multiplication. There are many ways we could imagine a hypothetical circuit or nucleus, or the cortex itself to do such math.

From HD computing you get wonderful properties, in meaning-spaces. (See HD/VSA).

The fundamental operations are bind and bundle.

With bind you find the contrast of 2 points in meaning space, it corresponds to the surface of a rectangle, made from width and height.

With the bundle, you find the 'between' between 2 points in meaning space. You might say the essence.

The 'perspective', 'zoom', and 'inside perspective' move:

  1. Find the essence of a cloud of meanings (with bundle)
  2. Use this essence as a perspective point. I.e. you bind all the points, highlighting the differences between the points.

This would be a beautiful on-the-fly perspective mechanism, blowing up the meaning spaces around a point that matters.28

You would be able to temporarily and totally dynamically make the brain into a brain that is good at representing 'this kind of meaning space'. And that is allowed to be completely arbitrary and so forth.

This fits me with the observation that a different perspective completely changes the kinds of thoughts we think.

Via a zoom like this, we could move around in meaning-space. From outside, there is a cloud of meaning points that all look similarly far apart, after a zoom, it would be like sitting at the center of this meaning cloud. And seeing the differences between the meanings.

The notion of contrast and perspective is one of the challenges to purely connectionist models. At first glance, I would think everything is mud if everything is sort of associated with everything else.

Modeling a threshold-device

See Wilson, Cowan 1972. It is intriguing to consider the 'population dynamics' of inhibitory and excitatory neurons. Maybe all you need is a 'bouncy' excitation control,


Figure 2: From here, googled damped oscillations. This is the curve a spring makes, bouncing and then settling on some set point.

Maybe you get away with modeling such a bouncy threshold.

;; Imagine [I], the population of inhibitory neurons,
;; and [E], the population of excitatory neurons.

 ;;                                     P
 ;;                                     | (input from sensors etc.)
 ;;                                     |
 ;;                                     |
 ;; +--------+ <--------------  +-------+-+
 ;; |        |                  |       v |
 ;; |   I    |---------------|  |   E     |
 ;; +--------+                  +---------+
 ;;     ^
 ;;     |
 ;;     |
 ;;     |
 ;;     |
 ;;   inhibitory-drivers

If there is a lot of (sensor) inputs P coming in, your threshold control might automatically go into an oscillation mode.

Like pulling on a spring and then it bounces around. This might already be the mechanism to get thought pumps in the cortex.

I get an inhibition model with a few parameters:

  • The bounciness, how much I oscillate.
  • The set point of excitation, that I make the system go towards.
  • Maybe a min and a max of excitation/threshold.


(Idea also from Braitenberg when he considered the thought-sequences).

Another thing you can put in the inhibition model is attenuation, this is an (inhibition) rule that says, 'if you were active recently, it's harder for you to be active now'. I.e. fresh neurons are eager.

This is one of the known properties of biological neurons, and it has a use in a thought pump mechanism.

In effect, we memetically select for 'freshness', at least temporarily. Attenuation is a memetic landscape concern.

Another way to put it is fresh memes are eager.

Consider: Verbal attenuation is very quick. I think I can say a word 4 times in succession and it becomes weird. 29 You can do this experiment yourself by speaking aloud and repeating any word at random. Unless you cheat in some way and imagine speaking the word in different contexts or so forth, after a short amount of repetition the word will sound weird. You might even have the feeling 'How is this even a word?'. 'This is strange'.30

Almost as if we would experience now the syllables and the sound, but the meaning is gone. See Semantic satiation.

The neurophysiological interpretation at hand is that some neurons that represent the meaning of the concept, are now becoming attenuated. I.e. the meaning is hard to re-ignite, sort of gone.

There is a satisfying reason why the attenuation would be high in some language cortexes. And why we would model different neuronal areas with different attenuation parameters.

With high attenuation, you get 'guaranteed' thought-pump sequences. Since we make it harder for the same cell assembly to stay ignited. In a thought pump jump, we heavily bias the system towards settling down on a fresh idea.

If we consider that language is virtually useless, unless it has some kind of syntax. It seems like there would be an evolutionary driver then, to make language cortex with especially high attenuation.

[My vision is the activity flowing like an ameba, or the 'ball of highest connectivity' moving from place to place].

More conjectures:

I think perhaps the opposite driver would be found in the cortex that represents 'meanings that stay'. This might be the visual scene - objects stay around without their meaning changing from looking at them. I think I can look at an object for more than 10 minutes and still don't feel like it becomes meaningless. This is in a way the opposite challenge to models that simply say 'attenuation makes satiation effects'. You need to also explain why there are no satiation effects somewhere else, then.

The cell assembly configurations in a way are the attractor states of the system. (I am not a math guy, but it sort of sounds like something like that).

Either way, I want to give this a name conflagrant modes, which comes from the Braitenberg paper.

Names for the possible configurations of activity:

conflagrant mode, ignition modes, supported activity configurations, attractor states, activity flow states, interpretations, representations

"My conflagrant modes tell me this is a bad idea". (When you have a hunch, meaning that some of your meaning spaces support the idea this is a bad idea).

"My current interpretations are joyful".

Names for the cell assemblies that are around a lot:

stable meme cities, well connected meaning cities, stable ideas, ideas that fit many situations

"This just gave me a spark for an idea."

"You just ignited some interesting ideas in my networks."

"These ideas are so out there, I have to build an ignition infrastructure to support them first."

"Your perspective just made me flip into a completely different conflagrant mode."

Simple attenuation model [I'll split this into a blog post]:

;; ------------------------------
;; 'Summed History' Attenuation model
;; ------------------------------
;;    1. Sum history
;;      activation history
;;     +----+ +----+ +----+
;;     |    | |    | |    |
;;     | -X-+-+--X-+-+----+--->
;;     |    | |    | |    |
;;     | -X-+-+--X-+-+--X-+--->   how-often-active * attenuation-malus
;;     |    | |    | |    |
;;     +----+ +----+ +----+                       |
;;                                                |
;;     <-|----|------|------                      |
;;          n-hist                                |
;;                          +---------------------+
;;                          |
;;  2. apply malus          | + 1, so we don't divide by less than 1
;;                          |
;;                          v
;;     +----+            +----+
;; n   | 0  |            | .. |
;; n2  | 1.0|            | 0.3|
;; n3  | 11.|      /     | 2. |    ------------>  updated synaptic input
;;     |    |            |    |
;;     |    |            |    |
;;     |    |            |    |
;;     +----+            +----+
;;     synaptic input      attenuation malus
;; The malus can be applied:
;; - substractive
;; - divisive (like this). (thining absolutely with changing inputs is harder)
;; The malus can be determined:
;; - sum (like this)
;; - weighted sum (for instance 'more recent, more attentuation')
;; - ..
;; You can also decide that the malus is somehow %-wise of the current inputs
;; Because with the plasticity we always have to keep in mind that inputs are
;; only make sense to compare within a time step.
;; attenuation-malus
;; 0: no attenuation
;; Exactly 1.0: Half the synaptic input after being active once and so forth
;; I'll just say that attenuation-hist-n is something like 10 and factor is something like 0.1

Here is a 'rolling malus' implementation in the browser.

;; Here, I do 1 simpler that is just carry over an attenuation malus
;; 'Rolling malus' implementation
;; 1. Every time you are active, your malus goes up.
;; 2. With every time step the malus decays.
;; 3. The malus is applied to the synaptic input divisively
;; kinda simplest thing, 1-3 could also be more complicated functions.

(defn attenuation
  [{:as state
    :keys [attenuation-malus attenuation-decay
           attenuation-malus-factor synaptic-input n-neurons
  (let [attenuation-malus (or attenuation-malus
                                  #js [n-neurons])))
        ;; decay the malus from previous step
        attenuation-malus (mathjs/multiply
                            (- 1 attenuation-decay))
          ;; accumulate the malus on everybody active
          (.subset attenuation-malus
                   (mathjs/index activations)
                   (mathjs/add (mathjs/subset
                                 (mathjs/index activations))
    (assoc state
      :synaptic-input (mathjs/dotDivide
                        (mathjs/add 1 attenuation-malus))
      :attenuation-malus attenuation-malus)))

Code Here


Figure 3: neuronal area with low attenuation, looks like the same cell assembly stays active.


Figure 4: neuronal area with more attenuation, cell assembly is forced to have temporal structure, later sensory overload I suppose. (That is perhaps the sensor input neurons become the most viable cell assembly, perhaps I should play with more attenuation decay).

Ah, the joy of seeing one's ideas inside the computer.

This playground is up here: Assembly Friends #2.

That is a random directed graph with some geometry. The geometry wraps so that at one end the connections go to the other side (like a torus).

There is a threshold device, too:

(defn rand-cap-k-threshold-device
  (fn [{:as state :keys [synaptic-input]}]
    (assoc state
           (ac/cap-k (rand-nth numbers) synaptic-input))))

;; ...

                             [(* (:threshold-device-high
                              (* (:threshold-device-low

That is a cap-k, going between 5 and 10% of the neurons or something; Randomly.

It is one of the simplest threshold devices you can come up with I think. There is a chance it already gets the job done.

It would not find G. Palms good ideas though. Those depend on a threshold, not a neuron count cutoff. Because a good idea would mean that you have a high threshold and a lot of activity at the same time.

The white box is a sensory field. Whenever one of the stimuli (colored balls) is in the white box, a subset of neurons in the neuronal area are activated (unconditionally) - the projection neurons of the stimulus.

Finally, the connections in the area are updated via Hebbian plasticity.

Input Circuits And Latent Spaces / The Games of the Circuits

I am a Murray Sherman fan; There is some cool work they did to establish these things empirically [Talk 1, 2, 3 these are all super dense and elucidating].

Also, I'm only picking the 50% that land on my views here; And probably leave out the most important parts.

An Input Nucleus To Rule Them All

  • All neocortex (vs. Allocortex, the evolutionarily preserved, old cortex) receives driving inputs from Thalamus nuclei in layer 4.
  • All sensory inputs to Neocortex go through the Thalamus. Note that olfaction is the exception, it goes to Allocortex.
  • Thalamus is in a strategic position to modify the inputs.
  • LGN is the visual relay nucleus, MGN auditory and so forth.
  • Relay nuclei make driving inputs to Cortex (v1,…)
  • modulatory inputs are shaping activity, driving inputs carry information and are the key inputs for activation. (this distinction is a key contribution by Sherman and collaborators).

Cortical hierarchical arrangments:

conventional view, *wrong*:

                       cortico-cortical connections,

        cortex primary                  cortex secondary
    +-------------+                    +------------+
    |             |                    |            |
    |   ^     ----+----------------->  |         ---+-------------> ....
    |   |         |                    |            |
    +---+---------+                    +------------+
    |^  |  |   Thalamus, "Relay / Gate"
-----+  sensors

In this naive view, you say that the cortico-cortical connections would make information flow inputs up a hierarchy of (perhaps 'feed-forward') cortical areas. Say (v1->v2->Mt->…).

As far as I understand, Sherman and collaborates primarily established the concept of driving vs modulatory input, which leads us to this update:

udpated: Thalamus makes driving inputs to *all* cortics areas,

                                         +--  cortico-cortical are not driving inputs
                                         |    (their role is something more subtle)
          cortex primary                 |         cortex secondary
     +-----------------------+           |      +---------------------+
     |                       +-- ---  ---+  ----+>                   -+ --- - - > ...
     |          ^          | |                  |  ^             |    |
     +----------+----------+-+                  +--+-------------+----+
                |          |                       |             |
                |          |                       |             |
                |          +-----------------+     |             +------------> ...
                |                            |     |
           +----+------+                  +--+-----+-+
           |    |      |                  |  v     | |
           |^          |                  |          |        Thalamus
           ++----------+                  +----------+
            |   first order nucleus         second order nucleus
            |   LGN, MGN, ..                Pulvinar, ...

driving inputs

- - ->
modulatory inputs

  • Relay nuclei make driving inputs to the cortex.
  • A relay nucleus is in turn characterized by its driving inputs.
  • First order relay nucleus: Receives driving inputs from the periphery
  • Higher order relay nucleus: Receives driving inputs from the cortex (layer 5)

So this goes in a zig-zag with thalamus.

It is crucial, I think, to realize that the information flow is in a zig-zag, but the processing flow is parallelized across the relay nuclei of the thalamus.

'Bottom-up' vs 'Top-down' is somewhat misleading from this view, they label a virtual information flow that goes across elements, but those compute in parallel.

Similarly, 'ventral stream' and 'dorsal stream' are virtual information flows, but the computation is parallelized.

'Conventional' view:

The ideas being invoked:


      +-------------+                            ^
      |  ^          |                            |
      +--+----------+                            +-+
      |  |<-        |  ...                         |
      +----+--------+                              |  some kind of information flow
      |  > |        |  secondary area           +--+  going through cortical areas.
      +--+----------+                           |
      |  |          |  primary area             +
                                                 Either like some kind of lighting worm,
                                                 or some kind of waves of information processing.


     |          |   |
     |         /    |
     |     -/  \    |
     |     v        |

    At the same time, there is a top down processing
    sort of shaping and giving some context perhaps?
    (huge open questions)

Updated view:

             cortical areas            thalamic relay nuclei    ^ |
                                                                | |  virtual top-down and buttom-up
              +--------+                                        | |  information flows
              |     c4 | <------------------- d                 | |
              +--------+                                        | |
              |     c3 | <------------------- c                 | |
              +--------+                                        | |
              |     c2 | <------------------- b                 | |
              +--------+                                        | |
              |     c1 | <------------------- a                 | |
              +--------+                                        | |
                                                                | |
                                                                | v

a  - input relay nucleus
b  - second order relay nucleus, ...
c1 - primary cortical area, ...

The information processing flow is parallelized over the thalamic relay nuclei, i.e. information goes from a->c1, at the same time as b->c2 and so forth.

Perhaps it makes more sense than to say there is a virtual information flow, and that goes top-down, bottom-up. Note that the information processing arrrows are allowed to be shorter, this is the point of parallel processing. It means that you can do more in shorter timesteps.31

This model applies to ventral and dorsal processing streams, too.

My reasoning:

This makes me think we can take it as the base assumption that the cortical areas (fMRI) come from whatever input nuclei these areas look at, at Thalamus.

It's not clear whether it is the whole story, but the base problem of the (fMRI) so-called 'cortical areas' is explained by this. Cortex activation, presumably including the BOLD signal, would be driven by whatever thalamic nuclei they look at.

What the cortex people should look at is what kinds of 'information mix' you get from each thalamic nucleus, and where they go.

This fits with the observation from the infamous ferret routing study [Nancy Knawishers Lecture is nice]. If the primary auditory looks at LGN, you get vision and orientation columns and so forth in the primary auditory cortex.

The most parsimonious explanation I can think of:

  1. Neocortex is relatively generic.
  2. The neocortex will represent inputs, depending on what inputs come from the thalamus in layer 4.

It makes sense that evolution would hit on a more general-purpose computer and repeat its architecture over and over. Our computer chips are general purpose and there is a reason. It is easier to program. General purpose is more powerful than narrow purpose.

This doesn't rule out that some cortex is different from other cortex (see musings on attenuation above for a reason). What might be true in a weaker form is that all sensory cortex is relatively the same.

The strongest claim here would be something like: If you could experimentally swap say MT and Fusiform thalamic input circuits*, you would simply swap movement and face representing "cortical areas". Presumably without much loss of function.32

*) pre-development


  • Do the projections of the core ever split into multiple areas?
  • Do the projections of thalamic nuclei ever mix?
  • You might wonder if 'functional connectivity' [Dynamic functional connectivity] would be areas looking at the same nuclei. The challenge with this is, what is the use of different areas to look at the same inputs? It would be like loading a file twice into your operating system. Perhaps it is hard for architectural or memetic reasons to have multiple large cell assemblies in the same area. (Would on the inhibition model the brain implements, if it implements an assembly calculus). (Perhaps multiple cell assemblies in 1 area simply don't make much sense, since they merge). Perhaps that would be a reason to have multiple processing streams on the same information.
  • Perhaps another element in the basic circuitry of the cortex is needed to explain it.

Note that the neocortex has 6 layers, allocortex has 3 layers. This implies that the neocortex evolved by duplicating the cortical layers, and by hooking up to the thalamus<->cortical organization. (Allocortex doesn't have layer 4-6).

This means that layers 1-3 are the evolutionarily preserved layers. Making it obvious again: You need to understand the thalamus, if you want to understand the neocortex.

Thalamic connection kinds

  • Thalamic nuclei are heterogeneous, we differentiate at least a matrix and a core system [Matrix-core]
  • Matrix is less well understood and projects diffusely to the whole Cortex. It receives inputs from the striatum and subcortex [got this from another talk]
  • Core is making the driving activation to the Cortex, projecting to layer 4 neurons.
  • Core neurons receive glutaminergic inputs from the periphery

My musings:

Matrix would be a candidate for an Hyperdimensional vectors [hd/vsa] implementation.

At the very least, the interpretation that Matrix is implementing some kind of 'perspective', 'context', or 'landscape shaper' mechanism lies at hand. If you would look, it probably would sort of say a little bit what kinds of things are allowed to be active.

If the brain is doing hyperdimensional vectors, this would be a candidate 'context nucleus' in my opinion. One of the things you could do with such a computational construct is the on-the-fly perspective algorithm I mentioned earlier. Another alternative is that the brain evolved somewhere half way towards using hypervectors. There is a perspective that makes this option exciting: If this is a half-baked implementation but still works so well, what will an ideal implementation do for the cognition of that software?

One might muse that such an element would help create mid-term memory. In that case, some yet unspecified process would somehow make the hippocampus store hyperdimensional meaning points of a holographic cortex encoding. (see Statistical considerations on the cerebrum / Holographic encoding? for why). Another unspecified process would recruit neuronal activity from the hippocampus to the matrix, giving the cortex a context/perspective. From there, the cortex could then fill in the rest of what we call the memory of a memory. (more below).

TRN the gate

  • Thalamic Reticular Nucleus (TRN) is like a blanket around the thalamic nuclei
      |     ^  C              |  Cortex
      +-----+-----------------+                     3)
      |     +->B    |         |  TRN  <-------------+
      +-----+-------+---------+                     |
            |       | 2)                            | drives TRN and inhibits Thalamus
      +-----+-------+---------+                     |
      |  ^  | A     _         |                     | Serotin++
      +-----------------------+                     |
         |                  Thalamus   |------------+
---------+                                          |
 driving inputs                                     +---[brainstem etc.]

 modulatory inputs, cortex, subcortical

A - Relay nucleus driving neuron, makes projections to layer 4 in Cortex.
B - All relay neurons leave collaterals in TRN
C - layer 4 input neuron

2) - TRN inhibits back to Thalamus
3) - Neuromodulatory inputs from the brainstem and so forth are making strong inputs to TRN,
     also, Serotonin modulates the TRN

The TRN is a regular sheet around the thalamic input nuclei. The evolutionary drivers for TRN are easy to understand. It is an input gate. With low inputs from 3 (brainstem neuromodulator makers), in awake states, the gate is open. With high 3, in drowsiness [this is established experimentally with cats and so forth], the gate is closed.

TRN is controlling 1:1 how much activity flows into the cortex. This was one of the basic requirements you have for an activity management device, it controls its inputs.

So I am imagining a blue (blue are the sensors on my whiteboard) input nucleus ball, the ball receives inputs from the sensors, and represents the faithfully (drivers), there is a dynamic lid around the blue nucleus, that allows the machine to say how much of the activity flows into the thinking goo around.

Serotonin: At first glance the role of serotonin is ambivalent, it has to do with attention (cognitive neuroscience term), but also with drowsiness and sleep? [Lecture 8. The Thalamus Structure, Function and Dysfunction].

It lies at hand that closing the gate (activating TRN) is a mechanism happening during sleep. Shut down the inputs (mostly). This way the thinking goo can do something else, presumably it is relaxed from analyzing sensor data and performing as a behaving animal (broadly speaking).

Speculation: If we activate TRN, we shut down the inputs; Making the internal activity in the cortex less about the sensors and more about its 'own' context.

You can make this tradeoff presumably every time the sensor data is not that useful for whatever we want to do. Where the sensor data would somehow mess with the kind of information processing you are doing.

Note that it would be more useful if the default setpoint of such a gate is somewhere in the middle. This way, you could open the gate, and flood the cortex with inputs (awake, surprise?), or close the gate, and starve the cortex of inputs (drowsiness, perhaps thinking and imagination?).

I think we can assume this is a rather general element and operation in the system, useful for many kinds of mechanisms.

Of course LSD is a serotonin receptor agonist, and what is being reported from such altered states broadly maps onto what you might expect from the gate being closed, making the cognition be about itself - making the cognition about something like imagination perhaps.

From this, you will see that a maximally large dose of LSD will make you blind and deaf since the inputs going into the Cortex are maximally low, similar to a sleep pattern presumably.

It is ironic when substance users talk about opening their minds, while the mechanism that produces this state comes from shutting down the input gate to the world.

I suspect much more interesting and ultimately trippy thoughts are thought by building once understanding; Using pencil, paper, whiteboards and code editors as thinking aids;


  • Does TRN have any segments?

Some alternatives come to mind:

  1. TRN is a single-element concept, and it is inhibited and activated in unison


  1. TRN has clearly demarcated segments, that are active or not active.

2a. The segments correspond to thalamic input nuclei 2b. Perhaps the segments correspond to 'functionally connected networks' [Dynamic functional connectivity] 2c. Perhaps 'attention' is the story of orchestrating the activation of subsets of such segments. 2d. If so, are the segments pre-determined, or do they develop at some point? 2e. Perhaps such segments exist but are not sharply delineated.


  1. TRN is activated sparsely and allows fine-grained control by cell assemblies spreading into it.

    (Perhaps the fact that layer 6 neurons innervate TRN speaks for 3.).

Contrast And Alternative

The relay nuclei of the Thalamus have inhibitory interneurons [I], which receive inputs from Cortex [layer 6]. Layer 6 cells also make branches into TRN and they make branches back to input neurons at layer 4 - quite the puzzle! Let's concentrate on the [layer-6] -> [I] part fro a moment.

     |   [Ac]  [Ac1]   ^   | Cortex
        |       |      |
        |       |      |
        |       |      |   Bc is an alternative to Ac
        |       |      |
        |       |      |
     |  |       |      [B]  |
     |  |       v           |
     | [A]|----[I1]     [B] |
     +----------------------+  , where I inhibits [A] and [B]
                                 [I1] - inhibits [A]


     |   [Ac]  [Ac2]     ^ | Cortex
        |       |
        |       |        |   Bc's support is being inhibited
        |       |
        |       |
        |       |        |
     |  |       [I2]---|[B] |
     |  [A]                 |
     |                  [B] |
     +----------------------+  , where I inhibits [A] and [B]
                                 [I2] - inhibits [B]

Ac1 - hypothetical meme inhibiting its own activator
Ac2 - (actual) meme inhibiting its alternatives

At first glance there is something strange in the light of memetics, this circuit violates the basic tenant activate your activators. It inhibits its activator!

This is not so if we assume that all activity is sparse [Valid assumption afaik].

Consider 2 memes now, being activated by some input neurons A. Both memes go and activate some random I inhibitory interneurons, which meme is more successful? It is the meme with the random wiring that makes it inhibit all its alternatives.

If there is only so much excitability to go around (threshold device, see above), the memes need to compete for well-connectedness. You only need to be the best-connected guy in the network, in relative terms, not absolute ones. (Consider cap-k as an inhibition-model).

Therefore, Inhibit your alternatives is the second basic memetic move, and reified in this cortico-thalamic arrangement.

Murray Sherman mentioned this, too:

There is a problem here…

Please imagine that this circuit is based on this [cortex cells innervating relay cells, and neighboring inh. interneurons].

And we have lots of cells that are doing this at a single-cell level. Whenever one of these cells is active, some relay cells will be excited, others will be inhibited.

However, the techniques we use to study this pathway are limited because we don't activate single cells and record from single cells. Instead, we excite or inhibit whole groups. Now if you look at the way this is organized, if you were to excite all of these [many in layer 6], even if you use things like channel rhodopsin, the result is that all of these [thalamic neurons] will be excited or inhibited.

… If you had pattern activation, you'd get a very different response down here.

This is an important proviso for most of the experiments being done. When we talk about knocking down a pathway, lesions or chemical ways of reducing all these cells [whole populations of cells all at once] and then even with modern techniques. We are not seeing the actual way this circuit is activating/behaving. And it's for that reason I think that it's very difficult I think to get clear evidence for what the effect of this pathway is on relay cells. Because all the experiments I know of with very few exceptions either excite or inhibit large groups of these cells.

And you wipe out the beautiful, detailed single-cell circuitry that really is playing the role.

M. Sherman about layer 6 cortico-thalamic circuits

If we activate some patch of cortex neurons, there is a problem. It is like making all the pistons go up and down together in a car, you don't see the actual arrangement of what this thing does in vivo so to say.

In this case, you will see the inhibitory neurons being activated, inhibiting Thalamus->Cortex inputs. But the important thing in this arrangement is the fine-grained flow of what the activity does, you cannot make a mass action circuit analysis and expect to not see a distorted picture.

You cannot look at a circuit without considering what happens when the activity is flowing inside it. And the activity is sparse.

Parallel Processing Flows

Let me bring back the inverted pizza slice:

         \             /
          \     A     /                   meaning-level
           \         /
            \       / <----->   B
             \     /   inibit
              \   /
                X                        sensor-level

A - Stable cell assembly

Let's map this onto thalamocortical circuitry:

                                                      ....     .... n-relay-layers
                                                   +--------+                       +-------+
                                                   | H    1 |  secondary cortical   | H   2 | , .... 30+
         -----------------                         +--------+                       +-------+
         \             /                           +--------+
          \     A     /           meaning-level    | k      |  second order relay
           \         /                             +--------+
            \       /                              +--------+
             \     /                               | H      |  primary cortical
              \   /                                +--------+
             --\-/------------                     +--------+
                X                 sensor-level     | k      |  first order
                                                   +--------+  relay nucleus

A - Stable cell assembly

k - low dimensional neuronal area
H - High dimensional neuronal area

(Speculative, as the software engineering approach dictates).

There is an entity one abstraction layer up from the neurons and the circuits: The cell assembly. A cell assembly can stretch meaning and sensor-level neuronal areas. We can say it is a data structure, created by the fundamental operation of pattern completion.

From biological reasoning [could have been otherwise, but consider the anatomy], it makes sense to locate this sensor level at the first-order relay nuclei of the thalamus.

Assuming that 1 thalamic relay nucleus can have multiple cortical area targets, the information flow would fan out. [Reasoning from the existence of ventral and dorsal streams, how could this be made except by splitting the paths somewhere?].

Considering that there is much more cortex that is not primary sensory areas, we can assume that there is roughly something like the inverted pizza size shape; If you map a cell assembly to neuronal areas, the 'meaning level' parts of it are necessarily larger than the sensor level parts.

Further, this circuit supports one of the fundamental notions that we get from memetics. Via the inhibitory interneurons of thalamic relay nuclei. (Described at Contrast And Alternative).

Inhibit your alternatives, on every level of meaning.

         ----------------- -- ---
         \       \     /        /
          \     A \   /  B         ....
           \         /        /    k
            \       /        /     H
             \     / \             k
              \   /   \    /       H
                XA ---| XB       thalamic relay nucleus
            A inhibits B on the sensor level

A - Stable cell assembly
B - A possible cell assembly, without enough support from the dynamic network to be active

You see from the cell assemblies clearly, whatever interpretation the system is expressing at the moment at large, i.e. which assemblies are stable. These will modulate and create contrast at the level of the sensor inputs coming in - down to first-order thalamic relay nuclei.

In this view, the thalamus is far from a simple relay. It is an essential component of the memetic landscape of the cortex. The memes in the cortex modify the inputs to the cortex. To fall into stable interpretations, which shape the inputs.

Why do they shape the inputs? To reproduce themselves.

The purpose of activity flow is to reproduce itself.

Parallelism and Multisensor Integration

An assembly calculus will integrate multisensor information immediately, just as well as 'uni' - modulatory inputs. All you need are areas that mix inputs from multiple input streams.

This is a very important requirement for brain software.

A Curious Arrangement

From Sherman:

  • driving inputs are glutaminergic excitatory synapses.
  • We can say there are first-order and higher-order Thalamic nuclei, they are defined by their driving inputs.
  • Every cortical area receives driving inputs from thalamic nuclei in Layer 4 (Neocortex).
  • An example of a higher-order nucleus is the sub-nuclei of the pulvinar, they go to the 30-something visual areas in the cortex []
  • A higher-order thalamic nucleus receives driving inputs from the Cortex itself, from layer 5 neurons.
  • Curiously, all input neurons to higher-order thalamic nuclei make branching axons. The other branch seems to go to motor* centers. Asterisk because this was preliminary. Perhaps it is 'lower targets', not 'motor targets'.
  • Why have this branching axon? Sherman speculates this might be the biological implementation of efference copies. Consider when moving one head, almost everything in the retina/sensors changes, but the brain somehow should keep track of this 'self-movement' and factor it out in its perception.
  • You can see this yourself by pushing gently on the side of your eyeball. The world moves a little. The fact that this is not the case when we move our eyes shows that some processing is canceling out the changes in the world coming from one own movement. In order to do such an information processing, efference copies were postulated very early.
  • Observe that the efference copy in this model is the one going into the higher-order nucleus. Since the other branch goes to motor centers.

      cortex 1                           cortex 2
      e.g. v1                            e.g V2

+----------+---+ 1-3               +---------+-------+ 1-3
+--------------+                   +-----------------+
|              |                   |                 |
| ^            | 4                 |        ^        | 4
+-+------------+                   +--------+--------+
| |          | | 5                 |        |       || 5
+-+----------+-+                   +--------+-------++
  |          |                              |       |        pattern repeats, up the higher order nuclei
  |          +---------------+ 5a)          |       +------------------------+
  |                          |              |                                |
+-+---------+                |    +---------+--+                             |
| |         |                |    |         |  |                             |
| |         |                +-+--+-->      |  |                             +------->
|           |                  |  |            |                             |
+-----------+                  |  +------------+                             |
   first-order relay nucleus   |   higher order relay nucleus                |
   e.g. LGN                    |   e.g. Pulvinar (1 of ~30 subnuclei)        |
                               |                                             |
                               |                                             |
                               |  5b)                                        |
                               |                                             |
                               |  motor* centers                             |
                               v  colliculus, red nucleus, etc.              v

5a) A layer VI neuron making inputs to a second order 'relay' nucleus, branch a.
5b) The branch b of the same neuron, going motor- or similar kinds of targets.

This is challenging to the conventional view, that cortical areas are somehow functionally specific. Here we have a circuit that implies that all cortex does at least a little bit of movement.33

This fits again with the view that the cortex is made from relatively generic, general-purpose elements. Additionally, such an element would affect animal behavior. So it would be immediately useful evolutionarily. The hypothetical proto-neocortex ('single area') would go to behavior targets, presumably already from layer 5. The input organization of this cortex would evolve together with the cortex. The logic of the efference copy would be a strong evolutionary driver for making the layer 5 output neurons branch, leaving collaterals in the input nucleus. Eventually, the targets of these collaterals would have an evolutionary driver for specialization, making a new nucleus. The whole organization would now have a reason to evolve a duplicated cortex element, which analyses the latent space, the output of the first area.

Something like that, perhaps the order is swapped somewhere.

What is the role of the cortico-cortical connections? Probably they are important, too.

From the lens of the cell assemblies and the memes:

To analyze a circuit from the view of memetics, we can imagine a few different 'possible memes', let them play the game of the circuit in our imagination, and see who has the most success.

Making the biologically plausible assumption:

All wires are kinds of wires.

The kinds of pre-and postsynaptic neurons, how you branch, to which nucleus you go and so forth are allowed to be determined darwinistially, but that the actual fine-grained connectivity is simply random (at least at the start).

Let's say we are some memes that are active in v1, looking directly at some sensor inputs. So the activation that makes us active is from a first-order nucleus, coming from the sensors.

The prime goal of the meme is to stay active. If it can bias the system to keep the sensors the same, it will do so.

From the circuit above there are 2 games these memes need to play:

  1. I spread into some layer 5 neurons, which make the appropriate motor output that keeps the sensors the same because this is where my activity comes from.
  2. I want to spread into the higher-order nucleus because this is my primary way of influencing the rest of the Cortex to keep me active. I.e. a meme that spreads to the higher guys and has friends up top will be selected compared to memes that don't do that. I.e. cell assemblies that stretch across cortical areas have more connectivity.

So 1 and 2 are two different memetic drivers, 2) makes you want to spread into as many layer 5 neurons as possible, but 1) gives this a twist: The behavior stream you are outputting shouldn't change the situation away from the situation that made you active in the first place.

The cell assemblies now have a memetic driver to output a stream of possible behavior (at layer 5 neurons). A stream of possible behavior that keeps them active.

You can look at a line in your visual field, and the cell assemblies in your primary visual areas make outputs at layer 5 that correspond to eye movements, staying on that line in the visual field.

Whoever is looking at the higher relay nucleus now (the efference copies of those movements), sees an encoding of things like lines, color blobs, edges and so forth. In the language of possible eye movements.

This would be to a low-dimensional latent space between the high-dimensional cortical areas.

It would mean that the latent space of the cortex is encoded in behavior streams, which keep the sensors/situation the same.

Let's imagine some red sensor memes in v1 when we are looking at an apple.

Now this circuit forces the memes to play with open cards in a way I think. You want to spread into the higher relay nuclei and influence the rest of the system, but the circuit forces you to make some motor output at the same time.

Imagine meme 1, which has a few random connections to the motor output, making the animal move its eyes for instance. This is not a better meme compared to meme 2, which makes the eyes stay in the same spot -> My sensors stay the same, so I stay active.

But meme 3 is even better, it spreads maximally into layer 5, and it outputs a range of movements; A range of movement that will keep the eyes inside the patch; Stopping exactly at the edge of the patch.

I am imagining the following experiment:

  • Make a monkey look at a red circle.
  • Somehow read out the activity coming from v1 to motor targets (the branching axon 5).
  • If you would run a decoder (or whatever you do), you would be able to read out a range of eye movements.
  • For a large circle there is either a correspondingly small or correspondingly large movement range signal, depending on whether this is encoding the freedom or restriction of movement.
  • Or, if it is a mix of such signals, you would have to decode more cleverly. Or perhaps setpoints are communicated, not procedural movements. Or a mix of those.

Alternatively, you would knock out all other motor pathways except v1; I think you would see eye movement. Small eye movement inside small patches of color, large eye movement for large patches of color. Line tracing movements for lines and so forth.

The memes that hit on the right connectivity to make movements that keep the sensors the same, will simply be selected in favor of the ones that don't. But the game is multilayered, the overall cortex context comes into play, too.

The memes that find the widest possible range of movement, which keeps the sensors/inputs the same, have a greater selection advantage still. Because it is those memes that spread most efficiently into the higher thalamic nuclei, and thereby have a chance to form cell assemblies with the higher cortical areas.

Ever since I came up with this idea, when I look at objects I wonder. Isn't there sort of a wall you hit when you move the eyes inside the edges of an object?

There is one other thing that comes to mind; Which is tactile mitgehen.

Alien hand syndrome Alien hand syndrome has been shown to be prevalent in roughly 60% of those people diagnosed with CBD.[6] This disorder involves the failure of an individual to control the movements of their hand, which results from the sensation that the limb is "foreign".[2] The movements of the alien limb are a reaction to external stimuli and do not occur sporadically or without stimulation. The presence of an alien limb has a distinct appearance in CBD, in which the diagnosed individual may have a "tactile mitgehen". This mitgehen [1] (German, meaning "to go with") is relatively specific to CBD, and involves the active following of an experimenter's hand by the subject's hand when both hands are in direct contact. Another, rarer form of alien hand syndrome has been noted in CBD, in which an individual's hand displays an avoidance response to external stimuli. Additionally, sensory impairment, revealed through limb numbness or the sensation of prickling, may also concurrently arise with alien hand syndrome, as both symptoms are indicative of cortical dysfunction. Like most of the movement disorders, alien hand syndrome also presents asymmetrically in those diagnosed with CBD.[7]

Corticobasal degeneration

Let's assume for a moment that in CBD the usual (frontal) motor pathways are impaired. Perhaps this is freeing up the memes of S1 and so forth, to make movement that makes the sensors stay the same.

If this is true, you would say that mitgehen comes from s1.

This idea doesn't depend on the branch b of the layer 5 neurons to go into motor targets. Could be 'chemical balance' or whatever nuclei, too.

I feel like cell assembly memetics can elucidate all mysteries of the circuits of neuroscience.

Consider the circuit, consider the memetic games of the circuit. Imagine a few alternative memes.

Memes know no boundaries, they will spread into whatever circuitry is available. If the circuitry forces memes to play a certain game, they will do so.

Claustrum speculations:

My reasoning: Claustrum looks like an element that implements some machine-level operation. It is orderly connected to everything else in the neocortex, stimulating it leads to loss of consciousness [Koubeissi et. al (2015] 34. It is at least interesting that this would overlap with what we expect from the hypothetical threshold device (further up). Stimulating it would either confuse this element to think that there is a lot of excitation, which needs a high threshold to keep in check. Or it would directly excite the inhibition neuron populations - In both situations, we would expect the activity of the cortex to drop. Perhaps leading to unconsciousness. But this is just speculation.

Further, there would be a class of epilepsy that would all be some kinds of claustrum pathologies. The opposite might also happen if you have too much activity in the claustrum - epilepsy in the claustrum. If the claustrum is the threshold device, this would lead to profound inhibition of Cortex.

It might be otherwise, but what if the mechanism of Cortical spreading depression is some kind of epilepsy in some inhibitory nucleus? Whatever that nucleus is, it is powerfully inhibiting the cortex. But what in neurobiology travels at 1.5-9.5mm/min? Some kind of offending activity, spreading somehow in a direction that is usually prohibited by the circuit perhaps. Perhaps something usually goes 1000 that fast, but in the other direction?


There is a literature on the 'microcircuits' of layer 5 output neurons.

…modulation of corticostriatal activity was shown to alter behavioral output in an auditory discrimination task (Znamenskiy and Zador, 2013), suggesting that layer 5 subnetworks may be differentially engaged by sensory inputs on the basis of current expectations or environmental demands.

Layer 6 cells


The Biology of Cell Assemblies / A New Kind of Biology

The cell assemblies then, provide a perspective on the structure and function of the cortex and its nuclei. (1 out of 12 required).

The cortical network is the meaning, but the ideas live inside the network.

Pieces of mutual excitatory sub-networks compete for activity sources. These can also be seen as stable data structures in a high-dimensional computing framework [Dabagia, Papadimitriou, Vempala 2022]21, representing the inputs.

The biology of the cell assemblies is a biology of hyper-dimensional, agenda-having, knowledge-representing entities, operating in subsecond timescales.

The basic tenants of this cell assembly memetics (what I have come up with so far):

  • A random directed graph with an idealized inhibition model, discrete time steps and Hebbian Plasticity [Hebb 1949, Kandel 1965] implements assembly calculus [Vempala 2022].
  • After the presentation of inputs to the network, a stable representation (called cell assembly) of the inputs forms rapidly.
  • (Note that we are allowed to require on-the-fly plasticity [from neurophysiology])
  • Cell assemblies are hyperdimensional: They are made of many small data points. Pattern completion is the basic operation of Cell Assemblies.
  • The timescale is sub-second. We only need a few neuron time steps to form stable cell assemblies. [per theorem, Vempala 2022]. (For igniting existing cell assemblies from partial inputs, I suspect you get a substantial part of the cell assemblies after a single timestep and the full one after very few. That is just from me hacking around with an assembly calculus model.)
  • Cell assemblies compete with their alternatives, with the best connected sub-assembly winning.
  • Assuming an idealized inhibition model, we can select the best connected cell assembly at each time step. [Braitenberg 1977]
  • Cell assemblies are subject to natural selection, cell assemblies are replicators with tiny agendas. I.e. assuming many possible connections initially, well-connected sub-networks compete for well-connectednes. In the context of the driving input activity. In other words, stable cell assemblies are the ones that exist. We can see that cell assemblies are replicators, replicating themselves across neuron timesteps. (But strategic cell assemblies exist, too. Biasing the rest of the system to be active again, somewhere in the future, not necessarily the next neuron time step). Note that gene-centered and extended phenotype views [Dawkins 1976, 1982] apply to memes, too. Giving us a large explanation lever on their structure and function.
  • Activate your activators is the basic memetic move. (The basic property of a cell assembly).
  • Memes are selfless: Memes will merge, given the chance. [from assembly calculus]. With assembly calculus, we can activate 2 cell assemblies together, and find an overlapping cell assembly. Another way to put this is that symbiosis is a primitive operation for the memes. Mutual activation is literally how the cell assemblies are formed in the first place.
  • Causality is the most fundamental concept, (see Hebbian rules, it is a 2 time step concept)
  • Association comes out of the rules by making the activity go back and forth.


  • Inhibit your alternatives is the second basic memetic move.
  • The meaning of a meme is the connectivity of its sub-networks. (This is only useful if the activity comes from the sensors).
  • The basic agenda of a meme is I will be thought again, or I shall stay active.
  • Memes don't know boundaries (memes spread into all wires available).
  • In principle, the cell assemblies don't care about some other part of the brain. They only care if they have to compete with somebody else that is exploiting that other part.
  • All wires are allowed to be 'kinds of wires'. The actual connectivity is allowed to be random, because in the second step, the memes will populate the network, and non-sensical wires will simply not be active.
  • All circuits in neuroscience can be understood in terms of the game of the circuit, the memetic games and the drivers a circuit creates for cell assemblies to play.
  • Because of the circuitry (see Contrast And Alternative on this page), whatever meme wins out fastest will go and inhibit its alternatives. Whatever meme is fast wins. (But perhaps during dreaming only parts of cognition are tried out, reducing the grip such a 'best meme' has on the system).
  • The computational paradigm of cortex is simply high-dimensional interpretation or representation, pattern complete, "template filling-the-blanks". For instance, v1 represents retinal inputs, and the real complicated computation was done by the retina. This get's us out of the question, why have a v1, when you can look at LGN? The answer would be to load the datastructures into the cortex. The flip side of this is that all activity might just as well be a representation of outcomes of computation. This throws a slightly different light on the idea that 'fusiform is recognizing faces'. Perhaps there was a lot of the system recognizing faces and fusiform is representing the outcome of this recognizing. (It is always semi-cringe to me when somebody says this and that is doing computation. When there is no theory of what the computation is, for all you know you are looking at a conveyor belt of information, of course you can decode some content out of that).

The Structure And Function of The Cell Assemblies Is Their ad-hoc Epistemology

Audio diary (stream of consciousness evolving ideas, better at 2x speed):

  1. Magic Gem Epistemology of Meaning-Level Wizards
  2. Musings On The Epistemology of Cell Assemblies
  3. Small Summary And Musings How Context Is Implementing Procedures In Assembly Calculus
  4. Speculations On The Mechanisms Of 'Good Ideas'

We can assume that evolution hit on a knowledge representing substrate. This substrate is populated memes - pieces of knowledge that replicate. This is only useful evolutionarily if this knowledge is useful.

It is fair to say that humans inhabit the cognitive niche (Pinker). Or something along the lines of the purpose of humans is to create knowledge (Deutsch).

The brain must support what I call an ad-hoc epistemology.

Alternative names: biological rationality layer, animal epistemology, living epistemology

Language Acquisition Is a Lens For Memetic Epistemology

In The Stuff of Thought (2007), Steven Pinker draws a very broad and deep picture of language and children's language acquisition. (also Hofstadter and Sander 2013, for a similar perspective).

Cell Assemblies Have Drivers For Generality and Abstraction

Cell assemblies want to be active. There is an obvious memetic driver for generality and abstraction, then.

Consider how children will over-generalize rules of language and pieces of knowledge. Pieces of knowledge are alive, they have an agenda. They want to reach as far as possible; Because if they do so, they are active more. And that is the 1 thing that memes want.

I was imagining high-meaning level memetic wizards. The wizards can use the magic of their matrix (see Memetic Engines Create Competence Hierarchies Up To User Illusions) to create little gems.

A gem is a piece of the network that receives more or less activity flow. (It's of course a cell assembly again). A gem will glow if the current context supports its activation.

If I have a piece of network that is supported by yellow and long-round shapes and the smell of bananas, that is allowed to become a gem, representing the abstract notion of a banana.

Higher-order memes will care about associating with useful gems. They will have an agenda for the gem to be as general and abstract as possible. To reach into many meaningful places.

The wizards have 2 new memetic drivers: 1: Make useful gems and 2: Make the gems you have more general.

A banana gem was glowing 3 to 4 times right now in your brain, and at least 1 wizard was happy about his gem glowing. That wizard will go out of its way to play the game of the matrix (virtual meme world) to grow a banana city of meaning. A city of meaning that makes associations with his gem.

Not because he knows what the gem means, not because he cares, but because it is a source of activity. And all activity that reproduces itself is the activity we see around.

At the user level of this meme world, some processes indulge in collecting many, beautiful, abstract gems, and giving them labels and pronunciations, the words of our languages. A gem is useful to the wizards that associate with it, and even more useful if it has a pronunciation.

They don't know it, but the gems contain knowledge. Knowledge that is represented in their connectivity to the network (the wizards don't know about that either).

A wizard simply sees a gem that is active sometimes. If the wizard receives activity flow from such a gem, he will have an agenda to make the gem active more. If that is a banana gem because it's connected to the taste of banana and the color of banana and so forth, then the wizard will have the agenda to make the abstract notion of a banana, which is active for all bananas. Wizard and gem are symbiotic memes.

In other words, it seems like there are memetic drivers on all levels of hierarchies to build Hofstadter meaning-cities bigger [2013].

The Computational Substrate of Cell Assemblies Makes Them Merge

The cell assemblies are strange. The substrate [assembly calculus], forces cell assemblies that are active together to merge. Creating a new cell assembly which is neither one nor the other. (At least temporarily, depending on the plasticity rules and so forth).

That is cell assemblies leave their identity behind to become something new. They are selfless in this way so to speak.

We can observe from replicator theory that the memes that merge in useful ways with other memes are the ones we see around. A memetic driver for merging with the right crowd, then.

There is no other replicator that I know that can do this; Except maybe ideas and thinking. Perhaps this is a hint that we are looking at the computational substrate of ideas.

Further, from the same logic; The cell assembly doesn't care about its neurons. It will simply leave neurons behind if that makes it have better connectivity. We see this a little in the logic of how cell assemblies form from input projections. It is not the initially activated neurons that form the cell assembly. These neurons are only a stepping stone for the activity to flow into its stable subnetwork.

This is another hint to me that considering the neurons is one abstraction layer too far down. A Darwinistic perspective takes the neuron as the selfish replicator agent is in principle viable, but less useful than the cell assemblies.

Context Is All You Need?

What is a hypothetical programming language on top of assembly calculus? What are the means of combination, what are the means of abstraction? [Sussman, Abelson 1984].


  • Which cell assemblies are active? The ones that are supported by the current activity flow in the network.
  • Assume for a moment are 2 main branches of activity flow top-down - the context, and bottom-up - the sensor inputs.

If the context is lines, edges, three-dimensional objects, or a few of them, then the system can 'go in harmony' with all the inputs it has. It is interesting to speculate how mechanisms like the thought pump above can steer neuronal activity into some 'useful' interpretation. An interpretation that 'fits well'.

There were these schemata from psychoanalysis. They are similar to the small situations of Hofstadter, too. The schemata are templates, which you use to build your psychology around or something. I think they are a good idea after all. We can make templates out of cell assemblies, they can be a piece of context and a procedure. It says 'pattern complete this'. This is what we can give the rest of the high-dimensional, parallel computing framework. The answer is then

  1. Whatever comes out after n neuron time steps


  1. Whatever comes out after n thought pump steps (oscillations)


  1. Whatever the system will stabilize with. Perhaps this state is reached quickly, never.

We can observe that there is a memetic driver for 'making the machine stop thinking'. Presumably by simply being well-connected enough so that even thought-pump steps don't change the set of active cell assemblies anymore.

This is the ideas getting into harmony with their context, their niches.

This is an extremely powerful idea because we make the means of abstraction in this programming framework the same kind of stuff as the fundamental data structure!

In other words, what we need to make arbitrarily interesting computations in this framework is simply situations, or templates, or contexts, or programs, or schemata.

The templates are represented by the same primitive data structure, the cell assembly.

So Cell assembly procedures. Those procedures should be parameterized, and I have a hunch how that is done. Since we are hyperdimensional, you can always express a mix of vague or I don't know yet information states. We can allow the notion unspecified until further data is gathered, but here is the direction of my hunch. These hunches, niches, and contexts, are allowed to be filled dynamically.

For instance, there would be a procedure that says I recognize objects in a scene. It doesn't say which objects, and it is good if it doesn't say. That is filled dynamically by some more sensor-driven parts. But it doesn't usually fill it with something else than a 'parsed science with objects', because that would not fit the context. (That would lead to major confusion I suppose, more below).

Something else is useful in this arrangement: The vague ideas are already useful from the beginning. This is a gradual information processing paradigm. A vague hunch I see x is useful already, even before details of x are supplanted.

Biologically, this is a very elegant design because now all the replicator theory/memetics of the cell assemblies would apply the procedures, too.

It is like having a programming language made from living play-dough. The substrate and its properties come out from the high dimensional computing framework the brain implements. If it is something like assembly calculus, then your fundamental operation is pattern complete. You get infinite malleability, just juxtapose and mix different information, and the computing framework will fall into an interpretation, which fits (a.k.a. it finds cell assemblies that have a lot of support from the currently active network).

The functions of brain software are simply memes, too. They are living procedural, parameterized, abstract data structures.

They want to be general, they want to be harmonious, and they want to make good languages, and good building blocks (see below). They are like clay, ready to be merged with other ideas, ready to be juxtaposed to get dynamic new stuff out. They want to be magical to use, this is how they play the memetic game of having friends higher up.

A computational framework where situations are the primitive means of abstraction.

I am imagining the high-meaning level wizards, putting together magic gems and playdough. The gems glow with activation, if they are supported by the sensor activation (thalamic input organization). The wizards that find general explanation structures, will be able to find powerful interpretations of the world. I.e. they will find meaning-level representations that are supported well by the sensors (Perhaps programming such a system would right now yield a mind on the spectrum, I'll talk more about the circuits that would support far-reaching imagination and superstition later).

Note that cell assemblies are allowed to have a temporal structure. The need for this is self-evident from considering how to represent motor movement. (More on such things from G. Palm 1982). In case you wonder if such a pattern complete can live up to rich, temporal sequences and so forth. In principle, that is covered by cell assemblies. Though perhaps there is more circuitry to build into one model. Perhaps some special areas or nuclei facilitate the temporal structures of cell assemblies.

I feel like I found Hofstadter's situations [Hofstadter and Sander 2013]; He went from the top 1 layer down and started talking about the cities of meaning. I am (attempting to) look from the bottom and here they are. Living networks of meaning with the agenda to grow.

Building Blocks / Harmony / Selfish Memes

From the selfish replicator view, there is a funny twist: They make harmonic teams, if they can. Dawkins explains this as the rower boat teams, it is those rowers that are useful together when thrown into a boat that are successful.

If the cell assemblies make ad-hoc epistemologies, then the ones that work well together, symbiotically so to say, are the ones being selected. This is a memetic driver for specialization, harmony, and elegance.

Elegance in the sense that if some memes make a good language, they have a memetic driver to be a good language. This is a deep topic of software engineering and design.

In other words, memes have a driver for making elegant ad-hoc languages, which are so easy to use and put together for the higher meaning-level memes in the system, that they are magical.

Confusion and speculative Socratic Wires

Socratic Wires, Champaign Bottle Memes, 1 idea about confusion mechanisms; Via absence detectors or null hypothesis element cell assemblies in intermediate meaning levels.

The absence of an explanation structure can be represented by cell assemblies, representing I know that I don't know. This is very useful to a system that (somehow) rewards good explanations.

  • Socratic sensors on intermediate meaning level

    Let's pretend we have a network laid out roughly like so, 1-3 are neuronal areas, or virtual neuronal areas (the layout would depend on the topology, these things).

    For some reason, if you have a network that represents sensors, then you make another network representing that representation, in turn, you get higher meaning representations [Rosenblatt].

        derived-meaning-level                   sensor-level
         +-----------+  +-----------+  +------------+
         |           |  |           |  |            |
         |           |  |           |  |            |
         |           |  |           |  |            |
         |   B       |  |           |  |            |
         |           |  |  M  M     |  |         A  |
         |    -------+--+>      <---+--+-------     |
         |           |  |   M       |  |            |
         |           |  |           |  |          ^ |
         +-----------+  +-----------+  +----------+-+
           3              2             1         |
                                            sensor activity
    B - high-meaning level cell assembly
    A - sensor-level cell assembly
    M - hypothetical intermediate-level cell assembly

    Case 1: There are some cell assemblies M active, they get support from both A and B, in effect forming a large cell assembly AMB, which spans all meaning levels. I speculate that such an arrangement corresponds to some kind of fitting explanation structure. By forming AMB, the system can predict its sensor input and so forth.

    Case 2: A and B do not "agree", their connectivity into 2 is not overlapping enough, and M is not ignited.

        derived-meaning-level                   sensor-level
         +-----------+  +-----------+  +------------+
         |           |  |           |  |            |
         |           |  |    ? <----+--+--------+   |
         |           |  |           |  |        |   |
         |   B       |  |           |  |        |   |
         |           |  |     ?     |  |         A  |
         |    -------+--+>          |  |            |
         |           |  |    ?      |  |            |
         |           |  |           |  |          ^ |
         +-----------+  +-----------+  +----------+-+
           3              2             1         |
                          M-box                   |
                                            sensor activity
    B - high-meaning level cell assembly
    A - sensor-level cell assembly
    ? - A and B try to ignite potential cell assemblies in 2, but there is not enough support

    I'll give 2 the name M-box, for 'middlebox' for a moment.

    The ? in M-box is interesting. It would represent the absence of an explanation structure that fits well from top to bottom.

    How to represent this absence in terms of cell assemblies again?

    Idea 1:
         |  0S  |
         |  ^   |
         |      |
         |      | null hypothesis nucleus

    Make a null hypothesis nucleus (presumably you put that element or instances of this element everywhere in the whole organization). If the null hypothesis nucleus is making an intermediate, random input to each area, then all cell assemblies have to compete with it. If the thresholds are balanced well, you would have the situation that A and B from above need to provide enough overlapping connectivity/activation, to win out agains the null hypoth

    The presence of the 0S cell assemblies in M-box is then allowed to represent the absence of a 'good explanation that fits everything'.

    Idea 2:
          M-box                   next cortical area
         +------+               +-------+
         |   ?  |               |       |
         |     -+ --            |   0S  |
         +------+   |           +-------+
                    v               |
               +--------+           |
               | _a ----+-----------+
               |        |
               | _a     |
               higher thalamic nucleus
    _a - absence detectors via inhibitory interneurons
    0S - Socratic cell assemblies /I know that I don't know/

    It is useful to use inhibitory interneurons to sprinkle in absence detectors, (Bratenberg talks about this in the biological notes on the Vehicles).

    One strategically useful place for this would be thalamic relay neurons again. Then everybody in the cortex can modulate relay neurons (via layer 6 circuits) and activate the absence of something else, too.

    If there exists a thalamic nucleus, which receives projections from M-box, then absence detectors in that hypothetical relay nucleus represent the absence of an explanation structure that fits everything.

    Whatever is activated then from this absence (0S), is allowed to represent the notion I know that I don't know. That is the basic notion of a Socratic wire.

    This model predicts that it is the 'intermediate meaning level' parts of the network that mediate "critical thinking".

    Now you might wonder if you would find 'confusion' neurons, perhaps everywhere, but perhaps especially in the parietal lobe, where the information processing flows seem to represent intermediate levels of analysis.

    Perhaps there is something about how mechanistic explanations, click the most? Perhaps it is real explanations, that stretch across all meaning levels.

    Perhaps there are m-boxes between everything. Perhaps Socratic cell assemblies are a generic signal to the system, that it is not 'done thinking'.

    What would be the m-boxes of social cognition? Something like person-mental-state analyzer pieces of the network perhaps? Funny, that TPJ is active during social cognition tasks, making the parietal look again like an m-box. Whether that means something I don't know.

    This idea in general says, you need middle-layer networks to understand, you don't understand.

    This maps onto things like hemispatial neglect; That would mean that the high-meaning level memes simply confabulate, that the signal of not knowing is simply gone. Presumably then because it comes from parietal middle layers of meaning (gone via stroke).

The Sensor Critics

How to get a rational [Deutsch] memetic environment? You need some critics [Minsky], that can say whether something is a good idea, or not.

One answer I have for this is that the activity flowing from the sensors can be the critics.

If your explanation predicts x, but there is no x in the sensors, this is where the system has the chance to revise the explanation.

Rational Memetic Drivers (they must exist)

These critics then should create a new kind of memetic environment, where actual, good explanations are better than bad explanations.

If everything is about predicting the sensor inputs [see literature on unsupervised learning] if predicting the sensors is a game that memes need to play; That is useful because it is hard to fake [Deutsch]. Explanation structures then have to represent something about the world in order to predict the sensors.

Perhaps the answer is already here: Make the activity come from the sensors and "Memes want to stay active". The memes will represent true things about the world, then. Because it is the memes that are connected well, in the context of the active sensor inputs.


Hofstadter analogies, these are again situations.

There is a memetic driver to use the existing explanation structure to explain new things.

Why? The same thing: Memes want to stay active. The ones that already exist will make clever reasons don't you see that is the same thing! I should be active and no further alternative should be searched!.

You might even go so far as to say this looks like analogy is the basic memetic epistemological move. And this all checks out because analogies are situations, which are allowed to be represented by cell assemblies again. And those are subject to the memetics of the cell assemblies in the system.

Symbiosis by reward narrative

(Getting Philosophical For a Moment)

I think there was/is a tension in neuroscience and computational neuroscience - that as a biologist you see a thinking machine, but what is the structure and function of this? The answer was it does computation. So now you say this thing does this computation and so forth. But this is like looking at a car engine and saying this does acceleration. I would say that you can replace computation with wizardry in a neuroscience textbook and you get the same information content from it.

What is missing is having a shape of the function in mind, because it is this shape, the rationale for what the structure should be, that allows us to satisfyingly explain biological systems. If the shape of the function is not perfused with an explanation model of what this should be, then you will always only see single pistons turning and gasoline flowing and so forth, you will see the trees but not the forest.

Now the cell assemblies are sort of a little bit 1 perspective on the intermediate level, the level of the language and primitives of the cognition software.

And this computationally intermediate plane is understood in terms of something living. The cell assemblies have a biology to them.

My philosophical view and agenda is to make the ideas come alive in our brains so to speak. Now these ideas have quite different evolutionary environments, landscapes, and pressures, they run on a computer, and they create software, that competes with other software creating agenda-having entities. They live on subsecond timescales in vast meaning spaces. Building an intuition around the properties of this world and its inhabitants is an aspect of explaining cognition in my view.

The mechanics and evolution of these little entities, their structure and function, are allowed to dissolve in our minds, the way that the structure and function of biological systems do.

But they are computational constructs, with the malleability to represent computation, knowledge creation, psychologies, societies of a kind and so forth. One perspective on them is how they construct building blocks, very chemical. Another perspective on them is how they make harmonious societies, very system-biology, ecology thinking and so forth. Another perspective is how they create knowledge, very epistemological, practical philosophical and so forth.

So from biological cybernetics we get this substrate, that is allowed to be put together so beautifully complicated.

This then is one perspective and candidate intermediate land and language between us and the neurons. This would consist of a computational framework (assembly calculus, thought pumps, hyperdimensional computing and so forth, something that says what kind of neuronal areas and nuclei you have, something that says what kind of attention mechanisms are available). The behavior of this system is described in terms of memetics. That is sort of alive, sort epistemological, sort of ecological, sort of software, sort of building-block making. It is psychological only in the sense of making sense of large-scale processes inside the system. The entities themselves don't have psychologies, they only have tiny agendas.

Magic Is Stuff, Not Illusions, Fixing Dennets Mistake

One of my main influences. And I think there is 1 thing wrong. Something political. That is the world illusion in illusionism and user illusion.

I think the problem is that people think this somehow means that consciousness doesn't exist.

This of course is not the view. Just like illusions are real, mouse cursors, and software are real, and magic tricks are real. Hallucination and illusion are very much real things.

What will satisfyingly explain the mind to me is when it is made from stuff. My agenda is making the mind out of stuff with mechanisms.

Illusionism is true, (see Memetic Engines Create Competence Hierarchies Up To User Illusions), but I like to call this magic interfaces instead.

This brain software of ours implements magic, I find this fucking amazing. The magic is made from living entities, infinitely malleable playdough that plays the games of the circuitry provided. Which supports epistemologies and has memetic drivers for finding explanations.

Anil Seth calls perception a controlled hallucination. Same thing. But overlaps a lot with my views here.


  • How big are cell assemblies in this view? Tiny, or is there 1 at each time step in a brain?

    I suppose the answer might be it depends on how you want to parse it, and both ideas are good sometimes.

  • How do cell assemblies relate to interpersonal (cultural) memes? (The ones from Dawkins and Blackmore).

    Cell assemblies are a brain computational construct, interpersonal memes are abstract replicators representing some knowledge of how to spread between brains.

    Both of these things are pieces of information that replicate and are subject to natural selection. Both of these can therefore represent useful information. This is interesting in light of the epistemology of the memes and memetic landscapes. (See David Deutsch's rational memes.) I think an analog of rational memes is the ad-hoc epistemology of the animal brain.

    It is intuitively obvious, how a piece of knowledge can spread interpersonally. Dawkins's tunes and habits of walking.

    It is required for such a meme to spread and be 'expressed' by the computer it spreads into. Any creative cognition machine is susceptible to such memes. (It must support it in the first place to count as a cognition machine in my book). This works, regardless of whether that cognition machine is implemented in terms of cell assemblies or something else.

    (But the landscape of available brains is the landscape of the social memes, so incorporating truths about certain animal implementation details of the brain you spread into makes sense. For instance, a meme might work by making you angry, but this same meme might not work on a Spok brain. In a world of Spoks, such a meme would be a bad meme. In the world of angry apes, it is.).

    The cell assemblies out of which a 'brain-phase' substrate of an abstract meme is made are different between persons (just to be clear common sense and neurophysiology apply).

    A social meme must navigate the messy biology and cognition of many brains in order to work.

    There are other classes of abstract, timeless memes: Knowledge about the regularities of the world, truths about epistemology and so forth.

    This gets super wild but:

    Consider a multiverse, the abstract memes which are more frequent, are the ones that are useful to brains. I.e. if you are a good meme, because you are a truth about navigating the world as an animal, you will 'spread into' evolutions. We can make this move because time is out of the way in this view. It makes sense then to say that evolutions and abstract memes stand in symbiotic relationships. This is cool because it puts us closer together again. If everybody had their brain software running, how are we the same? But if our evolution hits on how to make exactly this brain software, then we all have the same thing running again. (Of course, this software implements open-ended creativity, else it would not be cognition software).

  • What is the nature of the evolution of cell assemblies? Do they have variations, and the best variations are selected?

Why are there Jennifer Aniston neurons, when single-neuron encoding is logically excluded?[see Grandmother cell].

In the light of cell assemblies: The neurons are dynamically, potentially temporarily, allocated data structures. If a single neuron dies, the network will represent the same information with a different subset of neurons. Since this is high dimensional computing, parts of information are meaningful in the first place. And you never had the same symbol for your grandmother anyway (logically excluded from neurophysiology anyway since the network changes constantly). It is that you have many, many tiny sub-symbols of your grandmother, that contribute to the temporary conglomeration of tiny meanings, that make the symbol of your grandmother.35

Why do we think the orientation columns are innate in the monkey, but the auditory cortex of the ferret also makes orientation columns when wired with vision input? [Kanwisher for an overview].

The real question to ask: It looks like the memetics of the thalamus<->cortex produces orientation columns (on memetic timescales). Whatever is stable in the network comes from the arrangement of relay nuclei and cortex, and it is stable immediately. The network supports orientation memes somehow. And orientation memes somehow are the most stable memes in the network.

This is not yet a grand unifying theory of brain function. This is a simple first toy model to think further with.

Not answered:

  • Is anything essential left out? Neuromodulators, frequencies? Some nuclei?
  • What is the second hemisphere doing? Why symmetrical connections in the corpus callosum?
  • How do language and reasoning work?

More properties of cell assembly 'memetics'

  • The network you make is the network the memes live in. Suppose you make a nucleus that is hard to get spread into via certain criteria. You will select memes that are capable of spreading into this nucleus.
  • Suppose you give the memes the chance to inhibit their competitors by inhibiting their inputs, they will do so. (more on thalamus organization later).
  • Parallel Processing and multisensory integration is virtually free in an assembly calculus, (see my computational modeling of this coming soon). Since the system is representing its inputs, after extremely few steps (I guess 1-5), your 'association areas' will simply immediately represent the combination of inputs, too.
  • I sort of have 1 perspective (more required), but this sheds a different light on the nature of 'bottom-up' vs ' top-down' processing.
    1. There is an existing situation/context/interpretation set (the set of current cell assemblies) in the system.
    2. If a new input comes in, it simply is immediately part of the new context, In very few steps, we will determine a new interpretation set. (Find which cell assemblies fit the situation).
    3. This arrangement in principle doesn't differentiate between top-down and bottom-up. It is simply that the driving input of some part of the network comes from higher-order inputs (see the thalamic organization, Murray Shermans overview is great).
    4. This is a massively parallel arrangement, both so-called top-down and bottom-up pathways immediately represent an interpretation of the inputs.
    5. To get out of the conundrum of hallucinating top-down interpretations, I submit that a fundamental aspect of such a system is the topic of surprise, confusion, 'and dissonance'. I submit that some processes must decide on an interpretation to be a 'good fit', then you can stop thinking, otherwise you must consider the inputs again.
  • 'Psychologies' could implemented in terms of memetics, with an overall arrangement like this:

   +--------+                              +-------+
   |        |  higher order neuronal area  |       | sensor level neuronal area
   |        |                              |       |
   |        |                              |       |
   |        |                              |       |
   +--------+                              +-------+

  interpretation:                        interpretation:

  "I understand the world"               "I see objects in the scene"

                                                        1. sensor activity
                          +----+  driving inputs        2. "I understand"
                          |    +------------>
                          |    |
                      sensor input nucleus

This is easier if there are 2 kinds of inputs to each neuronal area:
1. The overall context
2. Specific inputs

(driving input is coined by Murray Sherman. Opposed to modulating, it is activation inputs that carry information and that are the main source of activity).36

The meaning level situation 'I understand' is the situation of the network. Only the memes, whose meaning fits the overall situation (well enough), are the memes we see. I.e. only the cell assemblies whose connections are supported by the network are selected.

This is just 1 tiny glimpse of a perspective. Still thinking about this.

This arrangement seems to overlap with Lateral neglect and the space of confabulation we observe from cognitive neuroscience. If my psychology memes say "I understand", the rest of the system will simply fill in the blanks to fit the situation. It seems like the capability to make useful sense of the world is disrupted, but the capability to be confident is not. (This is semi-obvious from the view of the meme action heroes above, confidence is an essential property of this mechanism).

The deeper question then is why are hemispatial neglect patients not confused about their failure to make sense of the world?

The answer, I believe, has to do with how confusion works, and how 'I can stop thinking now' works. (That I will try to explore further).

I can stop thinking falls halfway out of the memes already:

If I am a meme, my goal in some ways is to make the meme-engine stop thinking, since I want to stay active.

The Structure and Function of Cell Assemblies is their ad-hoc Epistemology


  • Supporting the concepts of alternatives is strong. It automatically supports reasoning by exclusion, so to speak.
  • If 'red' is an alternative to 'green', you can say 'red means not green', too.
  • One way to get this biologically, is with inhibitory interneurons. This way you can sprinkle in the notion that 'x is absent' into your input neurons. From assembly calculus, we will see that there are cell assemblies that represent both the presence of a thing and the absence of its alternatives. (For instance, if the presence of green always means the absence of red, too.)
  • Another way to get the notion of alternative is to give your memes the chance to inhibit each other
  • The basic meme agenda I shall stay active might already support object permanence (as the base problem).
  • From this, we can glean the notion that there might be ghostly, potential meanings, permanent objects of a kind, that represent the entities of explanation structures. For instance, if the sun is making things hot and I feel a hot stone, then I can have the explanation structure that the sun was shining. If I have some cell assemblies representing the notion that heat flows from the sun into objects, that is useful.
  • From this, I sort of glean an alive epistemology, kinda growing.
  • This is allowed to go memetically again, so you have many potential meanings and the ones that keep making sense simply survive.
  • Similar concepts to David Deutsch's rational memes apply. (More later). But on the layer of cognition, not the layer of sophisticated cognitive users guiding their knowledge-producing processes. Sort of an animal layer of rationality. I think this is the one that developmental cognitive neuroscience finds when they talk about how babies are scientists.
  • There is more going on when a toddler starts interacting with the world, I guess. Perhaps there is a flip to proactive reasoning and experimenting. (Explaining how you get this from your cognition machine is required for a model of cognition I would say).

It is interesting to note that there are no visual illusions that make you see red and green at the same time on an object. I think this could have been otherwise and is a clue: The structure of the network makes it impossible (afaik no illusion can do that).

But how do the higher parts of the circuits decide on what are alternatives? Why is a square an alternative to a star (presumably)? This is not clear to me right now. I suspect that memetics will have some answer that looks obvious in hindsight.

Abstract Replicator Theory

A replicator is a piece of knowledge that knows how to replicate itself.

Dawkins [1976] established the notion of abstract replicator theory; This is how he was able to predict computer viruses in the The Selfish Gene, in 1976.

The genes are only one kind of replicator, and the abstract theory of evolution applies to all replicators. Similar notion: universal Darwinism.

Turns out I am a lumper in this case. I would go so far as to say there is 1 kind of thing, an abstract knowledge container. That genes are only another kind of meme, too. It is some pattern that has a high frequency in the multiverse or something. A piece of knowledge that 'fits many pieces' of reality.

Speculations On Striatum, Behaviour Streams

This is bound to leave out 80% of what the striatum is doing.


Speculations On The Puzzle Of Tourette's, Musings On The Hypothesis Of Behaviour Encoded Situation Analysis


  • Input to the striatum is from 2 types of neurons in layer 5 of all cortex. [insert sources]
  • Striatum is implicated with 'generating' 'movement patterns'

    Other expressions of emotional states, such as smiling, also originate in the striatum and pallidum, and we are very sensitive to the emotional states these movements represent. ‘Faking’ a smile with conscious movements uses cortical motor control instead, and produces a result that looks false to others in most cases.


  • Striatum makes modulatory, inhibitory inputs to thalamic relay nuclei [M. Sherman]
  • Consequently, this doesn't look like the pathway carrying the information for how to move. It is much more parsimonious to consider the role of striatum as some kind of selector, filter, or orchestrator.

    (This is counter to the current mainstream thinking about striatum).

  • Striatum by default inhibits thalamic relay nuclei (via globus pallidus int.)
  • Parkinson's is a degeneration of Substantia Nigra Pars Compacta, which makes dopaminergic innervations to the dorsal striatum etc. Pars compacta
  • Sleeping Sickness,

    The disease attacks the brain, leaving some victims in a statue-like condition, speechless and motionless

  • Oliver Sacks in Awakenings (1973) talks about the sleeping sickness patients that were rejuvenated by L-DOPA. (Small documantary about this).
  • But this had the effect that the patients had compulsive behaviors.


  • Let's assume that layer 5 is the 'behavior output layer' for all Cortex.
  • Further, layer 5 behavior streams are the latent space encoding of higher-order thalamic relay nuclei (see A Curious Arrangement on this page)
  • There would be 'behavior streams' that simply participate in the information processing of analyzing the situation.
  • This would only be useful if the layer 5 neurons that go to thalamic relay nuclei are correlated with the ones going to the striatum. [which I don't know whether that is the case or not. Considering that column activity is correlated, the question is whether those neurons are co-located in columns, and that sounds likely to me].
  • Striatum is an element that listens to all "latent" behavior streams of Cortex.
  • What exactly is relayed at those thalamic nuclei? The answer is part of the puzzle of the striatum.

But here is sort of one idea that makes some general sense:

Say that the striatum is a selector, it looks at all latent behavior streams being expressed at the cortex and takes a value judgment from some Darwinian wires [Braitenberg 1984, also called Darwinian Brain]. And removes the brake from the streams that get the most approval. Perhaps it is selecting only a tight subset or a single one. This would solve Minskies 'pulled by 2 spirits' challenge.

Consider an animal that is roughly equally thirsty and hungry, it stands just between a water hole and a pile of hay. By the time it walks over to the hay, the thirst dominates and it goes over to the water. But by the time it is at the water, hunger dominates. You probably want to select one behavior and stick with that for a while.

     what striatum sees:

                  /--                     /-|
                /-                     /--  |
             /--                     /-     |
          /--                     /--       |
        /-                      /-          |
      +----------------------+--            |
      |                      |             /+
      |  +---+       +-----+ |           /-
      |  |   |       |     | |         /-
      |  +---+       +-----+ |        /
      |        +---+         |      /-   time flow
      |        |   |         |    /-
      |        +--++         |  /-
      |           |          |/-
                  behavior streams


                            cortex II
                       |              |
                       |              |
        cortex I       +----------+---+
     +---------------+ |          |XX |   layer 5
     |               | +----------+-+-+
     |               |            | |
     +----------+----+            v |      /----+-
     |          | XX |              +->  /-   /-|
     +----------+-+-++                 /-   /-  |  (pretend striatum takes all inputs into account
                  | |                +-----/    |   at once for simplicity right now)
                  | |                |     |    |
                  | +------------->  |     |    |
                  |                  |     |  /-
                  v                  |     | /     |
            +-----------+            +-----+/      |
            |           |             striatum     |
            |         1 |                          |
            +-----------+                          |
            Thalamus, relay nucleus    |-----------+ inhibition via GPi

            +-----------+                          |
            |           |  |-----------------------+
            |         2 |                      |
            +-----------+  |-------------------+   |
            +-----------+                          | n per nucleus?
            |           |  |-----------------------+
            |         3 |                          |
            +-----------+  |-----------------------+ n?
                ...                     |
             n-relay nuclei       behavior orchestration organization
                                  perhaps 'the piano of behavior'

XX - active layer 5 neurons, correlated, with targets in the thalamus and striatum

In this model:

  • GPi is like an array of gas pedals that are 'off' by default, 'the piano of behavior'.
  • The striatum is removing the brakes, orchestrating what behavior is expressed.
  • The striatum is a brain, concerned with selecting, switching, and committing to behaviors.
  • It would make sense if this selection takes a Darwinian value judgment into account.

    |      +---+   |
    |      |   |   |   possible behavior streams
    | +-+  +-+-+   |
    | +++    |     |
       |     |
       |     |
       |     +---------                               +---------| THA 1
              commitment assemblies?  --------------| +---------  THA 2
                                                      +---------| THA 3
            'commitment organisation' ?               +---------
                                             behavior orchestration
                 | value judgment  ^
                 |                 |
     +--------+  |                 |
     |        |  |                 | dopamine ++
     |    D   +--+                 |
     |        |                    |
     +--------+                    |
                               [VTA?, SN ]

D - Darwinian Brain
THA - Thalamic targets
VTA - Ventral tegmental area
SN - Substantia Nigra

Note the failure modes of this arrangement:

  • Switching commitment too much: Erratic behavior?
  • Switching commitment too little: Failing to multi-task?
  • Committing too little: Parkinson's? Sleeping sickness? Catatonia? Basal Dementia?
  • Committing too much: Overeating, violence
  • Selection too permissive: Ticks, Tourette's?

Why the tremor of Parkinson's? Perhaps something tries to offset the lack of commitment?

Why swearing and ticks with Tourette's? In the model, the striatum is too permissive. (But the striatum is not encoding behavior information, it only selects). The logical conclusion is that there are behavior streams that output 'swearing behavior', virtually constantly.

But why is this useful for the information processing the brain is doing? Who says 'Thank you this is the kind of information I needed to do my processing'?


  • Swearing is a more primitive form of language (see Pinker 2007).
  • From biological cybernetics: When wondering how to build language into Braitenberg vehicles. It is a useful first step, to make vehicles make sounds for the situations they are in.
  • (semi-wild, but parsimonous I think) An information stream representing the notion 'I am dissatisfied', could be encoded in terms of swearing behavior.
  • With the bonus hunch that the insular/cingulate cortex would represent dissatisfaction (hedonistically, socially,…), and so forth.
  • If the striatum is too permissive, it could accidentally commit to expressing this behavior.

Not because the person wants to swear, but because part of how we analyze situations includes an encoding of dissatisfaction, virtually always.

I believe this dissatisfaction is a deeply animal aspect of our psyche. It puts us as an animal in our situation. It is simply part of the whole arrangement. Part of what gives it its flavor is hooks for memory lines and so forth.

Why ticks?

Memetics says there exists a meme, which is partly made from the tick behavior. If the striatum is too permissive for some reason, the memetic landscape is changed. A meme can strategize to span across striatum and express behavior which will, in turn, activate its activators, and so forth in a happy behavior loop.

This shows us that the healthy memetic landscape probably has a bias to not make the same behavior over and over and things like that.

Internal Behaviours

Elegant, nice hack: Use the same organization to orchestrate internal behaviors, 'cognitive moves'. Stuff like having a train of thought, deciding to remember something, guiding one's imagination and so forth.

Why is there no name for this, what the hell?

Names? 'internal behaviors', 'thought moves', 'guided cognition', 'stuff like imagination', 'stuff like using working memory', 'internal muscles', 'muscles of the mind', 'mental avenues to take', 'though-muscles', 'mind movement', 'mind behaviors', 'internal storylines', 'narrative muscles', 'narrative moves', 'internal cognition', 'reflexive cognition', 'attention', 'stuff like remembering', 'moving one's inner eye', 'moving in thought space'

This then sheds light on basal dementia (Corticobasal degeneration). Roughly, if the striatum is gone but perhaps the gas pedals (GPi) are still 'off' by default. You stop being able to think and move.

It was always very dubious to frame the basal ganglia purely in terms of 'behavior' as in muscle movements.


  • Other input outputs of striatum?
  • If the medial temporal is a Darwinian brain, what is the relationship between the striatum and medial temporal?
  • What makes the tremors of Parkinson's?
  • What is the detailed circuitry of the striatum->thalamus?
  • What is the relationship between the cortex, thalamus, striatum and cerebellum?
  • What is the nature of the 'commitment organization'?
  • What is the nature of all elements I have left out, GPe, subthalamic nucleus and so forth?
  • Is there 1 organization of cortex-striatum-thalamus repeated across the whole neocortex?
  • What is the nature of nucleus accumbens and reward pathways?

A High-Level Story for Midterm Memory Retrieval

You don't know how you remember something - and you don't want to know.

Let's consider an alternative to the action-hero meme thought experiment for a moment. Imagine we implement a cognition machine software, where the fundamental data structure is a 'narrative unit'

|                   |
   narrative unit

In order to remember something, the cognitive machine can pull a move like this:

    (mix I-know-what-I-remembered random-noise))


 +----+--------+-----+  narrative unit vague

X - concrete "I start remembering"
O - unspecified random noise
RO - Semi specified "I remembered"

This narrative unit doesn't have much substance you might think. Isn't there so much random noise?

Vague states are great templates, they are context to the meme-machine.

So you are some competent meme that knows if there is a situation 'I start remembering'. Then you can go and activate some other dude that will provide some half-baked memories to the system, and so forth. You don't know how it works either, but you know some buttons you can try in order to make your cell assemblies go in harmony with the overall situation. And the overall situation is that the cognitive user is remembering something right now.

Perhaps the thought pumps will not settle until some satisfying connectivity is reached in the memes. (some details are still left out from my story here).

But something happens that makes the narrative unit go in harmony with the rest of the network and tada - you have a filled in memory:

 run meme-machine,


 +----+--------+-----+  narrative unit concrete

X - "I start remembering" (concrete)
C - filled in memory states (concrete)
R - 'I remembered' (concrete)

Funnily enough, I think that is the one you remember afterward, too.

So the cognitive user will only ever see this thing working crisply and clearly.37

What happens when it doesn't work? Here is a pet hypothesis on tip-of-the-tounge from this perspective:

 run meme-machine,


 +----+--------+-----+  narrative unit - very concrete

X  - concrete, "I start remembering"
OM - mix of vague states and failed attempts
RT  - 'I remembered', 'I speak',  concrete (prediction)

The 'I remembered' part never really goes into harmony with the rest of the narrative unit, because the meme machine failed to deliver the memory details. You start having very concrete story snippets about failed attempts and 'search process' information states. They all didn't help. Leaving you very unsatisfied. The release of a 'good idea that fits well' (see More Cell Assemblies Thoughts) is missing - presumably creating a frustrating feeling.

I think what happens now is that the machine usually also has some tricks for delivering. Perhaps make the whole narrative unit (or situation) more clear/salient/important/central in the mind. Presumably, you do this by activating all the cell assemblies with high frequency, and increasing the threshold so other memes are drowned out. (this is not clear to me yet and very interesting topic of thought-mechanisms). (Such things are labeled extremely broadly as 'attention' in cognitive neuroscience).

Or perhaps you take the narrative unit states, put them into the thalamus and make it the inputs to the rest of the mind, but I'm not sure yet. Perhaps that gives you a fresh perspective on things.

Either way, you sort of sharpen everything in the unit, including the RT part. That is the part that is right now 'in the future', where you have remembered and you speak, using your tongue muscles. Since you activate the whole thing you activate all the memes strongly here - including the memes that have to do with moving your tongue. You want this story to work out, in a high level of detail.

Perhaps this feeling also tells us something about how speech is represented in the mind in the first place. Something that has to do with moving the tongue, that is for sure. And meaning feels like it can 'sit on the tongue'. It's funny that it is the tip of the tongue.

The bottom line is with vague narrative units you can build a lot of cognition. (If you have a nice meme machine at hand, too).

The fundamental notion of the cell assemblies is 'pattern complete'. We can get the whole out of a part. This is useful, it makes vague ideas meaningful.

Perhaps then 'confabulation', 'filling the blanks', 'everything a little bit until something fits', and 'auto-completing to success' can be the fundamental operation of a cognition machine.

This is simply to wonder how far you get with this main notion in mind. (See The Software Engineering Approach. You need to consider what the building material you have in mind can already do in a wider context. So that you can see what it cannot do yet or where it doesn't fit at all.)

'Harmony' here means what kinds of activations are supported by the network. These could be described as the 'resonant' modes of the system.

Braitenberg had the idea to call these 'conflagrant modes' maybe. To separate it from physics.

I wrote this before thinking about some more derived topics on this page.

Contrast And Alternative gives a mechanism of how the system 'stops thinking'. The cell assemblies that manage to ignite in each neuronal area can inhibit their alternatives via the thalamic input organization. Thereby making the system stop thinking.

A signal that 'keep thinking' is required might come from Socratic wires, which can represent the absence of a 'fitting' interpretation. Or perhaps the absence of some 'fitting' situation makes the thought pump oscillate.

Also, all Cortex activity is input to the thalamus already. It would not be a special move, but it is the default architecture.

The Slow Place: Evolutionary Drivers For Mid-Term Memory

  1. Imagine some early cortex, simply representing some olfaction data. [Braitenberg 1977 mentions this, I guess he wasn't wrong. Also makes sense given that the olfaction cortex is the least derived cortex.]
  2. Now make some neurons extra sluggish, or make some Autapses (neuron simply activates itself),
    • make some slow neurons that store activity patterns longer than the other neurons.
  3. There is an evolutionary driver for 'memory' in biological cybernetics. Memory is fundamental to computation. Biologically intuitive: Memory must be one of the main reasons to have brains in the first place.
  4. If you have some slow neurons, the cell assemblies spreading into the slow neurons will live across longer periods. This way we have something in between short-term and mid-term memory.
  5. It makes sense we have evolutionary drivers for mid-term memory. For instance: Make a population of neurons, add things from 2.
  6. You have a problem now: The activity of staying around is great, but always remembering everything is useless. I.e. I don't want to remember constantly the banana I looked at 1h ago. So another driver to isolate the slow neurons.
  7. Now you have an evolutionary driver for storing and retrieving memories from that slow place.
    • A load/retrieve organisation.

You have another reason to isolate the slow place, that is epilepsy from all the activity.

The hippocampus has been called a flash drive. I will just take over this nomenclature and call this hypothetical slow place the flash drive (area) implementation.

Flash drive substrate

Hypothetical neuronal substrate that stores activity over longer time spans. The evolutionary driver for this is to keep cell assemblies alive for longer.

  • make many small neurons
  • slower neuron tick rate, because the activity just needs to stay around
  • make autapses or tight self-activating cell assemblies

Load, Store, Retrieve, Query, Result, Remember

From simple software engineering reasoning, we can be sure about some aspects of a flash-drive memory:

Data must be loaded, in a way that enables retrieval later on.

Data must be stored, frozen for all we care. For all we care that is allowed to be implemented by a USB stick (flash drive).

There must be a useful retrieval operation.

You don't want to remember all the things all the time, it is useful for the storage to be relatively isolated, and have some regulated way of saying 'Give me a memory… with criteria a,b,c …'

Retrieval can be further split into a query and a result.

Finally, the rest of the cognition machine can use the result for making a remembering situation. See A High-Level Story for Midterm Memory Retrieval for a cognitive-level idea about remembering.

I think it will always have to work like that if the data is not accessible in the network (long-term memory), then the network will have to load the data. The cognition machine and the flash drive stand in a similar relationship as the cognition machine and the sensors.

A-box flash-drive communication

It seems parsimonious to assume that the flash drive simply evolved as a sub-population of an existing conceptron (assembly calculus neuronal area).

Evolutionary early:


   |                  |  'online' cognition area
   |  +------+        |   ('fast' area)
   |  |    A |        |
   |  |  A   |        |
   |  |   A  |        |  slow sub population neurons
   |  +------+        |
         neuronal area whole

A - Cell assembly spreading across whole area


   reset the online cognition area, or new sensor inputs



   |         +---v--+ |
   |         |    B | |
   |   ^     +------+ |
   |   |              |
   |  ||  A  |        |  slow sub population neurons
   |  +------+        |
         neuronal area

B - Cell assembly with support form the sensors
A - The slow place part of A survives the reset,
    flash drive A part is now another input to the online area

Evolutionary derived:

|                  |    online cognition area
|                  |    (conceptron)
|                  |
|  |         ^     |
|  |         |     |
   |         |
   |         | Load and retrieve organization
   |         |
   |         |
|  v         |    |  flash drive
|                 |  (formerly slow neurons)

Note that online cognition area and flash drives are allowed to specialize. The online cognition area doesn't have a driver to implement mid-term memory on its own. We can expect this element to be a good 'fast' cognition area, then.

So 'automatic' load and retrieval would be the evolutionarily preserved operation.

The evolutionarily more derived specialized organization (where the flash drive is more isolated) has the job of regulating this load and retrieval.

It is like we have a substrate that is eager to spread. And our job is to regulate how it spreads. You might imagine this with water pressure and valves. But in some ways, it is closer to imagining this in terms of a wildfire. The wildfire just wants to spread into the available forest. This makes load and retrieve trivial, you only need some stretch of forest between 2 places and cell assemblies will spread.

We need some kind of network valve, then. To say what goes between the flash drive and the conception. Note that this valve is in principle the same kind of stuff again, a piece network that is ready to be populated by cell assemblies.

Here is a speculative idea, called association-box:


             |          | conceptron
             |          |
             |         -+-------------------+
             +----------+                   |
 +------------+  +---------------+  +-------+-----+
 |            |  |               |  |       |     |
 |            |  |               |  |       |     |
 |            |  |    A?         |  |       v     |
 |   F        |  |         <-----+--+--- Q  Q     |
 |     -------+--+-->            |  |        Q    |
 |            |  |               |  |             |
 |    F       |  |               |  |             |
 |            |  |               |  |             |
 |            |  |               |  |             |
 |            |  |               |  |             |
 +------------+  +---------------+  +-------------+

  flash drive     association box      'query area'
                  a-box                'q-wires'

Q - Query cell assemblies
F - Flash drive cell assembies
A? - Potential overlap cell assemblies of a-box

This is very similar to the m-box of the explanation structures above. There is some intermediate level between the 2 areas, and with enough support from both areas, you get cell assemblies stretching all 3. But without it, you don't.

Result case:

             |+---+     | conceptron
             ||   |     |
   result    ||^  |    -+-------------------+
             +++--+-----+                   |
               |                            |
               |                            |
               +-----+                      |
                     |                      |
                     |result-wire           |
                     |                      |
 +------------+  +---+-----------+  +-------+-----+
 |            |  |   |           |  |       |     |
 | +----------+--+---+-----------+--+-------+---+ |
 | |          |  |   |           |  |       v   | |
 | | F        |  |      A  <-----+--+--- Q  Q   | |
 | |   -------+--+-->            |  |        Q  | |
 | |          |  |               |  |           | |
 | +----------+--+---------------+--+-----------+ |
 |            |  |               |  |             |
 |            |  |               |  |             |
 |            |  |               |  |             |
 +------------+  +---------------+  +-------------+

  flash drive     association box      'query area'
                  a-box                'q-wire'

FAQ - A flash-drive-association-query spanning Asssembly

Whatever cell assemblies are on inside a-box are the results, so then it is a matter of having some a-box -> conceptron wiring to get the a-box results into the cognition machine (for remembering).

We see that would be useful for a-box to have a very tightly controlled inhibition.

  1. If a-box is active too much, flash drive activation would leak into the result wires. And presumably, there is so much flash-drive activation that this is a liability, potentially causing epilepsy. It is not surprising then that hippocampus and related structures are associated with epilepsy.
  2. A-box is useless if it is active too much. It would be equivalent to confabulating mid-term memories.
  3. A-box would be evolutionarily built to be some sort of valve.
  4. Note that there doesn't need to be that many special things about a-box, it should be a relatively generic citizen of the rest of the conceptron.

Having frequent resets of a-box perhaps, and regulating it very tightly, would be evolutionary drivers.

Note that these are high-dimensional spaces with sparse activity.

      |         |Sit|                                               |  conceptron (cortex)
                |   |
                |   |
      |         |FFF|                                               |  flash drive

Sit - Situation cell assemblies
FFF - Flash drive cell assemblies

In this drawing, we pretend we can collapse the high dimension into a single dimension. Also, we align them topographically. In reality, these are allowed to be distributed cell assemblies in the network.

How would load work with an a-box?

Perhaps it is allowed to be the same thing.


             |          | conceptron
             |          |
             |         -+-------------------+
             +----------+                   |
 +------------+  +---------------+  +-------+-----+
 |            |  |               |  |       |     |
 | +----------+--+---------------+--+-------+--+  |
 | |          |  |   A           |  |          |  |
 | | F      --+--+---->    <-----+--+--- S  S  |  |
 | |          |  |     A         |  |        S |  |
 | |          |  |               |  |          |  |
 | |  F    <--+--+-----          |  |          |  |
 | +----------+--+---------------+--+----------+  |
 |            |  |               |  |             |
 |            |  |               |  |             |
 +------------+  +---------------+  +-------------+

  flash drive     association box      situation area

FAS - A flash-drive-assication-box-situation cell assembly

If we have a hypothetical situation area, could span cell assemblies across flash drive, a-box and situation area.

Just one alternative, example:

Situation areas, part of conceptron.

                    | ^     ^  | conceptron
                    | v     v  | situation areas
 +------------+  +------v--------+
 |            |  |               |
 | +----------+--+-------------+ |
 | |          |  |   A         | |
 | | F      --+--+---->        | |
 | |          |  |     A       | |
 | |          |  |             | |
 | |  F    <--+--+-----        | |
 | +----------+--+-------------+ |
 |            |  |               |
 |            |  |               |
 +------------+  +---------------+

  flash drive     association box

FAS - A flash-drive-assication-box-situation cell assembly

This situation area would be useful if it represents a hyperdimensional point in 'time-space-conception' space.

As the situation of the animal changes, the situation area represents new cell assemblies, and the a-box activity moves forward.

The activity in a-box would open a relationship between flash drive and conceptron. And a-box would not differentiate between the current situation and a memory query.

This would mean that every time you retrieve from a flash drive, you have the chance to modify the memory in the flash drive. In the limit, this would mean you are re-creating the memory in a flash drive for each retrieval. Presumably, this depends on implementation details like the amount of plasticity in flash drives and so forth. This seems to map to empirical observations from cognitive neuroscience of memory, afaik.

Pointing to the fact that there is indeed some element in the system that doesn't differentiate between query and load.

Perhaps déjà vu is triggered by some 'situation' at activates a lot of flash-drive<->a-box cell assemblies? Then some 'result' wires are on, even though the system was not in query mode.

Perhaps this would flip some interpretations into thinking that a memory is being retrieved. But confusion, because no query was set?

The temporal extent of déjà vu might then perhaps be a hint about some temporal aspects of the information processing of this arrangement. Perhaps after a few seconds the 'time-situation' moves forward, and the familiarity with the situation moves forward, too.

                           +----  at t1, the situation moved forward
                  t0      t1
      |         |Sit|--> |Sit'|                                     |  conceptron (cortex)
                |   |    |    |
                |   |    |    |
      |         |FFF|    |F  F|                                     |  flash drive
                  +-- unusual large cell assembly

Sit - Situation cell assemblies
FFF - Flash drive cell assemblies

And the strange overlap of situation and memory would be over.

2-mode a-box

Some mode for retrieval and a second mode for load.

  • From the neuroscience of the hippocampus



Figure 5: Schematic showing regions of the hippocampus proper in relation to other structures.

The neuroanatomy of the hippocampus says 'I need to be isolated'. Curled up with the extra ventricle and so forth.

It is interesting that empircally, cell assemblies can be re-ignited with single neuron activations. [See Rafael Yuste presenting this work].

It would mean that single target k-lines [Minsky] might suffice as retrieval implementation.

That is, simply make a bunch of random wires go from flash-drive to conceptron. Even activating single neurons will already ignite cell assemblies.

Simple Predictor Loop

See also

  ^     |
  |     |
  |     v
  |    p-state
  |      |
  |      +------[C] Compare the last prediction with the new input
  |      |
  +----sensor-state [inputs-brain]

Different design decisions can be made here.

In general, the idea would be that you keep around a representation of what your predictor [P] says, then you compare [C] the incoming sensory data with the prediction.

If the prediction is good, fine everything moves forward. (or decide here to reward [P]).

If the prediction is off, you can immediately make a surprise signal, and make the system be about the inputs.

[inputs-brain] is something that merely represents what the sensors say. Everything looks like the thalamus is the place where this happens in the brain.

Further, you can have evolutionarily drivers to make this the ultimate input nucleus, after all, you can copy all 'Input' machinery. But this time the input is some other pieces of the brain, internal inputs then.

You can hook that up to a Darwinian brain, too. Which makes value judgments on what kinds of inputs are interesting and so forth.

The basic move is to treat the predictor as a black box, you reward it if it does what you want. You, the smart Darwinian wires, never know how this thing works. But it helps with survival and procreation for some reason.

joy lines

Basic mechanisms for rewarding memes/cell assemblies:

  1. Make them active longer (via plasticity they will entrench in your network and will stay around).
  2. Make them active at a higher frequency (same thing, plasticity rules).

| X            X           | predictor / thinking goo
| |     X      |           |
  |     |      |
--+-----+------+-----------------X-------------   j-lines
---------------------------------X-------------   [joy maker nucleus]
                               Darwinian approval [ yes / now]

Here is a simple reward scheme you could make for cell assemblies.

  1. Make some random wires through the thinking goo, call them j-lines for "joy lines".
  2. Joy maker nucleus selects a random subset of j-lines, call it j-mix, and activates. This will morph the memetic landscape of the thinking goo. Intuitively - if there are 2 roughly equally likely cell assemblies F and K, then if there is a random j-line going though K, but not F, it will be more stable and win.
  3. Now you can run your usual cognition machine (for instance in the simplest case a sensor-predictor loop).
  4. From a Darwinian wire criterion you get 'worked well' or 'not good'.
  5. If good, take the current j-mix and shoot activity up the wires. Presumably, you will reward all the cell assemblies still in short-term memory. (I.e. the cell assemblies your j-mix is activating right now will stay around more, because of our plasticity rules). Perhaps you could also feed the situation / sensory inputs that lead up to the approval, to get a richer memetic network. -> This is of course allowed to become arbitrarily complicated. -> Eventually, the memes in the thinking goo will become capable enough so they try to hack the Darwinian appproval for joy. -> That's a fundamental problem a meme engine has to deal with.

More generically, I call such a mechanism perspective-lines. Because of the way they change the memetic landscape (dynamically). This is almost the same as the so-called 'attention' of current ANNs. Abstractly, it is changing a meaning-landscape into a different shape. 'Perspective mechanism' would have been the less confusing label. Whatever.

A toy idea for a meme-pruning algorithm / A Candidate Dream Mechanism

This is probably wrong, maybe too complicated.

Consider something very similar to the above:

| X            X           | predictor / thinking goo
| |     X      |           |
  |     |      |
--+-----+------+-------------------------------     perspective-lines
------------------------------------------------    [ dream-nucleus ]

Let's say the system goes into dream mode, where it doesn't need to perform prediction/perception on the fly all the time.

There are many things you might be doing during dreaming. It is a basic move of explaining biological intelligence to consider that 1/3 of everything this thing does is sleep - so you can put as many mechanisms into a sleep mode as you put into a performing mode.

Here is a meme-pruning algorithm (just an idea):

The main idea is that you can try a subset of your thinking goo38. If it manages to predict the situation, then reward the smallest subset that manages to predict - automatically punishing selfish memes that don't contribute to the prediction/ success.

  1. In dream mode (no or low sensory inputs, no pressure to perform cognition).
  2. Select a random subset of perspective-lines p-mix, activate that subset.
  3. Select a known working sensor->predition pair (allowed to be more complicated).
  4. Feed the sensor states into thinking goo. (For the sake of argument, your p-mix and the sensor state you feed are the only sources of activation in the system now).
  5. Run your usual cognition,

    If the predictor comes with acceptable predictions, then reward the current p-mix. (Shoot activation up the wires). This way you punish all selfish memes that didn't contribute to the prediction capability of the thinking-goo. And you will get the kinds of memes that predict real things about the world.

  6. If the predictor is not capable of having a satisfactory prediction for your input states, either select a new p-mix, or augment the the current p-mix to p-mix-2 by adding more p-lines. Then continue with 5 until you get results.

Open questions:

  • Would you feed high-level prediction states and see what the continuations are?
  • Or would you be able to feed sensor data? It doesn't look like thalamic input nuclei like LGN are on during dreaming, speaking against this idea39


  • This fits well with the anecdotal and experimental observation that dreams from early and later in the night have a different character.
  • Incidentally, it seems like the earlier dreams have less cognition 'on'.
  • The dreams later in the night then, have a self and language and situations and so forth. This fits with the idea that whatever is happening, maybe it is trying out subsets of cognition.

Maybe this can be simplified into 'try out subsets of your cognition' and simply have the rest of the machine reward memes like usual. You get the same benefit - rewarding 'sufficient' memes, pruning away the free-rider useless memes.

Notes on The Agendas of Memes

Political memes

Large memes, very smart memes, that know how to use much of the computer to be successful - to be thought again. They spread into mid-term memory, they hijack where you look, they make you feel this or that way and so forth.

They represent 1:1 the neuroscience of 'the belief immune system' [Didn't find the research links right now, at least one researcher is using such terminology, quite sure].

Considering the logic of our inhibition model, where 1 cell assembly 'wins out':

         \             /
          \     A     /                   meaning-level
           \         /
            \       / <----->   B
             \     /   inibit
              \   /
                X                        sensor-level

A - Stable cell assembly

A Stands in competition with meme B:

   meme A                 meme B (competitor)

+-----------+     alternatives                       high meaning level
|           |   |-----------|
|       A   |
|           |
|     +-----+--+            +- -- - -+               medium meaning level
|  |  |     |  |                     |
+--+--+-----+  | |-------|  |
   |  |        |               b     |
   |  +--------+            +-- -- --+               sensor level
   |                +-----+
   +------------->  | c   |------|
    activate        +-----+

A good meme

  1. Activates its activators, that is how it is constructed in the first place.
  2. Activate the alternatives of competitor memes, on every level of meaning.

Meme A here will activate c, even though c might not have to do anything with A at first glance. This will happen if c is an alternative to whatever sub-cell assemblies would support B, an alternative to A.

If meme A doesn't make this move, then it is just not a good meme and is discarded.

The dress:


Figure 6: The dress blueblackwhitegold

If you don't know it, it either appears blue gold or blue black. Funny, it used to be white-gold for me back when I first saw it. Now it is blue black. And I cannot flip into seeing it white-gold, either.

I think what happens here is there is a further layer at play:

  1. sensor level (looks blue)
  2. meaning level (I see a blue dress)
  3. psychology level (I understand the scene)

3 Says that you don't randomly flip something fundamental as the color of a dress. It is a larger situation, that a higher order meme is creating for the rest of the meme-machine.

You successfully predict yourself to competently parse the scene without going crazy. And only the memes that are well-supported by the situation will survive.

Baldwian Spaces and Darwinian Wires

The general view on what the cortex is doing for an animal:

It represents an ever wider analysis of the situation and affordances; This is only useful when it is about survival.

In broad terms, there are 2 elements in the system:

|                                  |
|  ^                               | thinking goo (cortex)
+--+-------------------------------+ (makes memes possible)
   |                                 (large network, many possible meanings)
   |                                 (made from stupid stuff)
 --------------------  [Darwinian Wires]
 organize, drive, extinguish activity - shape the memetic environment
 reward the thinking-goo when it is useful for survival

                     (not many possible meanings, pre-allocated meanings)
                     (precise and competent, implements math when it needs to)
                     (has the intelligence of evolution inside it, is highly engineered)

The thinking goo is the ultimate Baldwin space, that must be shaped in a late binding.

This doesn't have sharp delineation, like much in biology. So when you look at the cortex you still see this or that wire and this or that architecture shaped by Darwin's hand. We can expect that wherever evolution could help out, for instance with development plans, it did so.

Still, I submit the fundamental dynamic nature of such a navigating device lies at hand.

The basic problem the 'rest of the brain' has to solve is to make the thinking goo be about survival and success.

My view is that there are 'smart Darwinian wires' that come from evolution, that are designed to make a thinking-goo into a good thinking-goo. That probably means things like rewarding and selecting memes, making memetic drivers in the system for success somehow and so forth.

One job of the computational cybernetician then is to describe and fabricate such memetic landscape machine tricks on one hand. (Similar to the ones that are implemented by the Darwinian Wires in the brain).

And on the other hand to make a computational model of a thinking-goo. It is not required, possible or desirable to get all the details exactly right the way the human brain does. But it is very useful to have a working implementation, if just for the benefit of knowing that it can work.

Note also that we don't care about the contents of the thinking-goo per se. We care about the kinds of content that it should support, so we can build a resourceful implementation of thinking-goo. (This is roughly what it means to have a dynamic computer program. 'Data is the ultimate late binding'. And so with the brain, the network and its content are the ultimate late binding).

In a further step, we can provide machine intelligence with the power of the computer. This is an edge that is not available to biological intelligences, that don't have the benefit of running on a computer. Such things are on the topic of intelligence mechanisms, which I would say are one level above the main concerns of a model of cognition. Roughly speaking, intelligence is the realm of stuff that a cognitive user knows herself; While cognition is the realm of stuff that the cognitive user has no idea how they work.

The Puzzle of The Hemispheres

I don't have answers yet.

It fell out of favor for a while to talk about lateralization, because popular science was taking it as a beautiful story of 'your other side knows different things' Bla Bla.

I think it is now that more nuance comes back into our conceptualizations, And there is a rich puzzle of of lateralization after all.

Givens / Reasoning:

Split brain patients [Grazzaniga].

  • It seems like the hemispheres only have some trickles of information flow left.
  • Perhaps with assembly calculus we can make a computational definition of split/merged cognition machines:

that is the extent of the (theoretical) maximal cell assembly of the network. If it spans the 2 hemispheres, we can roughly say there is 1 cognition software running. If it doesn't, then it is 2.

  • It is funny that 'alien hand syndrome' is conceptualized as a motor-system problem so to speak.

When in reality the reason for this is crazily profound.

  • I think it is almost obvious that there are 2 'selfs' and so forth in a split-brain patient.

(everything speaks for it, nothing speaks against it).

  • Split-brain is interesting, but it tells us how the system looks in an unnatural configuration.

What we learn from considering split-brain and lateral neglect and so forth is like shadows, which can tell us about the nature of the intact arrangement.

Thalamus is ipsilateral

[Lecture 8. The Thalamus Structure, Function and Dysfunction]

  • The thalamus is an ipsilateral concept. I.e. it looks like the input organization is separate.
  • If the Thalamus Matrix system provides context (perspective) to the Cortex, this organization would roughly correspond to some kind of '2 perspectives mechanism'.
  • Cortex is connected via Corpus Callosum (CC), at least roughly to the same region on the other side.

    CC connects functionally equivalent cortical areas in each hemisphere with the exception of primary sensory and motor cortices, which are sparsely connected transcallosally (Innocenti et al. 1995). The common understanding is that the sensorimotor cortices (S1, M1 and premotor cortex) are connected monosynaptically via the CC only for areas representing the body midline area (Innocenti et al. 1995).

    Functional anatomy of interhemispheric cortical connections in the human brain

    The prefrontal and temporoparietal visual areas are connected interhemispherically by poorly myelinated, small-diameter, slow-conducting fibres (Lamantia & Rakic, 1990; Aboitiz et al. 1992).

    (from here again)

    This is a challenge to my 'stupid wires' model. If we assume the cortex implements an ideal assembly calculus, there would not be a reason to have slow-conducting fibers.

    I would postulate that the presence of slow-conducting fibers only makes sense if the information processing has something to do with time delays. The computational reasoning for this is clear: There must be an evolutionary driver for making timeless calculations as fast as possible. (per definition, a timeless calculation doesn't concern itself with the time passing. See functional programming, What is a calculation?). The only reason to put delays into your information processing is that your computation has something to do with time.

    The other alternative is that somehow those fibers are on their evolutionary way out. Or that a slow conduction was sufficient, so there was no driver for myelination. (This would be rather surprising, why myelinate all the fibers in Cortex but then leave some out?).

  • With a Hebbian substrate and a thalamus, you could do the following:
  • Connect roughly to the homotopic side, at the area that receives the analogous thalamic relay inputs, with random wires:

          |   |                         |   |
      +---+---+-+                  +----+---+-+
      |   |   | |                  |    |   | |
      |   | ^   |                  |    | ^ | |
      |     |   |                  |      | | |
      +-----+---+                  +------+---+
            |  C                          |   C'
            |                             |
            |                             |
            |                             |
      +-----+--+                    +-----+--+
      |        |TH1                 |        | TH1'
      +--------+                    +--------+
      |        |                    |        |
      +--------+                    +--------+

TH1 - Thalamic relay nucleus, side 1
TH1' - Analogous relay nucleus, other side
C - Cortical area, recieving inputs from TH1
C' - Contralateral cortical area with inputs from TH1'
C<->C' - A few random wires 'interhemispherically homotipically'.

If this is the arrangment, you will observe that via Hebbian Plasticity, we will select wires that find some kind of symmetry in the information representation between the 2 thalamic relay inputs. Braitenberg 1977 was mentioning such symmetry detectors.

Simply imagine that from many random wires, the ones that connect to 2 pieces of network that is active on both sides at the same time, will be on. From then on, those symmetry-wires will pattern complete to the symetrical activation on both sides [Cell Assembly literature].

  • What is symmetry in derived meaning spaces?
  • The hunch is somehow that this would be a useful arrangment to 'share information' between hemispheres. (which must be the function of CC either way).
  • The 2 sides are functionally connected (see literature on functional connectivity networks).
  • Consider that organizations would have been possible:
  • Random wires between cortex hemispheres
  • Wire Thalamus -> contralateral hemisphere
  • Wire contralateral hemisphere -> Thalamus
  • Or wire contralateral hemisphere -> other subcortical structures.
  • The corpus callosum is curiously absent at v1. Exactly the area where we would have expected to gain use from 'symmetry detectors' [Braitenberg 1977]. (all primary sensory and motor cortices are sparsely connected transcallosally).
  • In general, this arrangement seems to mix aspects of converging information flow with separate information flow, but why?
  • Why decusssions?
  • Decussation seems to be a neocortical concept, but not enforced (audition).
  • What is the nature of hemispatial neglect?
  • What is the other 'Wernicke' doing? There is some literature on this. Iain McGilchrist mentions that the right side has a different vocabulary and so forth; So perhaps it is not true that only the left is doing language.
  • If the hemispheres implement some kind of '2 mutually complementary perspectives' mechanism, then what is the computational-level explanation of how and why?
  • Perhaps the best way to make neo-neo cortex is to make yet another artificial sphere, trihemispheric, or quadrihemispheric.
  • It would mean we have strong notions of what the basic circuits are, then we can simulate another symmetrical cortico-cortical connected element.

Remembering In Roughly Two Steps


  1. Long-term memory, mid-term memory, and short-term memory are different systems [cognitive neuroscience]
  2. Consider patient HM. Without hippocampus (+ close by structures) mid-term memory is gone, with anterograde amnesia, but long-term memories (up to a few years before surgery) are intact. Short-term memory is intact, too.

Step 0: open one mind to receiving mid-term memories. I.e. activate the situation of I remember. I.e. activate cell assemblies that support the idea of remembering vaguely.

Step 1: Reinstantiate cortex activity more or less globally*, basically giving some context. You need a hippocampus, or at least a related temporal lobe to pull this off [see patients HM and EP].

Let's speculate: what comes from the hippocampus is quite course-grained, then:

Step 2: Fill the low-detail information 'scaffold' that came back from the hippocampus with life. Do this by using the cortex, its thinking, its imagination and so forth to mentally enact the memory. It's my hunch that this process even makes you voice act the people in your memories.

I think all steps 0-2 are neuroscientifically plausible, and fit with a high-dimensional computing framework like the cell assemblies.

Remembering in this notion is separated from the memory. Note that this applies only to mid-term memory. Other kinds of memory are virtually guaranteed to have different kinds of mechanisms.

Remembering is the process that creates cognition states/ imagination states. And presumably, that happens in Cortex. The Memory is allowed to be stored in the hippocampus, frozen for all we care right now.

Déjà vu

Doesn't it feel like you are 'inside' a memory during a strong déjà vu?

My idea is that whatever is saying 'Here comes a memory, please fill in the blanks, Mr. Cortex (step 0)' is accidentally on. And the system falls into an interpretation that says I am enacting right now a memory.

I.e. Déjà vu is remembering being on, without a memory coming in, but the interpretation of cortex is for a moment that the sensor input is the memory. And the rememberer resources try to fill in the blanks.

Déjà vu is the interpretation of being inside a movie of one's mind a little bit.

Perhaps this is a hint about the circuitry. The relative rarity of déjà vu might give a hint that it is usually not the same input stream circuitry where memory 'context' comes from and where sensory input comes from. But it is also not impossible to interpret one place as the other.

This of course all fits with the idea that Cortex is so vast. That Cortex has a tiny amount of sensor inputs compared to its interpretation. So it is always the interpretation that makes cognition, perception, imagination, memory, and dreaming … They are created by the vast interpretations of the Cortex and its nuclei.

Of course, step 2 depends on how fine-grained the memory coming from the hippocampus is. Presumably ruminating on an emotionally salient event, perhaps every few minutes, such memory would become highly detailed.

A strong déjà vu is almost existential, it is a very intense 'mind-moment'. Perhaps only psychedelics and things like out-of-body experiences are stronger. For this reason, I am very skeptical about inducing artificial déjà vu in the lab.

I think what you measure is something like vague familiarity or something. At best, the notion is 'very low-level déjà vu' and at worst, it is utterly confused and doesn't have much to do with an actual déjà vu.

*) perhaps via the thalamic matrix system

Smart Wires Evolve and Thinking Goo is Useful Because it is Stupid

Exploring and communicating the properties and nature of high-dimensional computing is a fundamental aspect of explaining the cortex.

Darwin did not need to be smart in order to make a useful thinking substrate.

We know from insect neuronal plans and so forth very detailed math and models what some circuits are doing, for instance fly motion.

The 'smart wires world', Vehicle 3. Those are wires that are shaped by evolution, therefore the meaning is allowed to be very precise. It means 'you move when x is true' and so forth. It also implements some complicated math: subtractive inhibition, divisive inhibition and so forth. If one of these neurons or wires is gone, that is a big deal.

The smart wires world stands in contrast to the ultimate Baldwian substrate (cortex). Welcome to a world where the wires are stupid and don't matter, but from the many stupid ones you get something useful.

I submit that the cortex looks like it implements some kind of high-dimensional computing [Kanerva].40 Therefore in some ways, everything is still the same, evolution makes wires that make computation. However since the computation is of a certain flair, there are completely different requirements for the implementation.

The wires in thinking goo (high dimensional computing device) are allowed to be random, in fact, they are only useful because they are random.

The nature of this computation is like alien tech compared to the computers that we know.

As a biologist, I had the feature detectors in mind and thought about those neurons being connected. And of course, then you say it is so dauntingly big. This is when you look at the trees but you should look at the forest.

Because of the nature of high-dimensional computing, you can see the thinking-goo as a kind of information representation substance. A piece of cheese has temperature, you don't need to worry about the atoms. So too, a piece of the cortex has information states, you don't need to worry about the neurons.

How this works is a computer science topic. But trust me it works. Check Santhosh Vempala explaining assembly calculus.

This is the source of confusion when you hear that this or that aspect of the cortex is innate. How does that fit with the data of the ferret re-wiring experiment? You get orientation columns in the auditory cortex.

The confusion comes from the fact that both things are true. The substrate is random and it starts representing its inputs more or less immediately.41

It is the ultimate dynamic late-binding substrate (to use software engineering terms).

Perhaps the reason the cortex evolved from a tertiary olfaction ganglion is not so surprising in light of the high dimensionality. Perhaps it is the nature of information processing of olfaction, where you have 50 sensors, not 3 like in vision. Perhaps this is the fundamental nudge towards the concept of 'mixing information' that is perhaps the most primordial aspect of the cortex then.

Similarly, you look at neurons firing and you say here is a Jennifer Aniston neuron. The network seems so smart, but it cannot work with single neurons either because:

  1. If you put concepts into single neurons, you don't have enough neurons for the amount of concepts.
  2. Neurons die, and that does not seem to disturb the system much.

The high-dimensionality computation perspective flip is satisfying:

Concepts and symbols can be represented by randomly allocated subsets of neurons, that temporarily ignite each other as cell assemblies. It doesn't need to be the same neurons tomorrow and it probably isn't the exact neurons tomorrow.

The neurons are important and unimportant at the same time.

You can take a cell assembly and lower the eagerness to a small center of 'important' neurons. They are important right now but they are important in a dynamic way.

If those neurons had not been there, there would have been some others. And they would have represented the concept of your grandmother just as well.

The Brian as an Activity Management Device

From this, I have now grown a view of brain functioning that centers around the idea that the brain is an 'activity management device'.

You gate the activity coming in (Thalamus), you store some activity at a slow pace (Hippocampus), and you ignite activity internally (striatum?). And things like this.

Baldwian spaces make memetic substrates.

I want to move up from the neurons, like seeing the forest instead of the trees. Just like a stone can have a temperature, neuronal substances can represent information. So you can sort of imagine a magic stone substance that can be loaded up with information.

In general, it looks a bit like the cortex is a 'substrate that supports memes'. It can represent `vast` spaces of meaning, sort of everything possible.

|                                  |
|  ^                               | cortex 1 ('everything possible machine')
   | [brain nuclei]
 organize, drive, extinguish activity - shape the memetic environment

From AI we know that 'everything possible' is not useful unless I have something that selects and explores this space.

My main view then of what the rest of the brain is doing is that it is creating a memetic environment, in which only some memes (presumably useful for biological success), survive.

So you get this view of the brain shaping the memetic landscapes, to say what kinds of memes are possible. And the cortex fills the activation spaces42

Further speculation on the circuits of a holographic encoding

If I would know that thee TRN was segmented, then I would go like holy shit that fits.

This would fit with a circuit/mechanism where the thalamus would have slaps of roughly 1/2 * 105 neurons, each making wires to the cortex, going through a segment of TRN, with 1/2 * 105 wires going through it. Half, because the thalamus is a unilateral concept (1 hemisphere).

This would fit, if you would want to gate 1 holographic encoding at a time. Because at first glance it would be a bit strange to ever separate one encoding out. (This consideration depends on the idea that TRN is a kind of activation gating 'attention' mask around the thalamus).

This is just 1 of many things that could be true though. Perhaps the Thalamus isn't encoding a context at all. And if it does, then maybe it is useful to have 'partial attention', after all. (The brain would be more complicated).

If the thalamus would represent the cortex context in this way, you might expect that there is a hippocampus -> [thalamus-HD-encoder-nucleus] -> cortex circuit; And the basic midterm memory retrieval mechanism would be to reinstantiate the cortex context. (See below musings on hippocampus, future).

'thalamus-HD-encoder-nucleus' should be roughly 1/2 * 105 neurons large. It would be one of the Anterior nuclei, then. If the simple 1:1 reasoning of 'what is connected to what' can be applied here. (Which is not clear).

Or, there are slaps of 1/2 * 105 neurons in such a nucleus. Where 1 slap would correspond to 1 context context.

Intuitively, this might be incredibly sparse somehow. Let's say that a tiny subset of compartments active is already meaningful. Speaking against the idea that you would need to encode all 1/2 * 105 neurons.

Cerebellum doesn't matter for HD computing

I have seen this being mentioned the cerebellum might be doing HD computing. I am not sure entirely what the reasoning here is, here are some thoughts why this doesn't make much sense:

  1. The cerebellum from all angles doesn't look like it is part of the most interesting aspects of cognition. In particular, it looks like it is doing fine-grained timing calculations, required for movement (Braitenberg On The Texture of Brains). At the very least you have to concede that the current literature conceptualizes it as some kind of movement computation-making computer.
  2. Perhaps the regular structure of the cerebellar cortex invites a hypothesis about it doing hypervector math. The issue here is that the parallel fibers are thin. They look like they have an evolutionary pressure to be slow (see Braitenberg). This doesn't bode well for a model of the cerebellum where you want to do timeless calculations involving the parallel fibers. (Every fiber in the cortex that does computation seems to have a driver to be fast, see Pyramidal cell activity above).
  3. The whole reason to consider HD computing in the first place is to explain challenging modes of thinking: succession, causality, directional relationships, contrast, perspective, essence representations (symbol representation) and so forth. It does not make sense then to point to the cerebellum to implement this, since we know neuroanatomically, that the cerebellum is not essential for the most interesting cognitive processes (language, thought and so forth).

Olfaction is a stepping stone into high dimensionality

The alternative:

What does make sense is to consider olfaction and the computations it implements.

Vempala is mentioning this here as a stepping stone to think about 'assembly calculus'.

Perhaps it is simply the nature of making information processing on olfaction sensors that simply gives the system a nudge to high dimensionality. This then would give us the evolutionary driver for the cortex, and its HD computing implementation.

  1. Take some tertiary olfactory ganglion
  2. It represents olfaction information states, high dimensions
  3. Make a few neurons sluggish, or make some neurons go back and forth, or duplicate the whole array
  4. Now the input to the next time step of the computation is allowed to be not only the sensors but the current 'situation' that the network represents.
  5. From biological/cybernetic reasoning, we see the evolutionary driver for analyzing bigger and more fine-grained situations.

The evolutionary story of the cortex would then be summarized as sniffing with more and more context.

Cortex, evolution

It evolved as a tertiary olfaction ganglion [Braitenberg, 1977].

One might consider the perspective that all of cognition is ever more fine-grained and ever more abstracted sniffing.

There is more time between me and my problem in the past.

This requires

  1. Some memory capacity
  2. Sniffing out the essentials of a situation
   [ ate toxin ]  <--   me
|                 time            |
|                                 |
     [ relavant-sitution ]  <--+  situation sniffer

There is an obvious evolutionary driver to increase my sniffing capacity here.

 me -->  [ goal ]

 me -->  [ problem ]

Evolutionary driver to make the time arrow longer.

If I can sniff out the relevant situations in advance, that is useful. If I can put more time between myself and my goals and problems, that is useful.

There is a use for a situation sniffer, with an evolutionary driver. Towards being able to think further ahead, make plans, have a feel for a situation, have a feel for a situation in longer timescales etc. etc.

Consider a tertiary olfaction ganglion. It might start evolving some more neuronal tissue, to be able to keep more memory states around, and to then be able to have more derived states. For instance in higher abstraction hierarchies or higher temporal domains.

One of the things that a theory of the cortex is doing is talking about how it is useful to have more neuronal tissue.

If I can sniff out a problem before I have it, I am putting time between me and my problem.

If I encounter a problem and I can keep track of what happened in the past, I can put more time between me and the problem.

When we go into the cognitive realm (like tree shrews, primates, and dolphins?…) there is enough the machinery of the system so that we can talk not only about the time between and my problem but inferential steps between me and my problem.


Thank for the hypervectors, (Carin Meier, Kanerva and friends). The hypervisors are such a beautiful building material. Able to mix and morph and be vague and about themselves and have directions. All kinds of things that I feel are important for building material of cognition.

The bundle is like the `mnemotrix`.

Rotate + bundle is like `ergotrix`.

Bind is maybe like a synchronous activity. It is I feel freshly derived signals from 2 signals.

Of course, since the brain is a messy information mixer, firstly it does all of them at the same time. (Except preserving the bind for important information?). And secondly doing them all at the same time is good design.

Cell assemblies and hypervectors

Hunch: if you have static, timeless cell assemblies, their function can be abstracted with a hypervector.

So I would like an assembly->hypvervector projection.

Hunch: maybe some frequencies of activation are there to hold ideas in the mind, without modifying them so much. There is certainly use in the system for timeless ideas. (But the brain needs to make activity back and forth, so it can only slow down time at best).

But we, thinking about the abstract machine that we can run on a computer, have the freedom to freeze ideas.

… ideas:

To find a hypervector representation of a cell assembly A (the symbol you might say):

  1. Find a kernel of neurons that can re-ignite a cell assembly A (meaning condenser…?)
  2. Look at an arbitrary abstract space of neuronal area (with geometry, you could imagine a straight line) (of size h-dim). You need to remember this space though, call it a hyper-line.
  3. Project into a hypervector (trivial), say which neurons are active 1, inactive -1: [ -1, 1, -1, … ]

hyper-line is in effect a projection map between hyper dimensions and cell assembly neurons.

(hyper-line neuron-idx) -> hyper-dimension

In order to go back from hypervector space to cell assemblies:

  1. Given a hyper vector and a hyper-line
  2. activate the neurons with a 1 in the hypervec, (optionally inhibit the ones with a -1).
  3. This re-ignites the cell assembly, or at least biases the system to do so.

So in order for this to work nicely, you can now do the math, depending on the dimensionality of the vectors you want to use. -> this is 1:1 to the space of neurons that map.

hm not sure yet, tried to have some ideas below. Doesn't flesh out yet:

I don't need to calculate the neuronal net at each time step if the function of the neuronal net was timeless.

Idea: from a neuronal area, you can make neuronal units a hyper-line mechanism.

It would be kind of cool if you would get this with pure assembly calculus.

This here is kinda of the first idea:

    nueronal area                  |
                         [ A ]     o ..
 +--------------------+            |
 |                    |            |
-+---X--------X----X--+------------o 1 hyper-line (part of the cell assembly A)
 |                    |            |
 |                    |            |
 |                    |            |
 +--------------------+            o 2
                                 hyper-hyper-line etc.

You can implement hyperline with cell assemblies/neurons, I am sure.

It seems tempting to attribute some of such function to the straight axonal collaterals of pyramidal cells through the b-system.

Say we have an assembly A active, now we can find a hyper-line or set of hyper-lines that are well suited to re-ignite the assembly A.

Cognition Level Mechanisms

As programmers, we have a strong urge to get all the abstraction levels straight.

The program runs on our brain, the thing that makes things like perception, memory, working memory, language, higher reasoning, social reasoning, feeling and so forth. Is a program that is implemented by the brain, but not in terms of neuronal activation or anything.

But in terms of its fundamental data structures and so forth.

One way to get the abstraction barrier between the 2 levels right is to have 2 implementations in mind.

I have so far 2 or 3 ideas of implementations, against which somebody might try to program a cognition machine.

One is cell assemblies, one is vector symbolic computing, and one is imaginary magic books in a vast library.

Consider motion detectors: They must represent their meaning in terms of the data structures of the cognition machine. They cannot be little stacks of motion movies, they must speak the language of the cognition states.

It's a bit like the transistors of your computer fire with electricity, yes, and you can imagine the memes flowing and so forth. But in order for you to be able to delete a file on the disk, it must be represented as a structure in the operating system.

Gregorian Cognition:43

There is maybe roughly 1 other abstraction level in a cognitive machine, that is when you start having a sophisticated user of the the rest of cognition. Making things like working memory, higher goals, structured thought and so forth. It is quite striking that the prefrontal cortex seems to be doing these higher executive functions, and that the pre-frontal is biggest in humans compared to other mammals.

This is then firmly the layer of abstraction that we know ourselves. I know how I use my working memory for instance by 'keeping the information in mind', for a while or by repeating a piece of language or visuals in the 'mental scratch pad'.44

These are internal muscles of a kind that allow me to use the rest of the cognition machine to bring to mind a piece of information etc.

This mental scratch pad is available to me as a user. What is not available is one level below - what I call the Cognition Level. That is how perception works, how memory works etc. I have no idea how it works. For instance, I just experience my visual field and that's it. There is approximately zero insight, by the user, into where this visual field comes from.

As an aside on AI history:

It is kinda of a cliche story of AI, that the people tried to put these highly derived symbol thought things into computers. The narrative then usually goes something like 'and then we figured out that you need something messier than symbols' or something.

I think the real story is quite a bit more nuanced, with the dynamic programming Lisp people talking about programs that modify themself, and build societies of small, contributing agents (who have competence without comprehension, see Minsky 2006).

Not to forget there is the whole branch of 'biological cybernetics' that could have been, but wasn't, the basis of AI and computer science. See Heinz Von Foerster etc. They built the first 'parallel computer' in the 70's or something; Completely different paradigm of computing based on McChulloch-Pitts neurons. If we had more of these ideas influencing computer science, who knows what kind of alternative thinking substances we would already have?



My hunch is that if we extend the cortex with an artificial cortex, it would have dementia. Then you ask why does this cortex not work. The answer then is that there was a basic circuit that had to do with the striatum that was necessary for the cortex to work.


Guenther Palm has called a similar idea the 'survival algorithm'.


For instance, in Dennett's Consciousness Explained 1991; one of the central ideas is that the mind is a 'virtual machine' running on the computer of the brain.


Else, you would not be able to program it. - The wonderful focusing nature programming. The philosophy which needs to be right, otherwise it doesn't work.

Of course, I just call this cybernetics, blissfully content with the idea that our mechanistic understanding and the power of abstraction will conquer all problems.


That probably has then to do with doing useful information processing, which means doing highly abstracted information processing. Because in order to have success, you want to be fast. And being fast and higher abstractions are somehow pointing to the same underlying logic in nature or something.

Has to do with resource constraints, the power of computer programming languages and building blocks and these things somehow.


Also Marvin Minksy, 'The Emotion Machine' 2006 and 'The Society of Mind'


Speaking of evolved programs strongly brings to mind the wonderful field of ethology - See Timbergen, Dawkins, E.O Wilson etc.


See Braitenberg 1977 On the Texture of Brains -An Introduction to Neuroanatomy for the Cybernetically Minded for wonderful musings on such topics.

  • imperative programming: A machine with instructions
  • declarative programming: Timeless data transformations
  • logic programming: Relationships
  • actor-based systems: Little agents

And so forth

In some programming paradigms, you don't say up front what it does:

Chemical computing, genetic algorithms, and neuronal nets.


Software engineering is philosophy, on multiple levels so.

We need to come up with ways of making explanations.

For instance Stuart Halloway.

Gerald Sussman talks about this aspect of software engineering, too.

Is "Chicago" the 7-character string, or is the city in the territory? The quotation and the referent are different things. But the programmer will decide on this meaning.


This doesn't mean that we program a program in terms of such a language afterward. The program we build is allowed to be a development plan, not a detailed wiring diagram.


Yes, that is /Snowcrash/s metaverse.

And yes those landscapes/cities are Hofstadter meaning landscapes.


Insert list of links to computer science talks and podcast


See Braitenberg 1986 for a bit of solid reasoning. The bottom line is you have way more connections between the cortex than connections that go in.


Vehicle 12, Braitenberg Vehicles: Experiments in Synthetic Psychology


If you are a biological system, you the genes will be discarded by evolution if you don't make a machine that has to to with survival



Seems like the Tübinger Version of cell assemblies is the brain child of Braitenberg together with G. Palm.


There is another operation, analogous hypervector bind, that highlights the contrast between 2 ideas. I am not sure yet how to make that with cell assemblies. Maybe project into another area?


A higher time granularity can be achieved with slow fibers. That looks like what the cerebellum is about. See Braitenberg 1977.


That idea is from G. Palm Neuronal Assemblies


Modifying meaning spaces to what matters is what 'attention' mechanisms in ANN are, too. I call this perspective mechanism because I think that is a better name.


Unless I cheat and imagine saying it in different contexts, then it jumps between contexts


There is a wonderful rendition of something similar in Scrubs. I searched 'I am cooliam'. Only found the german version.

'Ich bin coolich'.

This is a related phenomenon, where the syllables in a repeated phrase will morph together in a new way, you don't separate out the meaning anymore in the same precise way.


The parallel nature of the computational paradigm being discussed here is I think some core notion.


I'm embarrased for the field of neuroscience that you can go through textbooks and lectures and nobody talks about this.

Maybe they all rush to 'do science' on cortex because cortex represents mental content, so you can always find neuroscience about some mental content and that always sounds cool? It's simply a mistake from my perspective. From my perspective, this looks as if you think you can explain a car by explaining the arrangements of the seats or something.


It is interesting to consider that eye movement is known to somehow come from whole cortex. This is challanging to a 'cortical modules' view. I enjoyed this Lecture by Pedro Pasik quite a bit: Lecture 6. The Oculomotor System I-II Structure, Function and Dysfunction


But using high dimensional computing we can also express the essences of things, not only a cloud of associations. More on this later.

It is not clear to me yet, but ideas:

  • increase the threshold, so only a tiny subset of neuronal units is active. Theoretically, you could go down to a single neuron, representing the center of centers so to say of a grandmother (temporarily allocated).
  • Lack of 'limbic activation' [Ramachandran 1997] is part of the construciton of Capgras delusion. The cell assembly interpretation at hand is that something in 'limbic' represents a 'one-time' allocated symbol of your grandmother. I.e. there is one cell assembly per person representing the people you know, in the 'limbic' cortex. If the connectivity of this cell assembly is broken, you will see a face etc. without the meaning of who it is. This looks then like Capgras in the making for me, you would have the experience that a person reminds you heavily of somebody you know (you still recognize faces, but not persons), but your cognition simply represents the meaning that this is a new person to you. Since the rest of cognition seems to fill in all the blanks (confabulation), it falls into the interpretation that this person is an imposter.
  • From this, it looks to me like I can get symbols from just assembly calculus.
  • Something else that looks like symbols would be a widely distributed encoding of cortex activity. Considering a holographic encoding of the cortex becomes juicy, see musing on HDV.

Here this overview is great: Murray Sherman


I think that by default it's super vague. It is that you use your imagination to fill in the states, but you usually don't realize it works like that.

So vague state from hippocampus -> cortex -> cortex uses imagination to fill in the blanks.

I think you even use your inner voice, like a voice actor, to make the people in the memory come alive, lol.

It is different if you have something sharp and emotionally salient happening. I think then you repeat the memory over and over, producing a sharp and detailed version. Perhaps you need to recall a midterm memory every few minutes, or every few hours or something in order to get something crisp.

(and everything in between of course).

This assumes sort of average midterm memory capability. Some super-rememberers are far from the end of some bell curve.


thinking goo is a synonym for 'probably something like cortex'. Probably something that does high-dimensional, parallel computing. Although I am biased here towards considering it's implementing a cell assembly calculus. (What Braitenberg called a Conceptron).


The time scales of this 'immediately' would be very interesting to know.

It would show us aspects of the assembly calculus implementation the brain uses. And it might constrain what we can think in terms of architecture (like this predictor loop and so forth).


Here there are 2 ideas I have:

World A: The eager activation world: If you make a neuro-darwnistic rule that only the neurons with the highest activation survive, you have shaped your cortex substance into an eager activator substance.

World B: The harmonious memes world: You figure out some memetic drivers that give you memes that are somehow in harmony and supplement each other.

Perhaps these are equivalent, but why exactly I cannot say right now. Maybe something with abstraction and hierarchies.

Perhaps it has to do with the actuator branch of the system, it needs to do things in the world to be useful. So the memetic landscapes also need to be shaped so the system does something useful in the world.


Gregorian comes from Dennett, he points out that there these mental technologies, tricks and so forth. Pieces of knowledge - memes, that allow us to think.

Some of these he calls 'intuition pumps', or 'necktop applets'.

Not sure which book is from, 'Bacteria from Bach and Back' probably talks about it.


See cognitive neuroscience: Wendy Suzuki's Lecture is cool.

Date: 2024-02-08 Thu 12:45

Email: Benjamin.Schwerdtner@gmail.com