Neuronal Ensemble Memetics

Table of Contents

You can see the nervous system as a big computing system. This was McCulloch's big step.

The cortex is a knowledge-mixing machine.

Evolving notes on cell ensembles, possible meme-machines and models of cognition.

A software engineering angle that wants to be biologically plausible.

This page is the philosophy behind Conceptron (eventually a formalism of a Cell assembly computational framework described here). And my toy world visualization things of this in the browser


Before we write the algorithm, we know what properties the algorithm has. What the desired output is for a given input. For an explanation generator, we precisely can't do that, because we don't know what the new explanation is before we have it.

So the task is different. What is needed to achieve that? I don't know.

We only know from the nature of universality that there exists such a computer program, but we don't know how to write it. Unfortunately, because of the prevalence of wrong theories of the mind and humans and explanations of the theory of knowledge and so on… all existing projects to try to solve this problem are in my view doomed. It's because they are using the wrong philosophy. We have to stop using the wrong philosophy, and then start using the right philosophy. Which I don't know what it is. That's the difficulty.

David Deutsch in a podcast talk.


I started out calling them Cell Assemblies because Braitenberg used that term from Hebb (1949). The idea is over 100 years old. Different people used different names. I learned from Rafeal Yuste1 that

  1. Sherrington termed the name ensemble before Lorente (who called them 'chains') and Hebb ('assemblies').
  2. Biological Ensembles are not Hebbian (they don't rely on Hebbian Plasticity). (See Alternative Plasticity Models)

This is why I start calling them Neuronal Ensembles now. And I don't bother with modifying all my notes here. (As is the nature of evolving ideas. The old cruft only changes when it is touched).

I will try to make it a philosophy of biological software, and see if that will work.

I imagine an alien spaceship with something like an alien high-tech cognition ball floating in the middle. The ball has an outer layer we call 'thinking-goo'. The are some smaller 'thinking balls' at its base, pulsating happily in its rhythms of information processing. The technology is advanced yet simple and beautiful. In hindsight, it will seem obvious how it works. Instead of explaining the brain in terms of the latest (primitive) technology, I want to explain the brain in terms of yet-to-discover ideas on the nature of cybernetic psychology. This way the stupid computers of our time only expand, and never limit, what I am thinking about.

"We Really Don't Know How to Compute!" - Gerald Sussman (2011).

What kinds of thoughts do we need to think in order to explain and build things like brain software?

Summary So Far

I would argue that how neuronal activity is translated in thought is to me the most important question of neuroscience. How do you go from a bunch of neurons being activated to mental - to mental cognition?

And how do you build a thought? What is a thought?

The critical level is the neural circuit level. The nervous system is built from molecules to synapses, to neurons, to circuits, to systems, to the whole brain… Of course, everything is important, but if you want to go for this question "What exactly is a thought?"

In my intuition that is the neuronal circuits where it's at. And you need modules. So these ensembles could be modules that could be used by the brain to symbolize things.

Rafael Yuste

Brain software can be understood in terms of a hyperdimensional computing framework, creating software-level entities made from neuronal activation that replicates across neuron timesteps. I argue these are the replicators (memes) [Dawkins, Dennett, Blackmore, Deutsch] of neuroscience. (see The Biology of Cell Assemblies / A New Kind of Biology)

This cell assembly memetics is software but understood in terms of a kind of biology. The cell assemblies are living entities, which need to have strategies for surviving, competing, and harmonizing. They have a structure and function of their own, which I call their 'ad-hoc epistemology'.

A network of neuronal units has meaning by being connected to sensors and motors [Braitenberg 1984, cybernetics, Stafford Beer The purpose of a system is what it does (POSIWID)]. If a subnetwork is connected in a vast hyperdimensional derived point in meaning space, then this is what it means.

Brain forms groups of connected neurons, or microcircuits [Schütz and Braitenberg, 2001; Shepherd, 2004]. They are also called ensembles, assemblies, cell assemblies or attractors and exhibit synchronous or correlated activity [Lorente de Nó 1938, Hebb 1949, Hopfiled 1982, Abeles 1991, Yuste 2015].

These cell assemblies form spontaneously, they emerge out of simple 'Hebbian substrate' models [Vempala]. They are a high-dimensional representation of the inputs, and can be seen as data structures in a computing framework, where the fundamental operation is pattern complete [Dabagia, Papadimitriou, Vempala 2022]2. (Although, recently Yuste and collaborators showed that Cortex might not use Hebbian Plasticity3, see Alternative Plasticity Models).

They are self-activating pieces of the subnetwork.

Note that correlated activity depends on the time window, too. We say neuronal ensembles have temporal structures (Braitenberg). Neuronal ensembles might represent mental content, thoughts, concepts, expectations, ideas, symbols and so forth.

Neuronal ensembles are compositional. They can be made from many sub-ensembles. In programming, we call that the means of combination, neuroscientists might like to call this (and are calling this4) the notes and melodies, harmonics of the symphony of the brain.

Yuste calls them the alphabet of the brain. We call them a datatype. He is not a computer scientist.

Santosh Vempala

We see that depending on the network, you can find a 'maximal neuronal ensemble', which would be an epileptic seizure if active in the brain, but shows a connectedness property of the network (and it is exactly the point of a corpus callosotomy to prevent the spread of epilepsy).

Conversely, we see that all network activity at any given moment in time is a subset of this maximal cell assembly. It is useful at times to consider the complete network activity as a single ensemble, which I shall call the complete neuronal ensemble (presumably, there are 2 of them split-brain patients, but depends on the time window, too).

That is, neuronal ensembles are made from multiple sub-ensembles. All neuronal ensembles can be considered to be sub-ensembles of a wider ensemble. That is a larger ensemble composed of more sub-ensembles and/or with a larger temporal structure. Up the complete ensemble of the network.

Cell ensembles can be connected in just the right way to the network to mean things corresponding to actual models of the world. To see this, imagine a few alternative sub-networks. Some will stay active, even though the sensors change. They will represent causal entities in the world.

Braitenberg musings The synaptic structure of the nerve net will approximate the causal structure of the environment:

Macrocosm and microcosm. Insufficient as this picture of the cortex may be, it is close to a philosophical paradigm, that of the order of the external world mirrored in the internal structure of the individual. If synapses are established between neurons as a consequence of their synchronous activation, the correlations between the external events represented by these neurons will be translated into correlations of their activity that will persist independently of further experience. The synaptic structure of the nerve net will approximate the causal structure of the environment. The image of the world in our brain is an expression that should not be understood in an all too pictorial, geometric sense. True, there are many regions in the brain where the coordinates of the nerve tissue directly represent the coordinates of some sensory space, as in the primary visual area of the cortex or the acoustic centers. But in many other instances, we should rather think of the image in the brain as a graph representing the transition probabilities between situations mirrored in the synaptic relations between neurons. Also, it is not my environment that is photographed in my brain, but the environment plus myself, with all my actions and perceptions smoothly integrated in my internal representation of the world. I can experience how fluid the border between myself and my environment is when I scratch the surface of a stone with a stick and localize the sensation of roughness in the tip of the stick, or when I have to localize my consciousness in the rear of my car in order to back it into a narrow parking space.

(The Common Sensorium: An Essay on the Cerebral Cortex, 1977).

From simple biological reasoning (to prevent epilepsy), this activity needs to be kept in check, presumably by some area-wise (or global) inhibition. (For instance by only allowing a fixed amount of neurons to be active at each time step). This allows for parallel search mechanisms, which will find the best-connected subnetwork, given the context. [Valentino Braitenberg 1977, Guenther Palm 1982]. (Also called threshold device, inhibition model, a hypothetical oscillatory scheme of this is called though-pump; See Tübinger Cell Assemblies, The Concept Of Good Ideas And Thought Pumps).

This inevitably leads to the view that this activation can survive, i.e. replicate across neuron timesteps. Hence, the cell assemblies can be analyzed in terms of abstract replicator theory [Darwin, Dawkins]. The simplest meme is activate everybody, producing epilepsy - a memetic problem. Note the simplest pessimistic meme is activate nobody. This meme will immediately die out and not get anywhere.

How does activation have strategies? By being connected in the right way to the rest of the network. By being connected in just the right way that is supported by the current interpretation of the network. Cell assemblies complete for 'well-connectedness'. In a network where the connections have meaning.

This yields a fundamental, high-level understanding of neuroscience, its activation flows and circuitry; Unifying brain-software with abstract replicator theory and hence biology. The purpose of activation flow is to replicate itself. It needs good strategies (free-floating rationals) [Dennett], in order to be stable across time. Being on is the only thing the memes care about, fundamentally.

The brain creates the substrate to support a second biological plane, the cell assemblies; With their software-logic biology. They will spread whatever neuronal tissue there is available. They will try to be stable and bias the system to be active again. For instance, they will simply spread into a flash drive neuronal tissue, when available. There is only one thing you need memes want to be active. If they can inhibit their competitors, they will do so. If they can activate their activators, they will do so. They don't care about some other neuronal tissue, they only care because their competitors might find excitation support from that other tissue. If they can make the system stop thinking, they will do so. They will lay association lines towards themselves if they can do so. More speculative ideas on some circuits: (A Curious Arrangement).

Whatever cell assemblies are active can contribute to cognition, the rest of the network is important only because it supports all possible activation. All activation is meaningful because it flows into motors ultimately. Humans with their explanation seem to challenge this idea at first glance, until you realize that even a person coming up with a really interesting internal state of ideas in a deprivation tank, must come out of that tank eventually to share their idea. Alternatively, we build new kinds of sensors and BCIs, which make it possible to broadcast one's ideas out of one's internal states. But then we have arguably crafted a new kind of effector.

We can expect that the Darwinistically grown part of the brain implements mechanisms that shape the memetic landscapes, dynamically so perhaps. The wiring of the brain must create selection pressures for memes that make them about being an animal in a world; Otherwise, a brain would not have been useful evolutionarily. There is no guarantee that this arrangement will work, and we can observe many failure modes; Superstition, false-believes, and so forth. What we can say is that the memes we see around are the memes that play the game of circuits well. Therefore the structure and function of the cell assemblies will be their ad-hoc epistemology; Their connectivity is the knowledge they contain, about how to replicate in the network. If the network is biased in the right ways, this knowledge will represent knowledge about the world and being an animal in it.

For instance, a cell assembly that is on for all visuals of bananas and the smell of bananas and so forth has a wonderful strategy. It is connected in the right way to banana things so that it is on for all bananas, it is allowed to represent the abstract notion of bananas. Not losing sight of the fact that this is biological (hence messy), that the ideas need to grow and so forth; Truly abstract, timeless concepts can only ever be represented as approximations.

This banana meme is a competitor in the game of the circuits. If it can bias the system to think of bananas more, it will do so. If it can bias the system to make explanations and stories, using bananas as an analogy, it will do so. Memes want to be implicated in as many interpretations as possible.

The cell assemblies have their own replication rules; For instance, they will merge (associate) with other memes, leaving their identity behind.

The interpretation game

Cell assemblies can be 'Situations', 'templates', 'schemata', 'expectations', and 'queries', providing context. The software does 'interpretation', 'pattern complete', 'filling the blanks', 'results', 'autocomplete' or 'expectation fulfilling'. 'Confabulation' and expectation are the fundamental operations in this software framework, then.

The means of abstraction [Sussman, Abelson 1984] in this software paradigm are 'situations'. Allowed to be small or big, stretch across time and so forth.

Believes and explanation structures can be represented as 'expectations', shaping which memes are active across sensor and meaning levels. Memes can be said to participate in the interpretation game, a series of best guesses of expectations, or explanation structures. For instance, the belief I see a blue-black dress, is allowed to be instantiated in the expectation, of seeing a blue dress.


Figure 1: The dress blueblackwhitegold

If there is a cell assembly that activates blue and is activated by blue in turn (it is the same cell assembly, stretching across meaning levels), if it is stabilizing itself by inhibiting its alternatives (see Contrast And Alternative). If it also makes the system stop moving forward in its interpretation, i.e. it makes the system fall into an attractor state, makes it stuck in the interpretation 'I see a blue dress'. If it also stabilizes inside the vast amount of network not directly influenced by the sensors, which outnumber the sensor areas by a factor of roughly 100 in Cortex (see Explaining Cortex and Its Nuclei is Explaining Cognition), it will be on, even though the user closes their eyes, and it will encode a position and an object via the expectation of what the user sees when they open their eyes again (see Getting A Visual Field From Selfish Memes). We might say 'The system believes that there is a blue dress'.

Cell assemblies are allowed to be temporarily allocated sub-programs, which are self-stabilizing and represent an interpretation. From the anatomy of the cortex, we see that motor areas, sensor areas, and 'association' areas are all wired in parallel (see Input Circuits And Latent Spaces / The Games of the Circuits), so what I call 'higher-meaning-level' really are equal participants in an ongoing 'situation analysis'.

In a joyful twist of reasoning, perception is created by the stuff that we don't see. Just as science is about the explanation structures that we don't see [Popper, Deutsch]. Perception is a set of expectations, each being supported by and supporting in turn sensor-level interpretations. Note that this situation analysis always includes the animal itself, in the world.

In order to get output you might say the system is predicting itself in the world, including its movement. Although below I label a similar concept commitment instead. (Speculations On Striatum, Behaviour Streams).

Memes want to be active. From this, we see there are memetic drivers for generality and abstraction, a meme that is on in many situations is good.

Children will overgeneralize the rules of language. Analogies are allowed to grow. Memes with pronunciation (words) will want to be pronounced more. Further, there are higher-order memes that will look for these general memes and have agendas for such memes to be general. (see Cell Assemblies Have Drivers For Generality and Abstraction). This is because a meme that is associated with successful memes is active more.

Some memes want to be good building blocks, they want to fit many situations, and they will be symbiotic with other such memes that 'fit well together'. For instance, the memes represent the physical world like objects, their surfaces, their weight, whether something can be put into them, whether they can be balanced, whether they melt in the sun and so forth are symbiotically selected by being part of larger memeplexes, representing the explanation structure of the physical world, I call such an ad-hoc abstract language of the physical world common sense. The best memes will play the interpretation game by composing abstract building block memes in elegant ways.

This is software engineering reasoning, we will see that abstract memes, and memes that are composable with other memes (memes that make good languages together), will be useful software entities.

Tip of the tongue might be a leaky abstraction of this eval-apply paradigm. The cognitive user creates the situation "I remember and then I speak the remembered thing". The remembering (query, result, load) machinery fails for some reason and the situation, the procedural template is made salient (presumably because this is a trick of the system that usually helps with remembering), since "I speak the remembered thing" is part of the template, it's saliency is a failure of the system to completely hide the details of the workings of this software paradigm. (What we call a leaky abstraction in programming). The cognitive user then feels like they would be able to move their tongue muscles, using the retrieved data, at any moment. In our ensemble paradigm, we assume that there are neuronal ensembles that represent the expectation structure "Retrieve something from memory and speak it".

(The memes will play the game of the circuits in order to reproduce. If the circuit is laid out in clever ways, a meme might be forced to play a different kind of game. See Input Circuits And Latent Spaces / The Games of the Circuits for some ideas on thalamocortical circuitry and what kinds of memes it produces).

Assuming a memetic landscape with bias on navigating an animal in the world;

Memes have drivers for using the computer they run on, without understanding the computer they run on for speed. Why speed? Because the first meme which makes the meme-machine stop thinking (Contrast And Alternative for a candidate mechanism), is good.

Memes might be said to participate in the interpretation game. That is, whatever meme is putting the brain into a stable interpretation simply wins (also called attractor states).

From top-down reasoning, a meme engine that creates a virtual simulated world and runs a high-dimensional computing framework should create user-level entities, which use magical interfaces to the rest of the computer. Why this is a mechanism to build magic interfaces, see Memetic Engines Create Competence Hierarchies Up To User Illusions.

In brief:

Consider the alternative, a meme that uses the computer clumsily is discarded. Similarly, a computer-level meme which is hard to use, is discarded. The overlap of optimism and confidence yields magic interfaces. Where the user-level entities are allowed to produce half-confabulated ideas, which are filled by the rest of the meme engine.

  1. Make a simulated world (one of the fundamental goals of this software).
  2. Try out everything a little bit (high dimensions and parallelism make this easy) all the time.
  3. Reward information flow which somehow has to do with navigating the world as an animal. I.e. using the motors smartly and having smart ideas about what the sensors mean.
  4. and 2. are a natural selection algorithm. First, all kinds of meanings are possible a little bit. In the second step, the meanings that were useful is left over (I.e. subnetworks that were connected in just the right way to mean something, for instance how to use the computer, for instance how to retrieve stuff from midterm memory, or how to pay attention to something).

    Note that this algorithm can only do what the computer can do. If the flash drive module of the computer is gone, this algorithm will not develop mid-term memory for instance. Consequently, user-level entities cannot dream themselves into great powers, they are constrained by what the computer can pull off.

  5. You will select high-meaning-level software entities, which are competent, fast and confident. They want to be wizards, they want to use the computer without knowing how the computer works.
  6. You will select low-level software entities, which are abstract, general and harmonious (a kind of building blocks, a kind of language). They want to be magical. Then they can be used by many other memes. Then they can be on.

I think there is a reason why we speak of ideas sometimes in biological terms the seed has been planted, the idea is budding. It is because the ideas are biological entities, they are replicators.

Brain software properties:

  • The computer we run on is fast (parallel) (something like 100ms to recognize a new object)

    Cell assemblies can find interpretations within a few neuron timesteps. This stuff is fast and parallel. Every theory of what neurons do needs to address this parallelism in my opinion (or not be a better idea than cell assemblies).

    Thought pump mechanisms can make a global, parallel search. Finding the best interpretation available to the system.

    Assembly calculus makes multi-sensory integration trivial. The same arrangement will represent the combination of sensor inputs just as well. (Given a neuronal area with multi-sensor inputs).

  • Brain software is used seamlessly (literally feels like magic).

    (Memetic Engines Create Competence Hierarchies Up To User Illusions)

  • Brain software supports feelings, hunches, and intuitions

    Cell Assemblies happily represent 'vague' information states, pointing in a general direction without details.

    The fundamental computation of cell assembly memetics is 'situation interpretation'. If there is a situation analysis, which stretches across a large situation, and is vague in some way, that looks like a hunch or intuition to me.

    What kind of circuitry is needed to make a 'long scale' situation analysis? Open questions.

    One piece of the puzzle will be the hippocampus, for sure: (See The Slow Place: Evolutionary Drivers For Mid-Term Memory).

    My current idea is that whatever the medial temporal lobe is doing, it seems to be part of grounding us as animals in the world. I.e. feelings, hunches, long-scale situation analysis, possibly credit assignment.

  • The stuff of ideas is infinitely malleable. It can be put together in vast amounts of ways.

    This is supported by a high-dimensional, dynamic computing framework. The leftover question is how to grow the knowledge inside such a framework.

  • A single piece of explanation can change the mind of a person forever. Like natural selection does it to biologists.

    The way that the Gene's eye view and the Extended Phenotype of Dawkins did it for me. Also called Socratic Caves. Whatever piece of explanation one has seen, it cannot be unseen. I.e. you don't go back to a cave.

    I don't know yet what brain-software is needed to support this.

  • A single instance of a piece of knowledge is sufficient to be used by the brain software in the future

    A mid-term memory is necessary for this to work [see cognitive neuroscience on patient HM.].

    Cell assemblies form after very few neuron timesteps [Vempala]. If the brain keeps some information states alive for a while, it can represent its inputs to itself, and form cell assemblies.

    In general, memes will want to spread into a flash drive (mid-term memory), if available.

    We can assume that brain software is using its midterm memory to represent situations to itself, even single instances of them. This way the brain can create explanation structures, being frugal with the amount of input needed.

    One might muse about how to build such a system, perhaps marking 'unresolved' memories, then replaying them over and over. Perhaps in a sleep mode, one could try out different kinds of perspectives and explanation contexts and so forth, until a causality structure, explaining the situation is found. From the memetic drivers of generality and abstraction, we observe that such internally represented causality structures, what you might call a mental model, will now want to be part of as many explanations as possible. In other words, there are memetic drivers for more fundamental explanation structures.

  • Children over-generalize language rules when they acquire language
  • Brain software can represent counterfactuals, hypotheticals and imagination states
  • Brain software can take on different perspectives, which can immediately "flip the interpretation" ("globally" ? )

    For instance when walking down a street in a new city and suddenly realizing one was walking in a different direction than one was thinking.

  • Brain software seamlessly switches between high-resolution details and low-resolution global views

    For instance when remembering a trip. Hofstadter paints the picture of seeing first the mountain tops of memory, the highlights and global views, then zooming in and lifting the fog in the valleys. 'Like this day of the trip…'. Now the perspective switches and more detailed memories come to mind.

  • Brian software creates a virtual simulated world, which represents the real world, the animal and its mind (internal affordances and so forth). It is useful to conceptualize this as the user interface of the brain.
  • Brian software supports decision-making and explanation-making software entities.
  • Brain software has a multi-year early developmental phase together with its hardware.
  • Brain software has a second mode of operation, called sleep and dreaming.

    Sleep is essential for brain health (Walker 2017 summarizes the field). An explanation of Brian's software will include satisfying accounts for what happens during sleeping and dreaming (presumably multiple things).

Why Neurophilosphy Is Epistemology

From David Deutsch The Beginning of Infinity. Socrates dreams of talking to a god, revealing a Popperian epistemology to him.

SOCRATES: But surely you are now asking me to believe in a sort of all-encompassing conjuring trick, resembling the fanciful notion that the whole of life is really a dream. For it would mean that the sensation of touching an object does not happen where we experience it happening, namely in the hand that touches, but in the mind - which I believe is located somewhere in the brain. So all my sensations of touch are located inside my skull, where in reality nothing can touch while I still live. And whenever I think I am seeing a vast, brilliantly illuminated landscape, all that I am really experiencing is likewise located entirely inside my skull, where in reality it is constantly dark!

HERMES: Is that so absurd? Where do you think all the sights and sounds of this dream are located?

SOCRATES: I accept that they are indeed in my mind. But that is my point: most dreams portray things that are simply not there in the external reality. To portray things that are there is surely impossible without some input that does not come from the mind but from those things themselves.

HERMES: Well reasoned, Socrates. But is that input needed in the source of your dream, or only in your ongoing criticism of it?

SOCRATES: You mean that we first guess what is there, and then - what? - we test our guesses against the input from our senses?


SOCRATES: I see. And then we hone our guesses, and then fashion the best ones into a sort of waking dream of reality.*

HERMES: Yes. A waking dream that corresponds to reality. But there is more. It is a dream of which you then gain control. You do that by controlling the corresponding aspects of the external reality.

SOCRATES: [Gasps.] It is a wonderfully unified theory, and consistent, as far as I can tell. But am I really to accept that I myself - the thinking being that I call ‘I’ - has no direct knowledge of the physical world at all, but can only receive arcane hints of it through flickers and shadows that happen to impinge on my eyes and other senses? And that what I experience as reality is never more than a waking dream, composed of conjectures originating from within myself?

HERMES: Do you have an alternative explanation?

SOCRATES: No! And the more I contemplate this one, the more delighted I become. (A sensation of which I should beware! Yet I am also persuaded.) Everyone knows that man is the paragon of animals. But if this epistemology you tell me is true, then we are infinitely more marvellous creatures than that. Here we sit, for ever imprisoned in the dark, almost-sealed cave of our skull, guessing. We weave stories of an outside world - worlds, actually: a physical world, a moral world, a world of abstract geometrical shapes, and so on - but we are not satisfied with merely weaving, nor with mere stories. We want true explanations. So we seek explanations that remain robust when we test them against those flickers and shadows, and against each other, and against criteria of logic and reasonableness and everything else we can think of. And when we can change them no more, we have understood some objective truth. And, as if that were not enough, what we understand we then control. It is like magic, only real. We are like gods!

Deutsch asks us to imagine trying to explain the behavior of champagne bottle corks in the fridge in the kitchen of the lab next to a SETI telescope, it is impossible to explain what these bottles do if you do not explain whether there are extraterrestrial life forms or not. Explaining human minds means explaining almost everything else, too.

As soon as knowledge is at play, it becomes the important thing to explain.

Having Ideas

There is quite a bit of a lack of what I call neurophilosophy in neuroscience. There is no model or theory that in some overarching way talks about what the relationship of the brain and the world is and so forth.

Perhaps the conventional view:


       |                       |
       |                       |  ...
       |   ..<-[]<--[ ]        |  3
       |             ^         |  2
       |             |         |
       |          +--+---+     |
       |          |      |     |  1
       |          +------+     |
       |             ^         |
       |             |         |
       +-------------+---------+   brain
                     +--------------------  sensor data

Information processing flow

  • Sensor data goes into the brain(or more precisely, the Cortex, which is seen as the 'real' brain).
  • A presumable feedforward cortical -> cortical 'information processing' is 'feature extracting' information from the sensor data.
  • Some theories exist of what this information processing could be (hierarchical 'grandmother neurons', 'ensemble encoding'), but I leave that for a future topic.
  • It is interesting to note that the literature on object recognition centers on single neuron analysis (even 'ensemble encoding' has implicitly the idea that sup-pieces of information are encoded in neurons).
  • At the same time, the rest of cognitive neuroscience focuses on how cortical areas are active and inactive.

David Deutsch would note that this is based on a flawed theory of knowledge. It is trying to induce knowledge of what happened. The same misconception is embedded in current machine learning [Why has AGI not been created yet?].

The world is big and our minds are small, we must speculate. And after the speculation has produced some candidate toy ideas, they have the chance to survive.

I'll just jump to my current interpretation:


     |  +----+        +----+ |
     |  |    |        |    | |
     |  |    | +----+ |    | |  meaning level
     |  |    | |    | |    | |
     |  | A  | | B  | |  C | |
     |  |    | |    | |    | |
     |         +----+ +----+ |  sensor level
                                sensor data

A, B, C - Expectation data structures

Ironically, perception, just like science, is about the stuff that we don't see. I think we see this in the overall anatomy of the cortex. There is much space for meaning, what lives in the meaning spaces?

The brain is not a receptacle of information. But an active constructor of the inner world. Surely, philosophers of old must have had the idea already. I am aware of Prediction processing, [Andy Clark], similarly the iterative improvements on guesses that make a controlled hallucination by Anil Seth (2021). I agree with the notion of making the top-down processes an active part of the system.

Predictive processing still has the notion of information processing flows, where some kind of top-down computers are now iteratively shaping the bottom layers or such. It does not come with a computational theory, as far as I can see, that says how the brain is making predictions.

The neuronal ensembles offer an updated view.

If neuronal ensembles are high-dimensional data structures, it is like there are puzzle pieces or perhaps space-ship amebas, made from activity. The activity represents information in the network. Like an active data structure. These are patterns completed so that a piece of information is useful. This opens the door to information mixing. And whatever the current activity, it is in turn the context for what can ignite. To explain what a data structure is, imagine that you load a file into your operating system, this is roughly the relationship of a program and its content. Observe that in order to have any useful power over the bits that make up the file, you want the operating system, to serve as a system of affordances, including being able to observe the system - for instance, navigating the file tree of the filesystem. This is the relationship of brain software to its data, too. Data must be loaded, and when it is, it is a representation of the data. And the ensembles are suited to fulfill exactly this role.

They are data structures that want to be stable (Cell assemblies and Memetic Landscapes), this might mean that they have dynamic parts or fizzle-ling parts of temporal structure.

If there is a spaceship of correlated activity that says here is a banana, and if it wants to be stable, for instance, stay alive even if you don't look at the banana. If it also represents an expectation structure when the eyes are at this position, then the sensor level represents such and such data. It is implementing object permanence for us already.

From this fragment of explanation I sort of see how this can represent the causal structures of the world, eventually. It would perhaps be useless, if not for the fact that Cell Assemblies Have Drivers For Generality and Abstraction. Whatever data structures represent the abstract notion of a room, a box, or container, or hole, a volume, a surface, or the stickiness of a surface. Such pieces of activity are more stable than others because they are useful across many situations. IIIa): The Living Software. The composability of those must work on the fly and be reliable, too. I must be able to say x is at position y, binding these 2 pieces of information. (Raw ideas: Getting A Visual Field From Selfish Memes, Creating Sync Activation).

There is something deep about the physical world I think, that makes the notion of stable information useful for describing and navigating the world. Why this is useful is perhaps a deeper problem of a field that unifies theoretical physics and epistemology.

There must be something about making elegant, "minimal and sufficient?" expectation structures, using systems of building blocks (languages). Whatever the substances, the wires, and the devices around them - they must create mechanisms that cannot help but grow into software that arranges these expectation structures on the fly, which I go ahead and label beliefs and explanations. (The Structure And Function of The Cell Assemblies Is Their ad-hoc Epistemology).

This is a very parallel mechanism. More like a kaleidoscope where all the elements update all at once - but many times per second. The dimensionality of the arrangement is reduced, just like in a Rubik's Cube the elements are forced to move with each other.

Since these stretch into "sensor levels" (Input Circuits And Latent Spaces / The Games of the Circuits), they implement an expectation about what the world is like.

Expectation instead of prediction because it is perfectly allowed to represent a counterfactual. If you look here, you will see x is a valid and useful expectation to have;

The user is offered the affordance. And this is useful whether the user looks, or not. My theory is that this collection of object ideas can be interpreted as a visual field. (They all say where and what to expect when you look at that where).

I don't think prediction is the right word. Becuase those expectations are counterfactual. It is what you could look at, not what you will look at. And I suspect it is allowed to encode a range of such expectations, too (Because something want's to be as stable as possible).

Another update, taking Murray Sherman and the thalamus people seriously:

                                 meaning level
     |  +----+        +----+ |
     |  |    |        |    | | ....
     |  |    | +----+ |    | |<------ affordances
     |  |    | |    | |    | |<------ interpretations
     |  | A  | | B  | |  C | |<------ affordances
     |  |    | |    | |    | |<------ interpretations
     |         +----+ +----+ |  sensor level
                                sensor data

A, B, C - Expectation data structures

The layout of the circuits puts motor output inside the loops. That is because the higher thalamic nuclei get inputs from layer 5 pyramidal neurons, which all branch. They make one output to motor centers, and one output to the higher thalamic nuclei.

This arrangement is strange at first, but evolutionarily satisfying, along those lines:

  • A generic neo-cortex module evolved with thalamic inputs and motor outputs, thereby being immediately useful.
  • The logic of the efference copy, which ethologists of old have predicted, is a strong evolutionary driver for branching the axon, back to the input nucleus, this efference copy was then input to the system again.
  • Since there was a new kind of input to the system now, there was an evolutionary driver to duplicate the whole arrangement, into a higher-order input nucleus and a higher-order cortical region.
  • And the input to the higher stuff is motor command data, hence the latent space of the cortex is a motor output encoding.

This is a drastic update from the current conventional view, where we have only 1 motor output module, the motor cortex. But in the updated view, all cortex is contributing to behavior. Also, the sensory cortex might be relatively generic, instead of areas being specialized for certain functions, the new view suggests a generic computational module that is repeated (and perhaps secondarily has some specialization). And all these modules are capable of making motor outputs.

The (neuro-) philosophy of Constructivism [Heinz Von Foerster, Humberto Maturana] had this idea that the mind creates the world. That perception is actively created by the mind. I think they also said that perception and action are intertwined, that one cannot see without moving the eyes and so forth. And I think it turns out that they were right. Some Memetics On Eye Movement And Maybe How That Says Where. The sensors are not a source of truth, the sensors are a resource to an explanation making system. In my view, the ideas are growing. They are biological entities (The Biology of Cell Assemblies / A New Kind of Biology)

Some strands of thinking in this direction then go on to say that we all have our own world and reality. Fallabilism is the notion that we are never right, that we can always find a deeper explanation. This also means that the truths are out there, for us to converge upon.

Since the trans-thalamic messages are all forced to make motor output at the same time, (because the layer 5 output neurons make branching axons), one layer's motor output is the input to the next layer. This is a 'poly computing' [Bongard, Levin 2022] implementation. Where the computation of one element is re-interpreted from the 'perspective?' of a new element. This arrangement mirrors the logic of an Exapatation in evolutionary theory.

(1) A character, previously shaped by natural selection for a particular function (an adaptation), is coopted for a new use—cooptation. (2) A character whose origin cannot be ascribed to the direct action of natural selection (a nonaptation), is coopted for a current use—cooptation. (Gould and Vrba 1982, Table 1)

What is a range of action from one perspective, is input data from the other perspective.

Why the range of action? Because the ensembles stretching across those circuits have memetic drivers to be stable, if they have the chance to spread to higher areas, they will want to do so. If the circuits force them to output motor data, they will have to play a game that balances their conflicting effects. If the animal moves, the situation will change, and if the ensembles represent the situation but also want to be stable, they will have to play the game of finding the smallest set of behaviors that keeps them stable.

Imagine looking at a line, the ensemble on the first layer says move your eyes like this (straight line). Because it wants the sensors to stay the same. This eye movement data is input to the second layer, which says I interpret this data as 1 line, color filled in and so forth. (something like that). Where does the color come from? From whatever color ensembles are active right now. Because the system is looking at a position, and has a certain shape/color interpretation, all these ensembles are active together and either compose via some phase-reset and synchronous activation mechanism or associate via on-the-fly plasticity. Both of these we can biologically plausbily require.

What would this mean for thought-level mental content? Perhaps it means that for instance, the affordances represent a range of thought narratives, which will incorporate their ideas or something.

Circuits and devices (like the Braitenberg thought pump) can make the ensembles play different kinds of games.

This way, if a neuronal ensemble wants to stretch across meaning levels, it will have to represent affordances at layer 5. This interpretation game is also a usability game.

Easy to imagine with eye movements again. If the banana from up top still says move here and you will see me, it knows itself how to be used. In this way, visual object memes are used by being looked at; If object memes fail to output the right eye movements, then they are worse memes. (It is a topic of development why our object memes are good).

If they are within reach, they can say grab me with your hand you will feel this weight and so forth. This way the object properties become lists of affordances. How do you understand the very large and very small, then? With analogies, I think. The sun is a ball going across the sky, atoms are billiard balls, and electrons are clouds or inkblots.

Observe that inner affordances, acts of attention exist, too. This way a thought-meme can say think me. This is the same as having a handle, a set of affordances, on the entities in your operating system. But in this paradigm, all the data structures try to figure out on their own how to be used. They are buttons with previews. And they are symbiotic with ideas that would lead up to them. This way a train of thought can have a handle on the first wagon, and the whole train would bias the system into thinking of the first wagon, if it could.

A little example of this is singing A, B, C, D, E, F, G, ….

P and O are memes, too. They have a relationship to A, and even more so to A with the tune attached.

Parallel Information Flows

  • M. Sherman: An Input Nucleus To Rule Them All, Parallel Processing Flows.
  • Those stacks of interpretation and affordance are all arranged in parallel.
  • I think this will turn out to mean that the concepts of say 'a line here', 'a color stripe here', 'a painting here', 'I could scribble on the painting',… Are all immediately, in parallel, constraining themselves, and they all immediately go from lower detail to higher detail perhaps.
  • "top-down" and "bottom-up" are then ensembles that span meaning levels - technically sweet I would say.
  • Then it is not surprising that most meaning comes from something else than sensor data if one simply considers the amount of neocortex that isn't primary sensory area.
  • And because of M. Sherman's A Curious Arrangement, all the meaning is knit together via action.


The edge of this expectation representation game is perhaps just when waking up, doesn't it feel like the system is booting for a moment? My interpretation is that all the neuronal ensembles need to ignite, using 'flash-drive' lookups and cues from the sensors.

"Surprise" and Confusion need to be handled, too. (also: Confusion and Socratic Wires).

Best Guesses Are Always There

György Buzsáki's neurophilosophy is rich. The notion of the pre-allocated hippocampal 'sequences' (4) is an implementation of the idea comes first.

Even an inexperienced brain has a huge reservoir of unique neuronal trajectories with the potential to acquire real-life significance but only exploratory, active experience can attach meaning to the largely pre-configured firing patterns of neuronal sequences.

The reservoir of neuronal sequences contains a wide spectrum of high-firing rigid and low-firing plastic members, that are interconnected via pre-formed rules. The strongly interconnected, pre-configured backbone, with its highly active member neurons, enables the brain to regard no situation as completely unknown.

As a result, the brain always takes its best guess in any situation and tests its most plausible hypothesis. Each situation, novel or familiar, can be matched with the highest probability neuronal state. A reflection of the brains best guess.

The brain just cannot help it. It always instantly compares relationships [the function of the hippocampus according to Buzsáki], rather than identifying explicit features.

There is no such thing as 'unknown' for the brain. Every new mountain, river or situation has elements of familiarity, reflecting previous experiences in similar situations, which can activate one of the preexisting neuronal trajectories. Thus familiarity and novelty are not complete strangers. They are related to each other and need each other to exist.

Buzsáki, G. (2019). The Brain From Inside Out. My emphasis.

Joshua Bach has a similar example. One remembers going into a dark cellar to fetch something as a child and having ideas of various monsters in the dark. It seems like the creatures are there in our minds. Not mere imagination, but the stuff that makes perception.

In my words, they are the meanings, beliefs, expectations, schemata, frames or situations. They exist in the mind, decoupled from the sensors. Perhaps sensors and action are "merely" resources for the ideas, to 'criticize' their alternatives. And this way, "good" best guesses are left over.

Neither the world nor the mind was a clear-cut 'ground', but grounding is an evolutionary process. And the function of brain software must be to facilitate 1) The existence of ideas 2) The criticism of ideas, in the broadest sense.

Languages, Compositionality, Abstraction

The infinite use of finite means by Wilhelm von Humboldt is what we call the means of combination in programming (5).

  • Make some building blocks.
  • Make rules for composing the building blocks.
  • Buzsáki calls that the neuronal syntax of neuroscience.
  • Chomsky calls that the universal grammar I guess.
  • ???
  • You have a system of expression with general, and vast reach, perhaps with certain universalities.
  • I think eventually we might be able to quantify such reach, and we would be able to say that with s amount of ways of putting things together, and n amount of building blocks, you get to an exponential explosion of expressivity.
  • This expressivity is at the core of natural language, and computer programming languages, and I think of the internal programming languages of brain software, too.

The means of abstraction would say that you need to be able to create new building blocks (given for a particular natural language via word coining). This includes giving names to "frameworks" in a broad sense. A frame in Minsky (7) is "A sort of skeleton, somewhat like an application form with many blanks or slots to be filled".

In programming, a framework might be a map or record, where the "keys" or "variables" (also 'roles') are filled with "values" (also 'fillers'). It can also be a sub-program, where some variables are substituted with particular values.

This allows the user of the language to speak of the relationships of things. And that is detail-independent one might say. (The same way information is substrate-independent, the substrate is a detail, then).

In what ways brain software represents such relationships and frames is one of the important questions.

[ ? action is how we come up with relationships in the first place. ]

[ observe that action can mean messing with the internal state of the system itself, too - internal actions ]

[ Two things are identical when changing either changes the other. ]

[ In order for anything to be stable, it has to move ]

[ ? … Then perhaps it is the ideas that change in the right way so they are stable, even when the details change, that are the abstract ideas ] [ This logic is mirrored in the so-called perisaccadic visual remapping (etc.) of visual neuroscience ]

Brian software abstractions allow us* to eventually express I see, or I understand. Without caring about the details, which are filled in somewhere else.

*) "Us" in this case means us, the hypothetical programmers inside a brain software system. Not the cognitive user of brain software, who is removed from such details.

At Some Memetics On Eye Movement And Maybe How That Says Where, I have the idea that eye movement motor data is an abstract encoding of the extent and relationships of objects in the visual field. This is also a frame because the 'line data' is allowed to be composed with any color and texture data. Making blue or red lines and so forth. A line is a thing where you can move your eyes like "this" and you still see the same thing along the way.

A competent meme can say all the ways you can make transformations (motor movements), and it is still true - it can do that by being abstract and removed from the details. Something like a high-dimensional, procedural puzzle piece, a bag of counterfactuals, a puzzle piece that fits many things.

For instance, the notion of a container fits the counterfactual notions of poor water into me and it would stay inside me, put your hand inside me and you will experience my interior, turn me around with water inside and water will flow out of my opening. (You don't need to do these things, it can say what would happen). There are many transformations a toddler can do with their hand muscles and so forth, perhaps animated by some alternative memes of container. Still, container would stay true. Or be superseded by a better representation of container. In a high-dimensional computing framework, such a puzzle piece with many tentacle arms in meaning-space should be easy to say, once some details are figured out.

Then I understand the scene perhaps simply represents the notion of a scene we understand, with the details filled in later. This rhymes with various change blindnes effects.

This one from Nancy Kanwisher and collaborators is amazing.

This hiding the details is a property of well-designed software this is essential for making the cognitive user interface magical: Magical Interfaces, The Principle of Usability.

[we can recover composition and abstraction beautifully via high dimensional computing, more to come].

This happened for genes, that somehow were able to express the realm of possible biochemistry or something. It also happened for the evo-devo genetic toolkit (Wikipedia, Sean B. Carroll 2008), I think, that can express animal body plans (universally?).

Also: Digital infinity.

What we need is an analysis of programming and software that is deep enough to include brain software. I am not done with a computational layer. I want the logic of organization of the computational layer. The elements (neuronal ensembles or high-dimensional data structures), the means of combination and the means of abstraction are the starting point, not the thing to be explained. The way architecture is relevant for understanding a building, not the bricks.

I also don't want to think of details of human cognition and say how this or that could be modeled. I want a fundamental theory, the way evolution does it for animals. The details of human cognition will come out of the elaborations of the rules of a deeper process (That's from David Deutsch).

Note that I think this probably means building in a developmental phase, that prepps the system for representing physical reality. Growing in Kantian representations style. On Development. I believe making machine intelligence and explaining brain software means standing on the shoulders of computer science (formerly just cybernetics) and neuroscience. It would be foolish to ignore the existing knowledge of computing and programming when the goal is to figure out what the biological version of it is.

Some Layers of Neurophilosophy

+----------------------+  mechanistic layer
|                 |    |
|                 |    |  Neurons, wires, devices, transistors
               +--v---+ <---------------------------------------------------- ensembles
+----------------------+ computational layer
|                      |
|    ^                 | ? neuronal nets, conceptrons
+----+-----------------+   high dimensional computing frameworks
     |        +--------+ <--------------------------------------------------- data structures
     |        +--------+                                                      building blocks
+----+-----------------+ software layer
|    |                 |
|             ^        | Memetic engine matrix, Societies of Mind (Papert, Minsky),
+-------------+--------+ Meaning Cities (Hofstadter)
|    ^        |     ^  |
+----+--------------+--+ systems of expression
|    |    ^         |  | layer, layer, layer, (?) ..
+---------+------------+ The content of the software is software again
+    -   -+    ---   --+ cognition
|                      |
|                      |
+----------------------+ The user and her interface
                         magical - but not magic.

                 +-- This is biology again
  | M  |  C  |  S  ... Cog          |
       ^     ^
       |     |
       |     |
   abstraction barriers.

M - Mechanistic layer
C - Computational layer
S - Software layer
Cog - Cognition layer

A heart valve can be implemented in terms of the biological leafy portal structure, or a so-called mechanical valve, which has a completely different mechanism, but its function is the same. Namely, to prevent blood from flowing back into a ventricle.

In general, an abstraction barrier allows a function to be expressed without mentioning the details of the implementation or mechanism. (What an abstraction barrier is, is I think expressable by constructor theory).

That there is an abstraction barrier between the mechanistic layer and the computational layer is uncontroversial. Otherwise, you would claim that the brain is not a computer.

It is the computational paradigm the brain implements. That is currently widely assumed to be neuronal nets [Hinton, Sejnowski]. But many people feel that there must be some richer aspects of the brain's version of them. Geoffrey Hinton recently mentioned he thinks there should be fast weights, pointing to what others call the neuronal ensembles and so forth. [In conversation | Geoffrey Hinton and Joel Hellermark].

Note that Yuste hypothesizes that ensembles are created on the fly by increasing their intrinsic excitability, not the weights (Iceberg Cell Assemblies).

Sejnowski mentioned how neuromodulators modify the function of the net on the fly, and how traveling waves pose open question marks. [Steven Wolfram and Tery Sejnowski].

My idea about the computational layer is Braitenbergs "conceptron", with wires:

The Computational Layer And The Software Layer

Note that it is not clear that the computational paradigm of the brain is the best computational paradigm to make minds either.

Idea 1:

The brain imperfectly implements the abstract notion of a neuronal net.

This is like saying

The heart imperfectly implements the abstract notion of a blood pump.

But there are infinite kinds of possible computational layers, too. And if they are universal, they all could be used to implement brain software.

Many things have been suggested for this computational layer. Including chemical computing at the synapses [Aaron Sloman], and quantum computing at the microtubules [Roger Penrose].

Those things might be true for all we care about when analyzing the software layer.

Because (that is theoretically provable or proven I think), every software can be implemented in terms of every computational paradigm. If that computational paradigm is Turing complete.

You do not explain a building in terms of building blocks. You explain a building in terms of architecture. The bricks are an afterthought.

A neuronal net is a computational construct that is generally powerful, it can be used to implement a calculator, face detector, or anything else - it is universal.

It would be a strange inversion of reasoning to say that an operating system emerges from the transistors. The operating system of the brain, which we can label mind must self-assemble and grow. The computational paradigm then, must facilitate a self-assembling software. However, the software is not explained in terms of the computational layer. (bricks and architecture, data structures and operating system relationship).

The principles of the mind are its organization. Mechanistic and computational layers are not sufficient to provide a satisfying explanation of the mind.

The Microcosm Is a New Landscape

Braitenberg was inspired by Hebb, who emphasized how assemblies can represent causality. (Wire with the stuff that activates you. That is B follows A). He called it the microcosm, which resembles the causalities in the world, the macrocosm.

This microcosm is an instance of what I call a theory of mentality. A mentality model says how the system can represent its perception states to itself, how that relates to the world, and how it can do so in the absence of sensor input.

For this reason, the discovery of intrinsic activity is a very important flip in the philosophy of neuroscience. Because it is not the case that the sensors trigger an information cascade in the brain, like dominos [Sherrington's reflexes]. What is true is that the brain is active without sensor input.

Activity that is independent of the sensory world could enable it to build a representation of the world, which can exist and operate on its own. This virtual universe may be what we call the mind.

Yuste 2

Let's be inspired by the computational layer, to think about the functioning of the software layer.

Ensembles are stable pieces of information. Marletto calls information that knows how to be stable is knowledge. Hopfield attractors are a perspective on stableness, too. In a way a pattern complete is the procedural way of saying information that can reproduce.

Of course, reproduction doesn't need to go horizontal with copies. Reproduction is allowed to go vertical through time and even skip time steps. I.e. if you are an attractor, you can be ignited ('fallen into') in the future.

If one takes this seriously, we see that this is a computational paradigm that makes it easy for knowledge to exist in the network. Sounds like a play on words, what knowledge? In the first place, the knowledge of the ensembles merely says how to be stable in the network. I.e. are selfish memes, with extended phenotypes [Dawkins].

Why this is useful is easy to see with the visual field banana from above. The stableness of your object memes is already an implementation of object permanence.

This stable information would also be useful if it represents "patterns". For instance, a bird will invariably start moving around when I look at it. We can grow stable information via a natural selection mechanism.

We can observe how this knowledge can survive, or not survive. The context of the network is criticizing all knowledge pieces, simply by saying what is supported.

This landscape is a landscape populated by sort of living things. This makes the ideas have agendas on their own. The ideas will then represent sort of their range of possible implications on their own. And this looks like a nice building material to make a mind that comes up with explanations.

In this kind of epistemology-biology-software, you can ask Cui bono? the same way you can do for organisms.

The ideas have an agenda, to be stable in the mind. That might mean that they want to represent pieces of what we label beliefs. This memetics is a superset of prediction theories of the mind. Because the ideas will want to represent pieces of information that fit the sensors. But they only care about being stable, not representing actual truths about the world. The ideas and the brain stand in a certain conflict because of this.

Names: ensembles to emphasize their mechanistic and computational implementation, data structures to emphasize how they can be understood in terms of computer science, ideas to be as general as possible, memes to emphasize their agency and their nature of being replicators, attractors to have a common language with computational neuroscientists, playdough gems when thinking about how you could code using them, explanation structures to emphasize their structure and function, or software entities, knowledge representations to emphasize how they are knowledge.

How to get users with magical interfaces out of this is the topic of Memetic Engines Create Competence Hierarchies Up To User Illusions, where I say that the software might be implementing a magic banana maker matrix meme-engine. This is a general mechanism that should populate a computer with competent software entities that use the computer.

In order for this to work the mechanism requires a so called memetic engine. This memetic engine needs to make variations of knowledge possible. A natural selection mechanism is mirrored in the synaptic 'overproduction' and subsequent pruning of synapses. Presumably via this process, a network goes from very high possibility spaces to a more narrow possibility space (which is more useful seems).

  • Why are more synapses not better?
  • Why would neurons have a skip rate or noise? (Skip Rate)
  • Why can you not heal neuronal tissue?
  • Why is the dimensionality of the net reduced, e.g. cortical columns are correlated?
  • What is the purpose of sleep?
  • Why is there no illusion that makes me see multiple colors in the same spot?

Random firing rate and random skip rate are the basic implementations of a memetic engine, so here the high-level theory is yielding explanations for what to see empirically. With this view, it is not surprising that random noise in neuronal nets makes them more competent for instance.

Niche Fitting

Imagine an alternative history, where people still speak Victorian style. And where Darwin's and Wallace's theories were ridiculed and forgotten. Like tiny fireflies of insight, burning away in the dark.

The people of this world moved on without a theory of life;

'The mechanistic and architectural power immanent in protein biochemistry' from Warren McLoccuch and Walter Tipps in 1943. Pathed the way of seeing the cells and organisms as biochemistry devices.

Many biologists think that it is the job of biology to explain how the biochemistry of organisms is 'fitting the niche'. Their implicit bio philosophy is that 'fitting the niche' is a theory of life, and questioning how biochemistry relates to animism and complex adaptations is seen as a problem for philosophers.

Some philosophers speak of Élan vital, the substance of life. Some say that Élan vital can never be understood, that some mysteries are beyond the human intellect. Some say there is a substance dualism between life stuff and so-called ordinary matter. An animal is something else than a rock, and yet one is alive and one is not. How does the biochemistry give rise to aliveness? Some argue that aliveness is perhaps an epiphenomenon, along with the ride of the real thing that is happening, biochemistry.

Some biologists think that the mystery will be solved in 100 years hence perhaps not in my lifetime. They have searched and searched, and yes the ship is made from wooden planks on the left and the right. Now they are going to look if there are wooden planks at the front and back, too.

The problem of aliveness was always there and then labeled the hard problem of aliveness by Chavid Dalmers in the 2000s.

The field of machine proteomics is inspired by the 1943 McLuccuch paper, and implements statistical fitting of 'proteomic nets'. They started tinkering with nanotech, which mimics the function of proteins and other elements of cells. They created statistical algorithms and found algebraic tricks. The valley-slope mechanism allows for simulating the biochemistry of organisms, tweaking each protein parameter at a time, and 'fitting it to a niche'.

They have some impressive results and the field of artificial biology is a synonym for machine proteomics in the mind of the public. Artificial animals that do things like brewing beer and so-and-so.

In some ways, the impressive results of machine proteomics are more confusing than useful.

The prevailing understanding is that the biochemistry of organisms is implementing something like a proteomics net with learning. Those are networks of proteins, which interact with each other and produce the properties of cells. They are fitted to a niche via the process that tweaks the connections in the protein net called learning.

Many biologists are trying to use machine proteomics when they analyze organisms and find striking analogies between artificial and biological proteomics.

Biochemistry was once a deep topic for cyberneticians, but this field virtually doesn't exist anymore. Biochemistry became biochemistry science in university courses, where students are trained to contribute to industry, using artificial proteomics, and making money for businesses.

It looks like this organ is doing biochemistry for x. Is a respectable thing to say, even though replaced the word biochemistry with magic and get the same information content, yet, such rhetoric is required to be a respectable biologist.

For instance: It looks like leaves are doing the biochemistry of the tree. This is taken as a respectable and sensible thing to say. Students who ask but what is the biochemistry? are placated: I don't have a proteomics diagram for the biochemistry of a tree. The field simply is not this far.

Curious person: But why should this proteomic net be this proteomic net?

Scientist: Because this way the organism is doing exactly the kind of biochemistry that allows it to survive the challenges of the environment. Organisms are environment overcomers, and they make biochemistry that helps them survive.

Curious person: Alan Ruting's universality of biochemistry shows that all proteomics do all kinds of biochemistry. Doesn't this mean that there is something else to explain? Something that says how the proteomics is organized to do its biochemistry?

Scientist: People like Larvin Linsky in the 50s-70s have tried to come up with an organization of biochemistry, but they have failed. Because in the real world, biological niches are more complicated than the toy niches of Linsky. That is also the bitter lesson of machine proteomics. You can only get better organisms by scaling up the compute and niche data.

Curious person: But organisms in reality do not have millions upon millions of niche data points, from which they find statistical correlations of proteomic sheets that fit. The moment an organism is born, something already talks about what kind of proteomic net will be useful for it in its niche.

Scientist: Yes, that is the mystery of life. People like me are making progress in understanding what kind of biochemistry actual organisms use in order to fit their niche. But it is a slow process. Look at how we know that cell membranes implement such and such a function… And we also recently found that skin cells also do biochemistry!… Also, we found recently the biochemistry mucus membranes. I have a colleague that takes samples from real humans. They are now investigating whether those cells are also doing biochemistry… By decoding from the proteomics of organisms, we can say whether it comes from an eye cell or an arm cell. Isn't modern biochemical biology fascinating?

Scientist 2: At my lab, we take the idea of organization seriously. We use graph-based biochemistry, to model how real organisms might do proteomics in hierarchies and heterarchies.

Curious person: Nah, I mean something more fundamental than such details. And I think Linsky was, too. Why isn't there a deeper theory of biochemistry organization that is independent of the niche?

Curious person (to herself): I have to think something else than what these people are thinking. Imagine an alternative history where Alan Ruting died early and he didn't come up with the [ommitted] theory of brain-knowledge. What a strange world this would be. They would say the brain is doing information processing and this would be the end of their neurophilosophy. shudder.

Prediction-Action As An Explanation of The Mind

The image of the world in the brain is a dynamic representation; not only the neighborhood relations and other forms of association of things are represented, but also the laws that govern the evolution of the environment. The patterns of activity within the brain that represent the present state continually go over into new patterns even if the sensory channels are temporarily closed, by virtue of the internal synaptic connections. If the brain is well wired, this evolution of the internal state matches the evolution that would be otherwise imposed by the changes of the input. Thus the capacity to predict the future is an essential attribute of the brain. Normally, in the process that we call perception, the sensory input and the internal prediction are inextricably mixed. They are the two irreducible components of the process of acquisition of knowledge in all cases where the available information is incomplete. And the amount of information is always insufficient as long as the brain is so small and the world so large. In science these two components are called hypothesis and experiment. As in science, so also in the brain the discrepancy between the prediction and the actual evolution of the input is used continually to amend the predicting mechanism. Not only a new hypothesis has to be developed, when the old one proves wrong, but the principles that generate the hypotheses have to be revised. This is how the information from the macrocosm is incorporated into the microcosm of the brain. The brain as a foretelling machine clashes with the more down to earth view of the brain as an input-output device that gives appropriate responses to sensory stimuli. However, the two views can be reconciled. The prediction that the brain makes at all times by letting its internal state evolve in accordance with the most probable evolution of the world, contains also a prediction of the future position of the animal itself within its environment. We may suppose that the motor output secondarily follows the prediction of the next state of the motor system. In the action of tracking a moving object we notice how our output is not governed by the immediate input but is, rather, a consequence of an ongoing prediction. When we sing in unison with other singers, we notice the curious ambiguity between action and perception. Subjectively one's own action of singing is perceived in a smooth blend with the singing of the others, or both seem to emerge from a continual process of prediction. We are reminded of the anatomy of the cortex where the motor area is not subordinated to the sensory areas, but arranged in parallel with them in the continuum of the cortical plane.

Braitenberg 1.

This flip away from the input-output device [Sherrington's reflex model] to the brain with an active role, it's own agency, it's own inner states (- mentality). For some reason, this is still cutting edge neuro philosophy [4, 2, as well as A. Clark recently, are still pointing this out].

But prediction, or action experimentation - prediction (emphasized by 4) is not sufficient to explain explanatory universality, also called creativity by David Deutsch in 8.

If I say I imagine a wizard flying on a dragon through space, backwards in time to the big bang. What was the role of prediction? Marginal. What is the role of an expressive software language for representing dreams of possible realities? That's somewhat closer to the mark.

If Alice comes home from a job interview 20 minutes early, and Bob perceives her sitting on the sofa without saying anything. Then Bob has an idea structure in his inner world that represents Alice and her interview, probably with a place. This situation somehow is central to Bob's interpretation of reality. But the situation is in the past and the situation is invisible to Bob.

And I think that Bob probably had multiple different alternative ideas, one in which Alice interview went well, and one where it went badly. These expectations are made from counterfactuals.

I would say perhaps Bob would be able describe the place with richer and richer details if being asked: A desk, some stern interviewer, plants in the corner.

Perhaps we can label this the microcosm, inner world of ideas, or virtual simulated world, or the mental world of Bob. If Bob is using a foretelling machine to generate this, then he has fine-grainend competencies of instantiating a certain start state of this foretelling machine. We are then interested in the function and rules for instantiating these inner start states of his, which one might label situations (Hofstadter).

We wonder about the same interesting questions, for instance, how does Bob manage to not get confused between his inner start states, and the ones coming from the sensors?

One might say now that Bobs prediction machine is using his mentality as a resource to predict the world.

                     /------+        inputs, error  <+
                    /------+| <---------             |
                    |      ||                        |
           +--- -- -+- P   || ---------> prediction  |
           |        |      ||
                    +------+              ^
           |                              |
                              -- ---- -- -+ informs
           |     +---+
           +-----+ Ψ +----+  rich inner world
           |     +-+-+    |
           v       |      |
                   v      v

         ideas, plans, imagination - the mind

P - the predictor

But then the mind was again the interesting thing to explain. The structure, function and organization principles of what makes ideas and explanations.

Sometimes we are rooting for a certain kind of explanation. There are still bananas in the storage room. And we look for all the little hints that support the idea (called motivated reasoning in cognitive neuroscience). We are not predicting the world, we are grooming a belief.

I'm sure when toddler acquires language, they hear you say apple in a situation where there is an apple and a banana as possible referents. They will not only realize that apple is the word for the thing that they don't know yet, but they will do so retroactively, too. I think the way it works is that you lie in bed a week later as a toddler, and by now you have learned what a banana is, you somehow remember the apple-banana situation and you go: "aha! this thing must then be an apple". It's a bit of a stretcher to explain this process in terms of a prediction machine, or prediction-action-experimentation machine. This prediction machine is creating a fine-grained tapestry of explanation, where situations and micro situations are all made to fit in some kind of society of knowledge. And things like this.

Using A Cognitive Machine, Dreaming

The concern of how brain software uses itself is currently labeled 'executive control', 'top-down processing', […] and similar.

Perhaps it will turn out to be true that languages cannot use themselves, they must be obedient or not be a good language (The Living Language Problem).

If we want to explain cognition, cognitive control and these things in terms of prediction; We start saying something like a cognitive user of the mind is predicting themselves to remember or take a different perspective. Or to navigate in thought space and so forth. Do I predict myself to manifest a banana in my imagination and that is how I manifest bananas in my world?

There is a rich set of expressions, affordances and buttons of the system to itself. To say that such buttons work by internal prediction is a bit strange, because they say what would happen, not what will happen.

Similarly, an alien operating system interface is powerful not because it merely anticipates what I will say, but by providing me with a world of possible things to say. This is a deep counterfactual property of information to begin with [Marletto]. There must be a certain universality, presumably coming from a combinatorial explosion from the composability of such languages (sets of affordances) in the brain. And I think this freedom of expression is a core aspect of what David Deutsch labels creativity; 'The stuff of thought', or 'the stuff of mind', one might say.

My current idea about dreaming is that in general, it's keeping the information medium dynamic. This way everything is still able to be expressed - a deep requirement for a language to work. And because the thing being expressed is a simulated inner world, dreams (randomly?) explore the space of creating (inner) worlds. Our world has been called a waking dream, and I think this turns out to be true. Perhaps the mind can be seen as dream making engine, where a special dream, the waking dream corresponds to reality because it is implicated with sensor data.

The cognitive machine as a computing framework

The competencies of the system are allowed to flow from the user of this computer.

Imagine this experiment (which has been done):

  1. Have electrodes in the hippocampus of a human
  2. Show them random movie clips
  3. Ask them to remember a movie

Now you record some neurons (indirectly ensembles) and you can say "When the human remembers such and such clip, then those ensembles are active".

But how do you know that the human wasn't messing with you and issued completely random memory lookups when you asked them?

In this situation, there is a world of difference between the system under study and the actual thing going on. This is roughly the same difference between our current computer programs + AI's and the real deal. This gap can be called the fun gap perhaps. Because if the system is a person (i.e. they are creative), they can always mess with your experiment or the programming you put in. The system is so much more competent and expressive compared to the ways we come up in studying it, that it's a practical joke.

[I don't want to disparage this kind of neuroscience. But I want to flesh out the problem of explaining the mind].

It would be a level too far down to paint a picture of a vast ocean of information processing. And this then decides somehow like vortexes (attractors) and so forth to do things like memory lookups. (I.e. 'executive control information flows').

But the important thing is how there is a user and her world running on this computer that you are looking at. And this user has some kind of universal language or interface for using their brain (a mental world).

How a computing system can self-assemble into having these competencies, is I think the theory that will in a satisfying way talk about what minds are. Much of current neuroscience then is more like the study of a biological computing system. The real flip into saying what the software is that runs on this computing system is not taken yet.

We can ask What does the system need in order to make a user who competently uses the system?. Instead of asking how the system interprets and evaluates the situation and then makes a judgment on possible behaviors (like G. Buzsáki emphasizes in 4).

If we go into a deprivation tank for 4 years, the property of creativity is still happening in our minds. As David Deutsch points out (pretending for a moment you are a monk you won't go mad). Presumably, it is impossible to grow a mind without input and output, but once it is there, the function of the mind is decoupled from input and output. (It is probably a deep property of the mind, to begin with. Presumably, this has to do with imagination and dreaming, too).

The analysis of brain-software is the analysis of user interfaces*). And because the ultimate competence is creativity (8), a real explanation of the mind says how biological software can make a creative user.

Knowledge must be made from systems of expressions that make universal explanation languages. How the mind can be a self-assembled software, and what software and self-assembled software even are - I think those are some of the questions leading to a theory of what minds are.

*) the analysis of user interfaces is the analysis of computing frameworks and languages, they are all equivalent (I think I can show this theoretically).


The basic idea of reinforcement learning is that you approximate an f(x) that has competency in the world.

                | inputs, error
                |           ^
 -------+  <----+           |
/------/|  -------> output
|      ||
| f(x) |/ black box

This picture of the cortex as an unorganized machine is very satisfactory from the point of view of evolution and genetics. It clearly would not require any very complex system of genes to produce something like [… a kind of] unorganized machine. This should be much easier than the production of such things as the respiratory center. This might suggest that intelligent races could be produced comparatively easily. I think this is wrong because the possession of a human cortex (say) would be virtually useless if no attempt was made to organize it. Thus if a wolf by a mutation acquired a human cortex there is little reason to believe that he would have any selective advantage. If however the mutation occurred in a milieu where speech had developed (parrot-like wolves), and if the mutation by chance had well permeated a small community, then some selective advantage might be felt.

Turing 1948 (6).

It is a thing to attempt, not to explain what minds are, but to see if you get intelligence in a shortcut way. At maximum, it is an attempt at a mechanistic description of a software problem; That leaves out the software layer.

The problem is that f(x) needs to make your software. It needs to make an internal structure:

        |    |
        |    |     +---+
     <--+    +---> |-- |
        |          |-- |
        v          +---+

thoughts, ideas, plans, cognition

What is thought and things like this? Surely, it has to do with internal states and organization of f(x).

You are back to asking what is this large structure of competencies, that organizes and perturbs itself and so forth.

Learning As Part of Your Computational Paradigm

Now let's say that you have an architecture of a mind in mind, and you use machine learning to train little pieces of a larger organization:

| A <----+      f'(x)
+---+    |     +----+
         +----->    |
f''(x)   |     +----+
+---+    |
| B |    v
+---+       ----->

You think of A and B as cognitive modules in an organization. Perhaps you label them working memory, attention, association box, plan suggester, plan picker, expert delegator and so forth.

And you use learning and reward for all the little boxes, that approximate functions f(x), f'(x),…

You are in the business of finding the niches for your functions. This niche finding is the programming you are doing. You are back to thinking about the organization of cognition, i.e. you are doing scruffy AI.5, (7).

This is in some ways the ultimate behavioristic paradigm, that saying that the mind is made from myriad small, trained sub-pieces, that contribute in intricate ways to the processes we label mentality and cognition. But as Chomsky has pointed out (1967), that is the same as wondering what the mind is in the first place:

Stimuli are no longer part of the outside physical world; they are driven back into the organism. We identify the stimulus when we hear the response. It is clear from such examples, which abound, that the talk of stimulus control simply disguises a complete retreat to mentalistic psychology. We cannot predict verbal behavior in terms of the stimuli in the speaker’s environment, since we do not know what the current stimuli are until he responds.

Because now we are in the business of wondering what kind of organization allows for a rich 'I hold this thing in mind', 'I use this thing from the past to inform my behavior'. Saying that they are trained doesn't solve the problem of what the organization is.

David Deutsch:

I think that's why they haven't succeeded for decades. Because a philosophical advance is needed. And they are trying to do it without any philosophical advance. And that leads them essentially to sort of behaviorist models. Behaviorist models are nonexplanatory models. They are models that just try to explain output to input, without explaining why the output comes from the input and so on. And I think that approach can't succeed and it's the reason this quest for AI has not got anywhere during the last decades.

What we need first is philosophical progress in understanding how creativity - I think that's the key thing that relates all these unsolved problems about free will, consciousness and so on - How creativity is implemented.

And we know a few things. It has to be in the broadest sense an evolutionary process. It has to work by variation and selection. Or as Popper calls that in the case of science "conjecture and refutation" or "conjecture and criticism".

But we need to know the details, and the devil is in the details.

I guess that once we understand what it is, we will be able to program it.

There is an analogy here with Darwin's theory of evolution. Darwin's contribution in my view is not his scientific theory of evolution.

It is the philosophical progress that he made in inventing a new mode of explanation not just a new explanation, but a new mode of explanation.



It is explaining the process that could make things like elephant trunks, not explaining elephant trunks. That is left to the myriad details of the process.

Science Saturday: A Plan to Dye One's Whiskers Green | John Horgan & David Deutsch

If you wonder along the lines of "what if the system is dynamic enough so it finds the niches on the fly?", that is the kind of thing I label self-assembling software.

The logic of reward and training can only ever be of marginal interest in the theory that explains the mind. Training only relates inputs to outputs and is not an explanation.

+----------+ +--+
|          | |  | training
|   mind   | |  |
|          | |  |
+-+--------+ +--+
  +-- explanatory knowledge
  +-- imagination
  +-- counterfactuals
  +-- ideas
  +-- mental world

The mind is the interesting thing that we want to explain.

For a similar reason, it is not that our explanations of the world are induced from the sensors. In my view, the explanations must grow, and it is only then that some explanations can use the sensors as resources, in a struggle for survival. This inner explanation world is also a Kantian thing, too. We construct the explanations, and what we perceive are those explanations, not the world.

I don't know which philosophers were saying that the ideas are alive, either way, they were right.

Notes On How Neuronal Ensembles Are Different Replicators From Genes

What is the least surprising thing? It's a copy of yourself.

Micheal Levin

Audio diary (better at 2x speed):

Musings About David Deutsch's Challenge To Natural Selection Mechanisms Of Knowledge Creation In The Brain, Pretending To Know What Abstract Replicatory Is About

Dawkins [1976] established the notion of abstract replicator theory (called Genes Eye View of evolution in biology and abstract replicator theory by David Deutsch). This is how he was able to predict computer viruses in 1976. Way before mainstream internet tech.

A replicator is a piece of knowledge that knows how to 'replicate' itself. This does not necessarily mean that there need to be many copies of the replicator.

The knowledge substrate of biology are strands of DNA, the genes.

The algorithm of natural selection finds the pieces that represent more useful knowledge, the ones that are better replicators. That is they have high stability, fecundity and/or fidelity [Dawkins 1976].

Ultimately, this can be reduced to the notion of stable knowledge.


Chiara Marletto and David Deutsch [Deutsch, Marletto 2014: Constructor Theory of Information] say the information is substrate-independent, it does not depend on the media in which it is instantiated. It can also be moved between media, which they call the interoperability property.

Marletto says calls knowledge pieces of information that know how to be stable.6

This stability of abstract pieces of information is the fundamental property of replicators.

It is only incidental that a good strategy for doing so is replicating into many stable, coexisting copies, with the properties of stability, fecundity and fidelity of Dawkins.

Neuronal ensembles have a fecundity of one i.e. neuronal ensembles replicate across neuron time steps

I like to call 'copy horizontally' to increase the copy number in a single time step. And 'copy vertically' to stay stable across multiple time steps.

From neurophysiological reasoning, it does not seem likely that neuronal ensemble would be able to copy itself 'horizontally'. Neuronal ensembles exist as subnetworks, and each piece of subnetwork has its own identity. (They are non-fungible).

The person who is on the lookout for ensembles with a higher copy number than 1 would need to look for a neurophysiological arrangement that allows for fungibility.

This would need to be a Boltzman-Memory-Sheet (The Boltzman-Memory-Sheet) in order to copy one exact idea, one would need to create a Boltzmann brain of one's current brain state. This is not useful, because the purpose of the brain is to have an ongoing situation analysis.

Note that from software reasoning there are many places where so-called replicating or copying data would be useful:

  • for the load of a mid-term memory
  • for replicating to distinct modules for speed, (or because evolution didn't figure out how to deal with a single instance of the data): For instance for the cerebellar somatosensory map.

I would argue while this might be seen as a horizontal copy, this is of less fundamental interest. It is part of the function of the brain, but not part of the fundamental explanation of how it is creative, i.e. how it can create explanatory knowledge [Deutsch].

Also, note that we can see memes spreading into mid-term memory and subsequently being remembered, as a vertical copy.

Perhaps the arrangement of the 2 hemispheres, is an actual place of some approximate horizontal copying?

The complete ensemble is a meaningful concept

When we choose to take this perspective, we say that there is a single ensemble active in the brain at each time step. This surely looks a bit odd, if the selection mechanisms would depend on vast amounts of copies with small variations.

Neuronal ensembles are hyperdimensional

  • Parts of information are meaningful i.e. pattern completion is fundamental.
  • This property allows for a mix of information.
  • This allows to represent of vague or low-detail information.
  • many things possible a little bit is a valid thing to say.

If we understand the brain and its network in hyperdimensional terms, there are some counterintuitive properties:

  • The network is allowed to be random initially, if connected to sensors, it forms a representation of the input [Vempala 2022].
  • The output poses no problem, simply connect some elements of the network to the effectors (Braitenberg).

Since the timescales are on the order of low amounts of neuron timesteps. (12-20 by Vempala irrc), this might shed light on some empirical results of neuroscience. For instance, place cells are called 'innate' because you can record them in mice after 30 minutes of opening the eyes or so.7 Perhaps what was innate was a very generally powerful computational substrate, representing its inputs.

Hyperdimensional computing, Assemblies of Neurons Learn to Classify Well-Separated Distributions

Neuronal ensembles represent their counterfactual possible continuations

An ensemble has many possible continuations, depending on the dynamic context. I.e. what else is active in the network? Or the 'situation' at hand. Increasing the excitability of the network activates the halo of an ensemble, which represents its 'nearest' possible continuations.

It is like ideas are always a cloud of possiblities. From this we see that variation of a natural selection mechanism of ideas is allowed to be computational. I label the part that creates this variation of the ideas the memetic engine.

A thought pump searches well-connected ensembles in a massively parallel mechanism. Using small amounts of neuron time steps.

The thought pump (Braitenberg 1977, also Palm, see below Tübinger Cell Assemblies, The Concept Of Good Ideas And Thought Pumps). Is a parallel mechanism using the hyper-dimensional properties of neurons.

  1. Increase the excitability: You will activate the halo of the neuronal ensemble. It is a bit like saying we express all possible continuations from the current ensemble.
  2. Lower the excitability: You will find the best connected sub-ensemble.

This mechanism in the limit needs 2 neuron time steps. It doesn't bother with trying out possible ideas 1 by 1. It reaches immediately an interpretation that fits the network well.

Comparing this to natural selection, it is a bit like you can say jungle and the rest of the computer immediately pattern completes to a gorilla. The network says that many things are possible already and from the space of possibilities a high dimensional point is selected virtually immediately (down to 2 neuron time steps).

But note that multiple temporal layers must be at play. The network will have to be developed for the meaning spaces to represent useful ideas.

A parallel computer can look at everything at the same time (Foerster). Note that this is not a claim of absolute power, which is covered by universality. This is in the realm of what is easy to say, what are the building materials out of which brain software is made out of?

I think it is this kind of 'running on the physical reality' that Aaron Sloman would appreciate, too. He mentioned in an interview once

Imagine pulling on some strings, the shortest one will be the one that is straight first. What if the brain is using such tricks?

This is a parallel search algorithm in constant time that runs on the physical world. Compare this to the naive computer program with O(N) complexity (needs to compare all strings to find the shortest).

The neuronal ensembles might implement, or even make a general language to express such tricks somehow.

Neuronal ensembles evolve always

Ensemble theory (the notion that ideas are made from neuronal ensembles), if true, tells us that every time an idea is active, it is changing.

Why is this so? Because the plastic network allows it to associate with new things. Because no situation is ever the same, and ideas that are active together, have the chance to associate (find overlap) together.

This doesn't mean that there are not tremendously stable aspects of for instance how we can bring Beethoven's 5th to mind. It also doesn't mean that long-term memory is not some stable aspect of the network. It also does not mean that you never need to repeat and sleep over an idea to remember it, (consolidate its associations).

I think it was a philosophical question of old? Or if not, it should have been. The question whether your ideas are the same, or they are new every time they exist. Or the question is whether you store your memory, or store the memory of the remembering every time you remember.

Ensemble theory says well whatever - it depends on the properties of the network and its orchestration. But in principle, these things are updated every time they are active.

Gene pools - ensemble pools?

A gene pool is a shared set of genes that is separated from other such sets. It is a useful abstraction for evolutionary biology, when considering speciation mechanisms.

The important thing about a gene pool is how it is separated from other pools.

You might want to call the maximal neuronal ensemble from above a pool.

The halo of an ensemble contains multiple sub-ensembles. You might go that way and call that a pool, out of which a thought pump would now choose a subset.

I find these contrived and would simply not use the term pool in neuronal ensemble memetics.

'Meme pools'?

The closest to cultural meme pools would be things like 'information bubbles'; And the cutoff culture of uncontacted Amazon tribes. These truly might be the analog of a 'species' of culture. But why bother, if 'information bubble' is a good name?


A gene complex / Supergene:

A supergene is a chromosomal region encompassing multiple neighboring genes that are inherited together because of close genetic linkage, i.e. much less recombination than would normally be expected.

A gene eye view explanation lies at hand, they are selected together because they contribute all together on some function, presumably, if one is missing the whole gene complex would not have any effect on genetic fitness.

The genes in a gene complex form a coalition then. And contribute all to some function.

For neuronal ensembles, we see that all ensembles are made from sub-ensembles, even if they contain a single center in Braitenberg's sense. Because we can always imagine splitting an ensemble in two, and then each of them has a slightly different meaning depending on its connections. (Presumably random in brain).

This certainly is a thing for the ensembles, too. A compose would presumably be a fundamental operation of neuronal ensembles. That is juxtapose two ensembles together, forming an ensemble that is made from both, perhaps with temporal structure (or not).

Neural ensembles are their own composite data type, so all ensemble mechanisms will have a closure property in the programming sense. That is, the same operation will work for the composed ensemble in turn and so forth.

Vempala calls this hierarchical composing a merge. So he probably would call those merged cell assemblies?

Names: Supersets, composed ensembles, aggregates, super ensembles, ensemble complexes, the complete ensemble (in the sense from above. The largest ensemble present at one time in the network).

The other way around: Centers (in Braitenberg sense, imagine lowering the excitability, which neurons would be left over?), sub-ensembles, elementary ensembles,

Theoretically, you might label an atomic ensemble one that cannot be divided further for ensemble properties to hold. This would depend on the neuronal area (its inhibition model and so forth). It would also need to take into account the time window.

Rafael Yuste mentioned that they could not produce bigger artificial ensembles than 100 neurons8, perhaps this puts the size of atomic ensembles in a rough ballpark. (Certainly smaller than 100).


Ensemble composition into super-ensembles is not to be confused with the different notion of association, which finds the overlap of ensembles. Conjecture: This is another basic operation in ensemble theory: Activating 2 ensembles in an area together will rapidly create an overlap ensemble. (This is proven for assembly calculus [Vempala 2022], which uses Hebbian Plasticity). We say they associate. I also have called this merge. But given that Vempala calls the composition from above a merge, it would probably be less confusing to call it associate, or overlap find, or create a new ensemble that is similar to both.

This is a bundle in hyperdimensional computing.

This overlap-making is remarkable: The logic of the substrate simply forces the ensembles to create new ensembles, similar to whatever is active together. This is an example of where an explanation of the level of the neurons would break down. Why would activity move forward, leaving its neurons behind?

Reasons for brains / The success algorithm

  1. The brain evolved as an animal-navigation device, in a broad sense of the term.
  2. The basic building plan is the one from Vehicle 1:

   ^            v
   |         sensors
   |           +
   |           |
   |           |
   |           [B]---------- 'wiring
   |           |             (meaning)
   |           |
   |           |
   | (body)    +

                       'wiring' (meaning)
    +---> sensors --------[B]--------------- actuators
    |                                            |
    |                                            |
    |                                            |
+---+----+---------+                             |
|   |    |  body   | world  <--------------------+
           (body is part of the world)

Note that it is perfectly fine for the wiring to go in both directions and so forth. The state of the actuators is again a kind of sensory input to the system etc. Also, the actuators change the world and the sensors perceive this change.

The wiring is allowed to become more complicated, by adding more elements, called [B] for the brain. From this we see the evolutionary driver of this system, to evolve more useful meanings.

Consider Vehicle 1 (Braitenberg 1984):

 ( )     temperature sensor
+-+-+     +
| | |     |
| | |     | connection
+-+-+     v
 [ ]     motor

This is a bacteria-like intelligence (incidentally insulting bacteria intelligence, which is made from thousands of such elements).

If the sensor is on, the animal moves, or vice versa. Depending on the sign and connectedness of the wire - it's meaning.

The wire can mean either: 'I move in hot water' or 'I move unless there is hot water'. (By negating the excitation effect of the sensor to the motor).

This allows us to draw a 2x2 matrix, imagine the case where hot water is detrimental to the animal in its environment:

  move!      stay!    (the meaning of B for the animal given a world, hot temperature)

|   S     |   X    | competent (sensors and motors work)
|         |        |
|   X     |   X    | incompetent
|         |   S    | (luck)

X means you die and are discarded by evolution, but you are allowed to be tried out. S means 'success' so these are the animals that we see running around [Darwin 1859].

Whatever complexity B is growing into then, Its fundamental evolutionary driver is 'success'.9

Here is a challenge to computational models of cognition10, what is this program and software then, that runs on the brain? As a computational cybernetician, I need to think about the thoughts that say 'Those are the kinds of programs that are cognition machines'. I need strong ideas about the nature of this program.11 I cannot be content with the general direction of the idea, I need a building plan or a development plan.

One perspective I have is that the brain implements a 'success machine'.12 We get an evolutionary driver for `abstraction`. Because I need abstract ideas in a resource-constrained world in order to have success (short and meaningful memes). From this, the computer programmers' intuition about the nature of abstraction and the power of programming languages starts forming a brain-software-shaped question mark, waiting to be filled with ideas of what kind of building blocks, computational primitives and organization rules could exist that would shape the memetic landscapes of an idea-machine.13

I believe you should be able to point to the algorithm/software/program in its infancy and have a story of how that evolved.

I call this move 'putting the algorithms into the world', to realize that the programs are evolved entities, too.14 A brain must be fundamentally optimistic15, we can consider a pessimistic brain 'if I am in hot water, I will stay in hot water'. It will be discarded by evolution. It is only the brain that says 'If I am in hot water, I will not stay in hot water' that can have success.

Alternative names: 'Evolution favors competence', 'The survival algorithm' (G. Palm), 'Meaning evolves', 'Purpose', 'Purpose-Engine', 'optimism-drive', 'abstraction-driver', 'abstract memes are useful', and 'the space between your sensors and motor needs to be short'.

I will keep coming back to this holy overlap of competence and optism. Turns out that is a deep principle of memetics, too.

How The Brain Relates To The World

This is not a high-highfalutin problem. This is a simple analysis of what are the wires between the world and the brain, and what mechanisms allow ideas to exist as a kind of software on the brain.

This analysis is needed in order to have a complete explanation of what the brain is. I would label this 'neurophilosophy'.

All neuroscience is parochial unless it has a neurophilosophy. Its explanations are small, and will not talk about how things could be, and then say how the brain is doing one of the possible solutions to each problem. Without a neurophilosophy, you would not be able to look at a brain or its software and say this is a kludge. The way we can look at the anatomy of the eye and say this is bad design, the ganglion cells should point the other way, like in mollusks. (Blind spot)

The Science Fiction Software Engineering Approach

This is an exercise in fictional science, or science fiction, if you like that better. Not for amusement: science fiction in the service of science. Or just science, if you agree that fiction is part of it, always was, and always will be as long as our brains are only minuscule fragments of the universe, much too small to hold all the facts of the world but not too idle to speculate about them.


One of the important things you need to do as a software developer is to keep the overall picture in mind. What does the system need to accomplish? It is like dreaming up dream castles floating in the air, not because they are cool, but because this is the first of a 2-step process.

In the second step, we wonder what foundation we need to support these kinds of castles. It is a constant back and forth, between the kinds of stuff the system needs to be in the first place, and the kinds of stuff we can build with the materials at hand. Between sup-parts of the existing ideas, in juxtaposition with a potential new idea. The most beautiful thing about programming is that part of what we do is coming up with new kinds of building materials. Different names for the same thing are 'languages', 'tools', or 'building blocks'; each with a different emphasis.

These are sort of the main overarching principles I use to circle in on models of cognition. The software development approach.

Think of what kinds of building material the brain might be making, then think about what kinds of problems the system needs to solve, and then think about what kinds of mechanisms can use the building materials at hand. Then realizing there are more requirements for the kind of stuff we need to express and so forth.

Imagination is the only way we can hope to build models about the world. I think that explaining cognition will be similar to explaining life and natural selection. I want my explanations to click, like beautiful toys and colorful candy. It is the spirit of the Braitenberg Vehicles (overview).

The words of the language, as they are written or spoken, do not seem to play any role in my mechanism of thought

Albert Einstein

I am programming this neuronal area and getting a feel for what the neurons do:


Figure 2: Combining a few mechanisms, toy neuronal area with a sensory field. Effector neurons pull sensor balls into the sensory field. Sensor neurons are statically allocated per color and fire when a ball is in the field. The neuronal area resets every 5 neuron ticks. So ensembles need to re-ignite.

Thinking and dreaming about those neurons lead me to the idea of adding a skip rate and an intrinsic firing rate recently (see below, this is actually useful).

We can put that into those vehicles again later and so forth.

Programming Is Different From Science With Computational Methods

Programming (or software engineering) is a kind of practical philosophy, which can be applied to any kind of problem domain. This page is an attempt to apply it to the domain of explaining biological intelligence.

It is a blend of art, science, and engineering, it cannot be otherwise. It might be a going back and forth between building blocks and their reach (described here The Science Fiction Software Engineering Approach). How exactly to do it is poorly understood.

On a content level, the mind must be a kind of software (see Magical Interfaces, The Principle of Usability). My view is that the philosophy of programming is a relevant perspective on brain-software. This kind of interpretation has found very little attention so far. Note that the subject of computational neuroscience is models of the nets, their wiring and their attractors. The subject of programming philosophy is programming languages, what affordances they provide, how to design them and so forth.

The Middle Land (Other software engineering approaches to AI)

Minsky was explicit about trying to find the land in the middle, between neurons and psychology. I see his approach more than anything else out there as a 'software engineering' approach. Compared to this here, he was thinking top-down, high-level on what kind of software entities are needed in order to make minds [Minsky 1986, 2006] (which he developed together with Seymour Papert). This is software engineering.

Minksy had the knowledge lines and so forth at the bottom, I have the neuronal ensembles and I think they yield a powerful and neuroscientifically grounded intermediate layer. Helping us create an explanation of brain-software from bottom to top.

Note that high dimensional computing was the only thing since the 90's. Kanerva's intro is from 2009 (Hyperdimensional Computing: An Introduction to Computing in Distributed Representation with High-Dimensional Random Vectors). In case Minsky knew about neuronal ensembles, I think he simply missed out on them.

High-dimensional data structures are versatile and expressive. They can represent symbols, situations, interpretations, pieces of knowledge and so forth.

[high dimensional computing addresses the challanges posed connectionism]

[neuronal ensemble memetics is not genetic programming Notes On How Neuronal Ensembles Are Different Replicators From Genes]

I consider Hoftstadter's ideas an attempt to talk about this land of the middle, but looking from the top, from psychology and the cognitive psychology of language and so forth down into the meaning levels. [Hofstadter and Sander 2013] Those meaning entities (cities of meaning, analogies/little situations) correspond to software entities in my view.

Debugging, Error Correction, Best Guesses, (Programming is Popperian)

The mind is a programmer, programming itself. Because who is programming the program? It is exploring itself. That only says sort of what is this thing now, it doesn't explain yet how.

You can't think about thinking without thinking about thinking about something.

  • Seymour Papert

I'm not entirely sure what the meaning of this is. But a possible meaning is that the ideas need to be first. You first need a bit of a guess, and then you can think about your guesses. Another meaning is when analyzing the, you need to have a first guess of how things could work. This is the meaning of The Science Fiction Software Engineering Approach and the Popperian epistemology of Deutsch that I find so joyful.

The idea comes first

  • David Deutsch

Gerald Sussman is one of the few actual philosophers of programming; Influenced by Minsky and Papert and the MIT AI lab16.

Funny, that Minsky considered himself "not a good programmer"17. While at the same time influencing a whole branch of thinking about programming.

Gerald says programmers must solve philosophical problems18, the method we use is a series of 'best guesses', with subsequent 'debugging until satisfactorily sufficient'.

A bug is a wonderful thing!

  • Gerald Sussman

This "debugging" notion is at the core of this programming philosophy. It says this is a method of creating explanation structures.

Minsky is touching on this notion [The Society of Mind Lecture],


There must be tremendous knowledge in the minds of people like Gerald Sussman, they must have many tricks of how to debug.

I.e. there must be methods of finding flaws and coming up with improvements of explanation structures. Minskies hunch was that minds (what I call brain-software here) must implement methods like this, and must have meta-methods of creating them.

Arguably, this is a Popperian epistemology. You first have conjectures best guesses, then you have a process of finding flaws, bugs, and improving the guesses. As a programmer, this might mean you realize that the space of problems you had to solve was larger than the current system of explanation covers; And a deeper foundation is needed.

This mirrors the scientific process of finding deeper explanations that explain more things (unifying fields).

Note the levels of analysis:

One is a model of what a programmer does. They create knowledge (the knowledge of what kind of program will solve the given set of problems), so they are using some kind of practical epistemology.

The other is a theory of cognition, that creative, knowledge-making entities must use an epistemology. And the idea that mind-software should use something like the epistemology of programmers ('making best guesses and debugging until partial satisfaction is reached').

There are necessarily two levels to this creativity: Firstly, it must make models to understand the world and secondly, it must grow meta-understanding, on which kinds of explanation-making work.

You might say The mind is software that programs itself, it must have ways of debugging its thinking. And it must debug its debugging, too. This must be a relatively stupid or mechanical process at the bottom, for it must be implementable with the wires that you can grow Darwinistically.

Note that the capacity of humans to reflect on their thinking might at first glance look like an impressive concept. But really, the magic was already done at that point. For then you have already crafted a universal thinking machine, and it being applied to itself does not yield any qualitative gain. This self-assembling programming or debugging we are interested in is the one that happens during development.

Kinds of Computation / Notations


Pardon me, Your Honor, the concept is not easy to explain – there is an ineffable quality to some technology, described by its creators as a concinnitous, or technically sweet, or a nice hack – signs that it was made with great care by one who was not merely motivated but inspired. It is the difference between an engineer and a hacker.

Judge Fang and Miss Pao in Neal Stephenson's The Diamond Age, or, A Young Lady's Illustrated Primer

  1. Church-Turing Completeness means first that all universal computers can compute everything computable.
  2. From McCulloch and Pitts [1943] we can assume that neurons are in principle universal.
  3. 'Universal' sounds grand, but the inverse insight is that we have an upper bound for their power, per theorem.
  4. Computer programming is not concerned with the (universality), absolute power of a computing system. It is concerned with what is idiomatic, what abstractions exist, what languages exist, that building blocks exist, it is concerned with the kinds of verbs a system provides.

    -> A central question of comparing computational systems, including brain-software is

    What is easy to say?

  5. Another way to see this is to consider that you could recursively express all possible lisp programs in a few lines of code. Or imagine you write assembly, you can always add the next instruction at the bottom. A program that generates all possible programs, including brain-software, is easy. This is sometimes expressed by saying All problems in AI and neuroscience are optimization problems19
  6. I think viewing a computational system from the lens of optimization is like viewing sex from the lens of health benefits. The alternative has to do with composition, harmony, good design - sensuality. A cognitive machine is efficient when it provides powerful abstractions to itself, just like good software is good because it is designed with care.
  7. Consider how programming languages and paradigms have different flair, and different strengths, and make different things easy to say.20
  8. When we say the primitives of the system, this is not derogatory. To the contrary. The so-called "primitives" are the fundamental building blocks that the designer of the system came up with. If it is a well-designed system, then the programmer can express what they want to say straightforwardly, in terms of the primitives of the system. This is the same concept as having nice building blocks, out of which something larger can be built. It is the harmony, the way the building blocks fit together, that is the property of a well-designed system. In other words, elegant systems are expressed in terms of their primitives, with straightforward swag, also called elegance. And the most powerful programs are self-evident21.

This is the meaning of "finding the building materials of cognition". Finding a computational system, a language, the fundamental data structures, or the verbs of the right kind - in terms of which cognition can build itself.22

When we say the computational properties of a system, or data structure. The meaning is what kinds of things does this make easy to say. That is on the plane of elegance, design, and software engineering. Not absolute power. And only incidentally has to do with its speed.

One view then, the view from the bottom, is:

The structure and function of the cortex and its nuclei is the kind of computation it implements.

The kind of computation the brain implements, I submit, is the layer in between us, the neurons and a model of cognition.

What you are asking for is "What is the computational notation that is relevant for neuroscience?"

What is the appropriate computational notation for neuroscience? The machine code - may be neural nets but what is the higher description? It might be mathematical, that might be the case. It might be like general relativity, which is mathematical, and very manifold-based. It might be something that is more like a computational language design problem. That's the thing I would like to be able to solve.

Steven Wolfram with Tery Sejnowski

Note that it does not say the algorithm, the output of this system is completely open-ended. The unended malleability of the system is one of its essential properties.

Magical Interfaces, The Principle of Usability

Audio diary: On Interfaces, Elegant Software, Fun With Ideas About Neurophilosphy, Comparing This With Minsky, Indulging In Speculations On Dejavu And Remembering

[evolving notes]

As far as I know, Dan Dennett was the only public thinker talking about how the mind creates a user interface. And it interfaces down, too. The user interface is not merely an analogy, it is a perspective of the fundamental nature of brain-software.

This view can be summarized in the idea that brain software must create a kind of operating system, (which I call a cognition machine), this operating system is capable of fulfilling the functional requirements of the brain - making explanations, thinking, behaving and so forth, by creating user-level software entities that use the operating system. The nature of the operating system, the user-level software entities, and what use means are some guiding questions for explaining brain-software.

Computational systems, from the view of a software developer, are not concerned with absolute power. Universality is precisely the notion that we don't need to worry about absolute power.

There is a wrong idea in software development currently that concludes that this means all programming languages are equally powerful. (see Paul Graham, the Blub paradox, Beating The Averages (for the origin of the term)). It cannot possibly be so, because programming languages are exactly the thing we build on top of universal computers. They are a system of expression that allows us to get more done with less effort, and this layer is somehow above the absolute power of universality. Arguably just as mathematics is said to progress with its notations and systems of thinking, programming languages and computational paradigms allow us to think different kinds of thoughts. And hence more or less powerful thoughts. What this power is, is one of the deep questions to answer. It is something that has to do with the so-called elegance of a system. This additional power beyond universality is the purpose of programming languages. Any challenge to this view can easily dismissed by the observation that programmers do not program in Assembler.

David Deutsch sometimes says AGI is the problem of software, not hardware. I.e. not a problem of absolute power, nor speed or memory. It is a problem of modes of creating explanations 23.

Since the brain is a universal [Turing, Church, Deutsch, McCulloch], classical computer [e.g. Braitenberg 1986, (see Pyramidal cell activity - The Gasoline)], it is this power, the software power, the one that makes programming languages useful that is the difference between brain-software and more primitive technology.

The structure and interpretation of computer programs, that is the analysis and understanding of what is easy to say; The space of language and interface design is how we compare computational systems.

My answer to Steve Wolfram's question above is that the relevant computational notation for neuroscience must be a computational language design problem.

We probably have to take a very broad perspective on computational language design in order to explain biological, self-assembled intelligence.

This was an intro to the analysis of what are computational interfaces?.

An attempt at defining user interface:

A user interface is a set of affordances, that constitute a kind of language, allowing the user, also called consumer of the interface to observe, transform, guide, modify, experiment with, or operate the computational state or resources of producer or provider of the interface.

This notion unifies programming languages, operating systems, mental affordances of brain-software, observability tools, shell prompts, peripherals like mice and keyboards and so forth.

We observe that computational interfaces are allowed to be arbitrarily powerful if the set of supported intents, also called the contract, includes a Turing complete programming language. That is the programmer interacts with their computer operating system and code editor to form an arbitrarily powerful system. This is currently not true of consumer user interfaces, which are thereby absolutely impoverished.

The notion of an interface necessarily divides the world in two. It is this splitting that allows the crafting of separate functional units. The same notion, but vertical, is called an 'abstraction barrier' 24, in 'stratified design' 25, we try to create a toolbox of explanation in one layer, the domain layer. Which is subsequently used by the higher layer, which might be the application layer. For instance, the basic laws of Newtonian physics can be seen as a system of explanation, with the primitives mass, direction, length, time (or something like that). In order to model a physics problem, we implement a description of this abstract, primitive layer and then express our problem in terms of the lower layer.

Somehow, splitting a problem into such abstraction layers, where a general layer is describing the kinds of problems26 we want to express. And subsequent layers use the lower layers in order to express the problem at hand is powerful.

This notion is also called the power of abstraction. That it exists is one of the things we can say for certain. I see a rhyme with this and the notion of elegance of explanations in theoretical physics (Deutsch) and with the technical sweetness of biological, engineering spaces (Levin). (For instance, the genetic toolkit of evo-devo creates building blocks, a language for exploring animal building plan space efficiently).

This is also called the problem domain, which is described in a domain model or domain layer and the application layer. Note here the relationship between the domain model and the application. It is analogous to the relationship between the user and the user interface.

In stratified design, as the name suggests, we are creating multiple of these layers on top of each other, like layers in a cake.

If part of the computer program is reaching too far below, skipping a layer of abstraction, this is called a 'level violation' and a sign that the system is not designed well. Similarly, if an abstraction layer is violating the contract of the interface, usually because there is more going on than expected, this is called a leaky abstraction.

Level violations and leaky abstractions point out to us that the ideal interface is powerful by allowing the subcomponents to talk in terms of contracts. This is reified in the following slogan:

I don't know and I don't want to know.

Rich Hickey

The meaning of this is, we can think of a component in isolation. If components need to know about the internals of other components, that is bad.


The philosophy of kinds of computing style and user interfaces has not made much progress since the 60s. If one looks at the history of computer user interfaces, one will find that Alan Kay, J.C.R Licklider, Douglas Engelbart and the early hackers of interactive computing and the internet have not only invented the user interface as we know it currently, they have built and envisaged systems with true dynamism. Conceptually far beyond current tech. 27

Such systems are operating systems and programming languages at the same time, they are crafted in layers of cake, and they reflect the true malleability and composability of ideas themselves.28

For this reason, calling brain software an 'operating system' is not derogatory, it does not mean that brain software is anything like primitive mainstream user interfaces. But it means something very cybernetic, that the space of possible systems we can call operating systems includes brain software. Appropriate programming analogies of brain software say that the space of possible programming is so large it includes the brain, not that the brain is so small it would fit into the current known space of programming.

[somehow something about how languages and interfaces are the same]

The comparative power of computational interfaces is an elusive topic and poorly understood. We know there exists a quality to well-designed systems, also called technical sweetness or elegance. It is currently understood mostly in the implicit technical feel, intuition or sense of aesthetics of programmers (and presumably engineers). What Paul Graham calls the The Taste Test [Succinctness Is Power], that is asking the simple question does this system of thought allow me to express what I want to express?.

I think most hackers know what it means for a language to feel restrictive. What happens when you feel that? I think it's the same feeling you get when the street you want to take is blocked off, and you have to take a long detour to get where you want to go. There is something you want to say, and the language won't let you.

The hunch is that there is an objective quality that has to do with the succinctness or ease of expressing ideas, in the case of programming languages that are in conceptual spaces.

"The quantity of meaning compressed into a small space by algebraic signs is another circumstance that facilitates the reasonings we are accustomed to carry on by their aid."

  • Charles Babbage, quoted in Iverson's Turing Award Lecture

David Deutsch's philosophy of objective beauty comes to mind, which ties together with explanation structure making. [Why Are Flowers Beautiful?].

Whatever this quality is, it is fair to call it elegance (technical sweetness) for the moment.

  • No objective measure of elegance exists. And new kinds of philosophical perspectives must be taken before we can do so.
  • Elegance has to do with the functional properties of the artifact. Elegant systems are robust, easy to understand, easy to maintain, easy to explain, easy to copy, they fulfill their function, they might be so self-evident that they are impossible to contain bugs,
  • Elegance does not have much to do with perfection. Perfection is necessarily a counterfactual, to which only approximations can be found (David Deutsch, Popper).

    Elegant software might even contain a certain amount of sloppiness, depending on the 'what needed to get done'.

    It will contain shortcuts and so-called kludges because time was expended more usefully elsewhere. And some kind of sufficient level of function is reached.

  • Elegance has something to do with 'getting stuff done'.

    The quote from Neal Stephenson's The Diamond Age, "signs that it was made with great care by one who was not merely motivated but inspired" might be misleading every so slightly. This great care is as much in the style and skill of the hacker as it is in the artifact. It is the fact that they can produce such a robust and well-designed system fast, that is the true elegance.

  • Stratified systems tend to be elegant. That is finding a language of abstraction, which perfuses a space of problems, the problem domain with understanding.

    This is similar to the difference between Aristotelian physics and Newtonian physics. Where one is a patchwork of special cases and the other is a minimal, sufficient, abstract description.

The User And The Interface, Brain-Software

Once the view is taken, it is obvious that brain-software must create some kind of advanced, self-assembling, 'magical' user interface.

This interface is so immensely good, that we might even take it as a benchmark of what a good interface is. We do this when we say this tool is like a glove on my hand, this is an extension of my arm, this operating system is doing everything automatically (meaning your intent is picked up and has effects seamlessly).

It is this seamless and prescient property, this "knowing on its own", this competence of brain software that I would like to call 'magical'.

The way I can decide to move my muscles and they move, without having any clue of how they move. The way I can decide to move my eye to an object and appreciate its effect on my mind - without the slightest hint of a clue of the details (i.e. the leakiness of abstraction is low).

The unlimited malleability and composability of 'imagination' or 'working memory' states. (i.e. the computational primitives are well-suited for expressing ideas and creating explanations).

'Bringing to mind', 'remembering', 'moving attention' and so forth are user-level internal operations with the same property of usability.

The user or users must necessarily be brain-software entities, too. It is a system that self-assembles into containing competent user-level entities, using elegant interfaces to producer-level software entities in the system. The computational paradigm must be extremely dynamic, that is the content of the software is being created and transformed on the fly. The content must contain the equivalent of procedures, too.

We can assume that the power of abstraction holds for brain-software, too. This means it would be level violations or leaky-abstractions if user-level entities were concerned with the details of producer-level entities in the software. This "detail ignoring", which is the nature of good interfaces, is what Dennett labeled 'the user illusion'. I.e the user doesn't know how the producer works (and shouldn't know).

When McCulloch and Pitts said the brain is a universal computer, this was not an end conclusion, but the starting point. Because different computational systems have different amounts of power. That is not their absolute power but something else. Their elegance, their swag, their expressivity, the beauty of their abstractions. Their ability to get stuff done.

Memetic Engines Create Competence Hierarchies Up To User Illusions

It is pretty much obvious now that the brain is an information-processing device [McCulloch] that creates user illusions [Dennett].


Vast amounts of tiny programmer memes that figure out how the computer they run on works, not because they set out to understand how it works, but because of the many that didn't mean the right thing, the ones that mean something useful is selected in a process of natural selection. In a substrate where your meaning is your connectivity. And the best memes are the ones creating magic interfaces for confident wizards. Creating user-level entities that use the computer with swag.

Audio diary version: Meme Machine Heroes.

If the brain has brain software running, we can go and understand the nature of this software, too. It is clear that it is a valid viewpoint to say 'the mind is a programmer, programming the computer it runs on'. Of course, whatever is programming cannot be competent by itself. It must bootstrap from simple elements. This by itself only kind of says the general idea of what is true though, it doesn't yet satisfyingly explain the mechanisms of what is this program, what the elements that program it etc.

I had this activity flow of the cell assemblies (see below) in mind and wondered how I could put this one level further up into the level of meanings. I came up with a thought experiment, called Banana maker matrix algorithm.

This is only 1 perspective of course (out of 12 required). But this really nits it together from bottom to top (both ways) for me:

Suppose we already have a nice meme engine, capable of trying out vast spaces of possible memes (and fast) and selecting them after some useful criteria.

Let's pretend for a moment this machine implementing a matrix algorithm, a big imaginary planet with a cyber-highway along the equator that makes you pass ten thousand cities, twenty times per second. 29 So it's vast, but it's possible to traverse it fast.

Suppose further that there are simple agents in the system, let's pretend they are super-competent action heroes, who need to act on a clock. These people are very, very fast. And very competent and confident. Simply because we tried out many action heroes and the ones that were not competent and fast were discarded [Darwin, Dawkins]. Let's suppose for a moment that the memetic engine can select action-hero short movies. In effect you, as an action hero, have a short amount of time, to please the memetic engine in some way. Of course, you can hack its mechanisms, too, if you are competent enough.

You get created (some activity flow makes you spawn) in a place in meaning landscape that you already have supplanted with useful tools and so forth. Part of what a good meme does is plan for the next iteration of its existence.

The other thing you do is quickly go to some city, where many generally useful memes provide information for more derived memes. The most successful memes will be part of vast societies of contributing, more generally useful, memes. One might imagine some other meme leaves a sticky note on a wall in a city. They don't know why they do it and you don't know why they do it. But information can flow, and more useful information flows are selected. Say the meme-machine selects action-hero timelines, including the cities they move through.

Here is the next thing: This matrix has certain competencies, like spawning new cities and roads or manifesting a banana.

Everything that the computer can do, the matrix can do, and the memetic engine might also try it out.

Now and then the matrix will simply manifest a banana in a place, just to try out what happens. Kinda implementing an 'everything possible' mechanism.

Now you are a competent, optimistic action hero. And you trust in the magic of the matrix. 'I hold out my hand and there will be a banana'.

Again, the 2x2 matrix of optimism and competence, the optimism-drive, produces the following:

All the memes that are optimistic and pretend they know how to use the magic of the matrix, have a chance to survive. All the memes that are pessimistic have no chance and are discarded.

Optimism is not the only thing you need, you also need competence. In this case, it is the competence of the rest of the computer to provide you with what you needed at that moment to be a successful action-hero wizard (with swag).

     banana!   I don't know.
   |   S     |   X    | competent
   |         |        | (the information flow is capable of creating a banana, if you ask)
   |   X     |   X    | incompetent
   |         |        |

S - memetic success
X - discarded

Commanding the magic of the computer without knowing how the computer works.

This holy overlap of competence and optimism is how there is a magic piece in this whole thing now. From the thousands of memes that said 'There is a banana', there is one meme action hero that confidently puts out her hand and says 'there is a banana', and the system tried out randomly what would happen if there is a banana in that place - this short story movie timeline then was especially useful to the system and selected. Because for some reason a banana is what this hero needed to be successful in its matrix short-term movie.

Nobody knows why: The memetic engine sees an information flow that worked well, and the hero sees the matrix that made a banana for her, similarly, the banana memes are just information flow pieces that had a chance to be active, and they do what memes do. Trying to be active more. The interface between the user and the lower software world is something like 'competent query' and 'competent result', where both are selected memetically afterwards, so that the matrix becomes more and more magical to use (obeys every whim so to say, and is very useful, retrieves from midterm memory for instance), and the user-level entities become more and more competent, artistically elegant, on-point, demanding, confabulating, and confident.

It is that the memetic-engine favors competent information flows. And importantly that means that high-level wizards, artists, and action heroes move confidently, using the computer in ways they don't understand. For if they would need to understand, some other meme would simply be faster than them. This creates competence hierarchies, where the higher agents in the system delegate the lower agents. To put it the other way around; The more generally useful memes have a great memetic strategy - be so useful that all the higher-level agents can't help but incorporate you into their short-movies. So that you are 'on', memetically selected, in many situations.

The memes that will be most competent are the ones that use the magic interfaces of the system, not knowing how the computer works. They don't have time to know how the computer works, they are busy with creating cognition.

When a high-level meme delegates lower-level memes to provide it with information, we can imagine little hacker wizard memes, in the substrate of the matrix so to say, that either figure out how the computer works and provide you with competence or not figure out how the computer works. Then they are discarded by memetic selection. Of course, they don't know how the computer works, they either represent a wiring that has meaning, or not.

The memes don't know what they mean when they start, it is only afterward that the meaning that made sense is leftover. This is the strange inversion of reasoning of Darwin's idea. Taken seriously for software-level mechanisms of computers that run meme-engines.

On the highest levels of this hierarchy, you have memes that are so competent, that they command everything that the whole computer can do with precision and artistic swag.

This then looks like a mechanism for a memetic machine to perfuse its computer with competence, to build information processing hierarchies that will use the computer, completely and competently.

Of course, the matrix machine is only a thought experiment, the matrix algorithm that runs on human brains is the one we know ourselves. What I like to call the 'cognition machine'.

I submit that this magic interface is the same class of software as our magic interface. Brain software is a magic banana maker matrix meme-engine.

And the brain has something to do with navigating the world as an animal. Not because in principle all meme machines are about that, but because this specific meme-machine evolved biologically. (So the wires that shape the memetic landscape are biased to be about navigating the world and being an animal).

A meme machine can create user-level abstract entities that have magic interfaces to the rest of the computer. Fulfilling the basic requirement we had for a brain-software.

When you go and wonder what is brain software, you probably will come up with something that sounds a bit odd at first, otherwise, you would be doing something wrong.

This puts half of the problem onto the machine hardware. It's how sick your sense organs are that allow you to create a world inside the mind. How well the thinking goo is tuned to lock into nice ideas while still having the dynamism to consider many possible things and so forth.

You don't get away from the fact that a neurophilosophy must talk about 'how the brain relates to the world'.

Those memes are allowed to get arbitrarily tiny, down to single-unit (whatever that might be) stupid computations. It's kind of sweet (technically sweet) to have a mechanism where arbitrarily abstract entities are allowed to coexist and refer to each other.

Shoulders: Darwin's strange inversion of reasoning, McChulloch to say that the brain is an information-processing device, Dawkins abstract evolution to replicators

The other important ingredient is the power of interfaces. This is the power of what software is in the first place, especially pleasing to a software developer.

An important aspect to keep in mind is that the brain seems to be doing these high-dimensional, highly parallel computations. This is not a synonym for 'very powerful computer' but this has a very distinct computer science and cybernetics to it. This is the topic of the computational models of things like cell assemblies, that I explore here and elsewhere. (Current work). The main point is that 'everything possible a little bit' is a valid thing to say in such a computational system.

Don't limit your thinking to the computers you experience, only expand it, never make it smaller.

Similarly, with 'software' I don't mean the buggy, half-working user interfaces of current primitive tech. There is a larger view, that we are only glimpsing. It is deeply philosophical and would answer questions about the nature of abstraction, live the universe etc.

A cybernetic level of understanding of what software is, is just in its infancy.30

With this space of thinking you can muse about civilizations of memes, that build vast stores of knowledge. And archeologist memes going through the libraries of eons of accumulated knowledge.

Harmonious memes, cooperating memes, that never truly meet across times and space and yet, together contribute to the budding ideas, that only artistic memes and old wise-wizard memes can glean from the flow of time and history.

Social attractors of a kind that are bigger than a single meme on its own.

Multiverse Meme Machine

What about the reality where Hitler cured cancer? The answer is don't think about it.

Rick Sanchez from 'Rick And Morty'

As a variation of this thought experiment, let's say the brain is creating a virtual multiverse. In our real multiverse, we don't need to worry about all the histories that are a chain of magical things happening.

Like the history where all objects turn into bananas when you point at them and say 'banana!'. We don't need to worry, because, for every universe like that, there are many more where it isn't.

Not so in the brain's virtual multiverse. Because some histories survive better than others. If there are magical universes that work, the memetic engine will roll with them.

I Imagine the counterpart of Neo in the matrix, he is moving around in the matrix and stopping bullets with his hands. Perhaps at the start, in most virtual worlds something incompetent happenend. Either he didn't think that he could stop bullets in the first place, or the matrix didn't make an information flow that meant bullets stopped.

The histories where Neo is stopping bullets, using what he percieves to be the magic of his reality, could at first be very few. Like the real histories in our mutliverse, where everything happens as if some magic system is actually being implemented.

They are like a tip of an iceberg.

In the second step, the histories where Neo and his magic are stable, are the ones we see around.

If information flows are stable because they implement software with a magical interface to users in a matrix, so be it.

'Try Out Everything' / Memetic-Engine Mechanisms

Intrinsic Firing Rate

(defn intrinsic-firing-rate
  "Applied after the threshold model.

  Says that neurons are allowed to fire at each time step with chance `intrinsic-firing-rate`.
  [{:as state :keys [n-neurons intrinsic-firing-rate]}]
  (update state
          (fn [activations]
              (or activations #js [])
              (mathjs/filter (mathjs/range 0 n-neurons)
                (fn [_]
                  (< (mathjs/random)

Give your neurons an intrinsic firing rate!

This is a substrate-level mechanism to allow the network to try out everything a little bit.


Figure 3: Neuronal area without any inputs, but an intrinsic firing rate. Ensembles come from the geometry of the network.

Skip Rate

;; ================
;; Skipping Rate
;; ================
;; Neuron 'failure rate' to fire.
;; This is biologically plausible. At first glance, this looks like an imperfection,
;; but I am reasoning that this would modify the dynamism of the substrate.
;; Higher neuron failure rate is similar to attenuation, it makes the ensembles spread more
;; and be less stuck. But high attenuation makes them move around, whereas the skip rate makes them spread more
;; if applied before the threshold model, and simply smaller if applied after the threshold model.
;; Intuitively, this means you might find other attractor states if you have some neurons that usually strongly
;; contribute to one interpretation, now you have the chance to find a new interpretation.
;; This has been described under the topic of 'assembly shift' [insert paper]
;; This could be applied before or after the threshold model, with distinct results.
;; Skip rate is counterintuitive but useful! And might (or not) be implemented 'explicitly' biochemically in brain.

  (defn neuron-skip-inhibition
      "Give every neuron at each timestep a chance to =skip=.

      This is an inhibition model modifier, applied `after` the threshold model.
      [{:as state :keys [skip-rate n-neurons]}]

Yes, this is useful. And these 2 together give the ensembles the chance to move around and so forth. (Also called assembly shift). Finding new connections between sensors and motors that nobody ever dreamed of!

We can imagine a neuron that fires together with another ensemble, through plasticity, it will be associated more with the new ensemble. If it now also skips when its old ensemble fires, this neuron will shift, and migrate to the other ensemble.

[I'll deploy my playground so you can play with those]


paper: 'assembly shift'

Random Orchestration? Random Commitment?

(Higher order memetic engine mechanisms).

Orchestration and commitment in the sense of Speculations On Striatum, Behaviour Streams.

Random Perspective?

Perspective mechanisms is a hypothetical class of mechanisms that simply make some random inputs to the neurons. Thereby modifying the attractor landscape dynamically.

If we also take some neurons and designate them as perspective effectors (p-effectors), then you can see how this the system can learn to dynamically modify its landscape.

Consider the case where there is a single large concept in the user's mind, like pizza. If you know how to press the p-effector buttons, you can play around with random perspective lines, until a landscape is reached that splits the concepts of pizza.

That is, you stretch and form the meaning of landscape so that you highlight some dimensions. And this new dynamic meaning landscape is for instance well suited for highlighting the space of pizza toppings. Vegan cheese, tomatoes, paprika, mushrooms. Where before the was the concept of pizza, there is now a nuanced view of the kinds of possible pizza.

A similar mechanism is labeled 'attention' in contemporary machine learning.

If 'doing some random things' to your p-lines is a possible thing to do, then you can evolve memes that are good at using this move for you.

The cognitive user can then create certain 'thinking' situations, that the rest of the system is allowed to fulfill, not only using the neurons but also such higher-order orchestration mechanisms.

Note that the cognitive user would simply create the situation 'zoom into the concept of pizza until I see its constituents'. It is the nature of having 'confabulation and subsequent best guesses until a stable interpretation of the situation is reached that makes the system stop thinking' as the mode of operation, that the goal state is in some ways pre-determined, but vaguely so. The shape of the situation is set, and the thinking machine goes and pattern completes the details, using whatever buttons are available.

I think we see the failure modes such as a layer of mechanism in things like 'tip of the tongue'.

Note that the intrinsic firing rate can be seen as the simplest perspective mechanism: that is everything possible is possible with non-zero chance evenly.


The Principles of Biological Programming

This does not exist yet.

There is the concept of a 'tech stack' in programming, which is the notion that one has different technologies that one puts together at different levels of concern.

For instance, one might choose a general programming language, and on top of it decide on a router library for one server and so forth.

We don't do it yet (see Sussman 'We Don't Know How To Compute!'31), but when we advance 'computational thinking' into 'biological computational thinking' (see also Micheal Levin), perhaps there is in addition to a tech stack, an emergent layers stack or something.

That is, you reason about the emergent properties of your substrate. And this emergent property stack is what you program; Even though your formalisms represent the substrate level only.

Micheal Levin says you are working with an active substrate. Biological programming then would be a kind of programming where we create substrates with certain competencies. This view from above, which says not what the elements are doing, but what the substrate is doing, or what its purposes are and so forth, is a deeply cybernetic thing I think. At higher layers of organization of the system, you think in terms of mining the competencies of the substrate. I think for many biological substrates that means to some extent you simply give it space to grow and become stable, whatever that space is, is what that means.

As biological programmers we are taking the view of the genes so to say; To say what machinery and substrate will produce the kind of machine we have in mind. Perhaps biological programming is substrate-oriented, but note that the way the substrates are used by additional layers of organisation is just as important.

It is the nature of abstraction, that the layers above necessarily can be implemented in infinite ways at the bottom. I don't think that there is something so deep about neurons, they happen to be the way biology is implementing an idea-making machine. But a mature computational theory of biological intelligence will be able to say that there is a whole class of formalisms that would get the stuff done.

Perhaps you can develop a paradigm that is inspired by amebas moving and merging and so forth, which would provide just as nicely a substrate layer as the neurons do.


  • Micheal Levin.
  • It is arguably in the spirit of Smalltalk and hence Erlang and actor-based systems to formalize elements out of which a kind of competent substrate is made. It is not a coincidence that Alan Kay is a student of biochemistry, and then came up with Smalltalk.
  • Sussman "Progenitors" (He talks about them , I think there should be papers on it?)
  • Chemical Computing is an experimental programming paradigm similar to the biological programming here.
  • Note that 3 ideas sound similar and are very different things:
  • 1. simulating and analyzing the domain of chemistry in a computer, this is an application problem, that is done by chemists and computational chemists to say things about chemistry.
  • 2. Making alternative computers out of chemical soup things or something instead of electrical computers. Such projects would give us completely new kinds of computers, with different strengths. It's interesting just for the imagination stretch it provides.
  • 3. A programming paradigm where the formalisms are inspired by analogies to chemical reactions, 'molecules', 'catalysts' and so forth. And only 3 is the sense I mean here.


How is this different from machine learning?

Arguably, machine learning is a biological programming, too.

McChulloch, Pitts, Hopfield, Hinton, Sejnowski all where inspired by neuroscience.

But I see much potential for new perspectives.

Emergence / Function

It is a physics concept; Bringing it into biology is I think a relatively modern notion. [who talked about this first? Maybe Hopfields physics perspective on neuroscience 1982?].

[Sean Carrol is a communicator about the concept of emergence]

[Daniel Dennet: Free floating rationals. These rationals already exist, they do not emerge from biology.] (But maybe a theoretical physicist would argue that the whole explanation stack needs to be able to emerge those rationals, too).

[Kevin Mitchel emphasizes higher-order organization, it exists on its own, not emergently]

[Chiara Marletto gives an example: To understand the bits of a computer, one answer is the rules of the circuits, but another answer is the program that is running.]

These questions are arguably at the heart of what the cyberneticians of old were thinking about.

Biologies explanations are high-level already, called structure and function. It is the poetry of biochemistry, physiology, organismic biology and so forth.

"The function of the heart is to pump blood" is immensely useful.

I am happy with using this solid biological reasoning on the functioning of the system wherever it suits me.

[Stephen Wolfram calls this computational reducibility. That is you can make explanations about a system without needing to compute all the parts].

Marletto asks us to consider the notion of an adder. There is no perfect adder circuit, the wires of electrical circuits might get damaged by heat and so forth.

Our circuits instantiate the abstract, counterfactual notion of adder. It is real and not real at the same time. It is only useful because the notion is perfect, but because of fallibilism [Socrates], we will never instantiate a perfect version of it. Similarly, a biological system would never instantiate a perfect blood-pump. It's almost nonsensical to wonder what that would mean. Because it would mean knowing all future science, too. For all evolution knows there is a better way to build heart muscles when one is exploiting quantum effects or something.

This is the status of the function biological systems. For this reason biologists don't have trouble accepting that counterfactuals are essential for explaining the world.

It makes sense to me to say that the function of the brain is to create brain software, and this has the same status as the abstract, counterfactual adder notion. Just that this time, the program has a slightly different flavor. But there is a kind of software that is being approximated. It is a program with a self-assembling development plan, yielding a range of late-bound content. The content, in turn, is software [Von Neuman, McCarthy, Minsky], which enables a dynamic user interface [Alan Kay] to a user, which also must be self-assembled.

This is why you are not done explaining the mind when you explain the computation that the brain is doing. Because it is the organization of the computation, how it uses itself, that is the actual function of the brain.

This is mirrored in the fact that we can draw an abstraction barrier between what I call a cognition machine, or magical user interface, or the operating system of the brain, and its implementation. The implementation can be neural nets, hypervector framework, or self-modifying Lisp procedures. It just needs to be a generally powerful computing framework.

The Layers So Far

The neuronal ensembles are great because they are the first layer up from the neurons. They are explicitly a theory about what the substrate of neurons does, they are emergent from the properties of the substrate.

We can study them the way we can study magnetism, which doesn't come from atoms but from many atoms.

Many still asks you to count the elements in a way. But the thing we study is something else than the elements, that is why I want to call it the substrate.

I: Mechanical Level

Ia): The substrate and the wiring

     neurons      -------------  thalamus, striatum, claustrum, ...?
     | ^          |
     | |        |||                 + [ flash drive ] (Hippocampus)
     +-+--------+++                 + ? ... gadgets
       |        ||
       |        ||
       |        |+---->
-------+        |

  • The neurons, their wiring, how they are connected to the outside world, how they are organized in nuclei and so forth
  • The orchestration, which I label the machinery that creates, amplifies, extinguishes, composes (perhaps) activity

  • biological cybernetics (e.g. Braitenberg vehicles and how to build them)
  • neuroscience

formalisms that describe the layer:

  • Conceptrons (hypothetical, Braitenberg)
  • Assembly Calculus (Vempala)
  • Computational models of neuronal ensembles
  • The formalisms that take the ensembles as their center of explanation should center on the notion of the substrate, because we think in terms of the emergent properties of the neurons, per definition.

It might be called the machine level, neuron level, wire level, circuitry level, or substrate level.

Ib) : The Neuronal Ensembles

It might be called the ensemble layer.

 |  |  +-----+
 +--+  +-----+
    |  |


  • Consider magnetism: The atoms (the elements) are not magnetic, but the substrate is.
  • Ensembles? See Summary So Far, Cell assemblies and Memetic Landscapes
  • The short version is that they are self-activating subnetworks, they stay activated by themselves. They are the entities on top of the neuronal network, like looking at the image instead of the pixels (Rafael Yuste's analogy).
  • I.e. the neuron doctrine is replaced by the neuronal ensembles.
  • In computer science terms, they are a data structure emerging from the previous layer of the neurons.
  • A high-dimensional computing framework made from the network and its activity.

If you have self-activating pockets or subnetworks of neurons, you can activate a piece of the network independently of sensor input, that is the doorway to mentality. That is a piece of information that can refer now to something in the world.

  • Lorente de Nó:

    The active neurons are arranged in convergent chains of relatively simple composition, which may be called multiple and closed chains through which impulses circulate. The multiple chain of neurons is the elementary unit of transmission; it supersedes the classical reflex arc with a fixed number of synapses (Lorente de Nó, 1938b).

  • chains 32
  • D. Hebb 1949: Cell Assemblies. (Although Hebbian Learning is not actually what cortex does (Alternative Plasticity Models).
  • John Hopfield 1982: attractors in Hopfield nets. (Hopfield network). Inspired by Enst Ising's model of magnetism. Fascinating concept: Explaining magnetism was a theoretical / philosophical breakthrough, it explaining the concept of emergent properties. The energy is a precise theoretical notion, representing how stable the state of such a network is. Emphasis on pattern completion: Imagine a ball falling down the energy slope into an attractor. If you put the ball anywhere on the slope of a valley, it will roll there. That is called associative memory. Observe that attractors are points in high dimensional spaces.
  • Valentino Braitenberg and Günther Palm 1977:

    If cell-assemblies are the terms of the cortical logic, there must be a device for the discovery of the ignition of a cell assembly and there must be the possibility of extinguishing cell assemblies. There must be a timer that regulates the succession of activated cell assemblies. There must also be a mechanism of attention that inhibits entire regions of the cortex once something interesting has ignited somewhere. Finally, there must be mechanisms that connect the rather cool cortical memory by correlation with the more impassioned system of inborn patterns of behavior, which is also subject to learning of some sort.

    Emphasizing that the brain must make use of the network and it's activity. If the activity is representing the pieces of mentality, there must be devices around the cortex, that are creating and orchestrating the activity.

    Cell Assemblies are heterogenious: They have multiple centers (attractors). Increasing the excitability of the substrate (globally), will ignite the halo of the assembly. Neurons that are associated with the assembly, but not part of the center.

    Decreasing the excitability will find the best connected center of activity. This thought pump mechanism is a member of a class of activity management devices that we can consider.

    These can then produce trains of thought, thought sequences, or refinements. The network and its orchestration can be in a dynamic state of refinement until it settles into some Hopfield attractors. But perhaps more often than not, the situation will have moved on by then, creating yet again the next evolution of the situation. (On obvious failure mode is getting stuck: Speculations On Striatum, Behaviour Streams).

    This query and result, this vague idea to a detailed idea, might the the fundamental eval-apply of brain software. This would mean that the means of abstraction (how to make procedures) in brain software are the same high-dimensional data structures again. (Context Is All You Need?). You can see how this makes sense by considering the state of the network (the situation) determines which activity is supported by the network (the interpretation). A vague idea is then allowed to be a query, where the pattern complete of the framework is the result. Or what Braitenberg labeled the trains of thought. The possible continuations of a state of the network. (This is a way to get to thinking, acts of attention, mid-term memory query lookups and so forth Random Perspective?, A High-Level Story for Midterm Memory Retrieval).

    It would be fair to say that the halo of an ensemble then is something like a cloud of possibility. This makes the ideas sticky playdough puzzle piece balls with solid cores and doughy outskirts. Since the thought-pump mechanism takes 2 neuron timesteps, far-reaching thought jumps are done fast (because parallel).

    Also note that we can consider assemblies of different shapes and sizes, some with temporal structures that can represent sequences or plans. Some timeless, representing images.

    Up to the complete ensemble, which is the activity in the whole brain, given a time window. This is a framework to consider the whole brain (and its connections to the world) at once with the perspective of the activity flowing inside it.

    (Tübinger Cell Assemblies, The Concept Of Good Ideas And Thought Pumps)

  • Moshes Abeles 1982: Synfire chain.

    Emphasizing the notion that synchronous activity could travel together through the network.

    Like dominoes made from clouds of associated neurons.

    Also, the electrophysiology makes it hard for an activity to stay. This must be fundamental, to solve epilepsy.

    A compositionality machine realized by a hierarchic architecture of synfire chain.

  • Rafael Yuste (also 2): neurophysiology, finding ensembles and some of its properties.

    Finding that 'artificial' ensembles can be created by activating groups of neurons via optogenetics.

    That they are not created by Hebbian Plasticity in Cortex.

    That single neuron stimulation can alter the behavior and presumably the perception of mice.

    Learning to play 'the piano of the brain'. [see Yuste talks below].

  • György Buzsáki (4):

    What a cell assembly is depends on the reader and the time window. With this perspective, neurons count as cell assembly, even if they don't activate each other, but when they contribute to the firing of a reader neuron.

    Neural Syntax: Cell Assemblies, Synapsembles, and Readers

    Cell assemblies do lateral inhibition via inhibitory basket cells. Implementing an Alternative mechanism.

    Cool neurophilosophical and empirical stuff.

    Neurons have intrinsic firing, this allows the internal ideas to be matched via action/experiment to the external world. (The brain always already has ideas about the world, it is not an empty receptacle that is filled with sensor data).

    A gamma cycle can be seen as a letter in the neuronal code. This is what I call an atom, or elementary data structure, which is composed in composite data structures.

    This would mean the atomic data structure is encoded in a 10-25ms time window. This overlaps with many physiological constants. Like the integration time of pyramidal cells and things like this.

    Then, there must be some scheme for composing the letters (the means of combination from 5) Perhaps something like Composition / Neural Syntax.

    [hippocampus, mental navigations, thought and externalized thought (what I label technology and cultural memes), hippocampal sequences are already there, which fits the 'the ideas must be first' idea.]

    [I was wrong to label the hippocampus a flash drive, it is an active element, providing pre-allocated trajectories to the neocortex, useful for both past and imagination and so forth]

Forgive me for saying it so but the neuronal ensembles are theory of mentality.

If sensor input is reliably activating some of these internal notes, we can say that the notes stand for the sensor input. The system now has the ability to epxress the world in terms of it's sensor inputs. Note this includes the effectors of the system, too. In this way action and perception together are allowed to represent expections of the world. If you move the eyes here, you will see x, and x has a color, too.

neuronal ensembles, built by the coactivation of flexible groups of neurons, are emergent functional units of cortical activity and propose that visual stimuli recruit intrinsically generated ensembles to represent visual attributes.

Visual stimuli recruit intrinsically generated cortical ensembles

Then, you reactivate them during sleep and so forth: Visually evoked neuronal ensembles reactivate during sleep

By activating ensembles, you can change the animal's perception (the most parsimonious explanation of the stuff Yuste did here):

Playing the piano with the cortex: role of neuronal ensembles and pattern completion in perception and behavior

In these experiments, they were able to trigger a learned behavior in mice by activating single neurons

ensembles appear necessary and sufficient to trigger visual perception

Great Yuste talks, summarizing the field and its history:

Cool overview of current neurophysiological topics:

I get the impression that systems neuroscience is discovering, or already contributing new perspectives, on the neuronal ensembles in the form of thalamocortical loops (and similar modules). The common language there is then the 'attractor states'. Input Circuits And Latent Spaces / The Games of the Circuits.

formalisms that describe the layer:

  • Observe that per definition, the ensembles layer emerges from the neuronal layer

Names: Neuronal Ensembles or Chains (Lorente de Nó), Loops (perhaps resonates with software developers), Reverberations, Synfire chain (Abeles), self-exicitatory subnetworks, surviving activity, activity replicator, perception state, mental element, Cell Assemblies (Hebb), Neuronal Assemblies (G. Palm), attractors (Hoppfield and nowadays dynamical systems people), thalamocortical loop (as an instance of one), The notes that create the symphony of the mind, The alphabet of the brain (Yuste), a datatype (Vempala), a high-dimensional datastructure (Kanerva would agree), a pattern completion or associative memory datastructure implementation

II The Software Level

This theory of mentality says what is the mechanism that is instantiating the computation of the brain, but a computation is a mechanical instantiation of something abstract. It says what are the primitive data structures and so forth, but it doesn't say how the brain is using those data structures to create explanations and navigate the world.

We label, rightly so, the mind or cognition as the software of the brain. This software needs to self-assemble because there is no programmer at hand.

I would argue that the brain's function must be this software and its entities. Otherwise, we would not feel like cognitive users running in a virtually created world with body and mental affordances.

What are the principles of self-assembling software? What kind of self-assembling could create something like our cognition machine? What kind of mechanisms does the brain need to implement in order to facilitate the growth of such software?

IIa: The Memetic Engine

+-X--------X-+  random orchestration
+-----X------+  random commitment
 +----------+   i.e. random flash drive lookups,
 |   X      |        random acts of attention, ...
 |          |
 |    X     |
 |X      X  |  intrinsic firing rate

memetic engine

  • Hypothetically, you add an intrinsic firing rate and tada, out comes an 'everything possible' machine; The memetic engine.
  • Assuming (plausibly so) that orchestration, commitment and so forth are made from neurons, they all can be a little bit possible.
  • I.e. your neurons all fire randomly a little bit. If you have an attention nucleus made from neurons you will also pay attention in random ways a little bit. (not saying it works like that, but then again, TRN the gate is sort of doing such a thing.)
  • Philosophically, there is something deep I suppose: There is an activity that comes from the sensors, and then there is an activity that comes from yourself. From this, the system has the chance to grow a relationship with the sensors (and effectors), inside its high-meaning spaces.
  • We can see this by imagining a neuron firing randomly. Usually, this doesn't do anything because there is no support in this space of meaning from the sensors and the network. But if there is an ensemble in the vicinity of a sensor neuron, the neurons firing in between allow the ensemble to 'shift' and represent the sensor inputs. And so forth.
  • In other words, we make the substrate more dynamic, so that the ensembles flow around and have the chance to represent different things.

Observe that we have crafted a substrate with wires and gadgets, that is eager to represent sensor data because the data structures that have support from the activity of the sensors are the ones that are on.

From the perspective of the memetic engine, we can see that there are different 'complete ensembles' at each time in the system. I don't know why but I like to label them activity flows in this context.

+--+--+    +--+--+   +----+          derived meaning level
|  |  |    |  |  |   |    |          i.e. cortical 'association areas'
|  X  |    |  O  |   |  O |
|  |  |    |     |   |  | |
|  X  |    |  X  |   |  X |
|  |  |    |  |  |   |  | |
+--+--+  , +--+--+ , +--+-+ , ...
 activity flows


              Darwinian wires
             ++ using heuristics

We can observe that whatever activity flow is stable, will be more around. And whatever activity flow is more around affects how the network grows (via the plasticity of the network).

Perhaps somehow a complete 'harmonic'(?) interpretation of the sensors is most stable. Something that stretches meaning and sensor level and makes a good explanation structure. I.e. the meaning levels will want to be as stable as possible, and this is useful because they will represent actual causal structures of the world by doing so. (Getting A Visual Field From Selfish Memes).

If the ensembles have all the chance to inhibit their alternatives, (via thalamic layer 6 neurons), whoever is fastest and most competent, wins. Because they go and inhibit their alternatives. I.e. whoever is active will try to make the machine go into a stable attractor state with no way out.

It is precisely the job of the machine (of the genes crafting the machine) to even the playing field. For instance by building in a restlessness, that forces the animal to move now and then. Or by having attenuation on the substrate, so the ensembles are forced to play longer games, in sequences. (Attenuation).

There is one other bag of tricks the genes have available: They can put some hardcoded heuristics like 'warmth is good', 'Mum smiling is good', 'cold is bad', then if you have a hypothetical nucleus (or Darwinian Brain) that can give a value judgment, you can build reward mechanisms. A wire that is "on" from some reward situation is what I call a golden wire. (Also: joy lines).

Golden wires can become about the machine itself. For instance, if it is possible to detect a situation where the system was making an explanation structure parsimoniously, in a short amount of time or with less amount of excitability perhaps, then this situation can be rewarded, too. (Why Flowers Are Beautiful, Elegance, Explanation, Aesthetics)

Whatever the rules are, we can analyze the entities in terms of abstract replicator theory [Dawkins, Deutsch]. It is whatever activity flows are stable and successful, including pleasing the reward heuristics, that will survive the most.

These activity flows will have a structure and function of their own. (I say they are a new kind of biology).

Names: everything possible machine, conjecture maker, variation maker, idea mulitverse algorithm,

IIIa): The Living Software

[vague ideas]

The activity flows of the meme engine have an inner logic. Why are they more stable than other flows? Because for every interpretation the network represents, they are supported more than other flows.

We can say that the complete ensembles are competing for stableness and connectedness. Ensembles are allowed to have temporal structure, so their activity is allowed to flow across neuron timesteps. That is when we bring to mind the sequence "A, B, C, D, E, F, G,… ", even better if together with the melodie (more association, more support from the network).

Observe that we can say that such an ensemble has a starting point, the A and whatever notions are preceding it. Igniting the ensemble somewhere in the middle is harder, presumably because there are not many associations leading to this situation.

The complete ensembles

  1. Are made from harmonic and symbiotic sub-cell ensembles
  2. Stretch meaning and sensor levels (per definition, they are the maximal subset of the maximal ensemble active). (Another way to imagine this is to say it's the complete activity flow in the brain at any moment).
  3. They are temporarily allocated interpretation represening machines, that do whatever they can do in order to be stable.

(also note that they can be strategic, they might play a game that goes across them being extinguished for a while).

How are you stable in this system? If the activity comes from the sensors, you represent things that are true across sensor changes. If the activity also comes from reward wires, you incorporate the reward wires, presumably by navigating the animal or the mind into a hard-to-fake reward state.

Machine-level mechanisms can shape memetic landscapes:

For instance, if we reset all or some neuronal areas now and then, we force the ensembles to ignite within shorter periods. In other words, we cut away the slow ensembles in the ensemble exploration space.

                                         The interpretation game

                                                                    +-----+ +----+
                                                                    |     | |    |  'commitment structures'
                                                                    +--+--+ +----+   user level
                                                                       |       |
                                Golden elegance wires?                 |       |
                                                                       |       v
                                      --------------------->           |       ----------- expression interface(s)?
                                reward wires                           v       |
                                                                               | ... allowed to be arbitrarily multilayered

+--------+                                                         +-----+     +-----    causality structures,
|        |                                                         |     +-----+         explanations
|        |         everything possible a litte bit                 +-----+     |                          |
|        +--------------------------->                                         |                          |
|        |                                                                                                | abstraction drivers
+--------+                                                 [.]         [.]         [.]                    |
   memetic engine                                           ^           ^           ^                     |
                                                            |           |           |                     |
                         +----------------------------------+-----------+-----------X----------           |
                         |  ?                               |           |           |                     v
                         |                                  |           |           |
              +----------++                             +---+----+ +----+---+ +-----+---+
              |           |                             |        | |        | |         |
              |           |                             |   A    | |   B    | |    C    |
              +-----------+                             +--------+ +--------+ +---------+ , ....
            dream mode                                     situations

                                         whoever is active across many situations is representing something abstract

Presumably this is Popperian: Some random ideas that fit the network a little bit start to exist (conjecture), then if they are good ideas, they will stay stable across situations. Otherwise some other ideas will survive instead. The same process should apply to ways of thinking (Minskies term), ideas that create situations for other ideas. (Like ecosystems).

Levin asks us to imagine a spectrum of fleeting ideas that come and go, then some ideas that stay a little and make some niche construction, and at some end ideas that spawn other ideas. There is no distinction between thinker and thought in such a paradigm (see Levin being interesting again). From one perspective, you see a thought with some internal structure that is spawning other thought, from another perspective you see pieces of personality and so forth. And these things are allowed to be on continua and so forth.

These ideas by Levin fit well into my framework here, where I say that the brain is implementing a 'stable information' milieu. That computational paradigm is made from datastructures that are stable. I.e. the neuronal ensembles are both the data and the situations for further data processing (Context Is All You Need?).

Perhaps put to much emphasis on the reward wires on this page. The reward wires can only ever do 2 kinds of things:

  1. Give a value judgment for situations
  2. Reward or bias the activity based on machine-level heuristics. (Perhaps Good ideas from Tübinger Cell Assemblies, The Concept Of Good Ideas And Thought Pumps)

This cannot take the content of the ideas into account. It must be something that counts the same for all kinds of ideas. Darwin's hand could only ever build a machine with a propensity for creativity.

My thinking now is that the machine cannot help but make explanations of the world and the reward wires are secondary to this process.

  • dream-mode
  • dream mode perhaps picks some 'salient' or 'unresolved' situations (with some Darwinian criterion).
  • Perhaps it simply mixes a bit of random perspective, orchestration with some situations from mid-term memory.
  • If we then run the machine like usual, we find stable ensembles
  • And the ones that are stable across many situations are the abstract and general ones.

All memes are interface pieces

Taking the logic of taking the cortex layer 5 arrangement seriously. See A Curious Arrangement, and . All ensembles have the chance to output behaviors, and they have a memetic driver to do so, otherwise they won't spread as much into the rest of the cortex via thalamic relays than their alternatives do.

I speculate that his arrangement forces the memes to represent behaviors that keep them alive. They are all buttons with a preview. Click me and your mind will change this way. Getting A Visual Field From Selfish Memes for some ideas of the mechanisms.

In other words, it looks like the ideas are always a composition of expectation (what sensor level stuff they are associated with etc.) and affordance, how to be used.

Action and perception are unified in the stableness of neuronal ensembles. And I find this duality aesthetically pleasing, (aka a nice hack). It gives a solid evolutionary perspective on the evolution of the cortex - i.e. a single area is already useful. And it says what is the information relayed at layer 5, higher-order thalamic nuclei and to the striatum? Possible behavior streams that make the ensembles stable. For instance, moving the eyes along a line in the visual field. This motor data can be interpreted as the shape of the line.

The Interpretation Game

  • Whoever has support from the sensors and the rest of the interpretation, wins.
  • Whoever is fastest can inhibit their alternatives.

The Ideas Need To Make Sense of The Past Continuously

This is easy to see when we imagine resetting, that is extinguishing the whole activity in the network every now and then, perhaps with every eye saccade every 200ms.

Firstly, the ideas need to re-ignite in order to be stable. Also, the ideas need to constantly make sense of the past. (Levin being interesting again).

For this reason I imagine the memes as archeologists at times, that need to make sense of the output of past civilizations.

It is impossible for the ideas to stay perfectly stable, this would only work by arranging identical Bolztman brains in a sequence. That this arrangment would be useless evolutionarily is easy to see, such an animal would be stuck and not learn from experience.

The job of the memetic engine is to mix up the activity. Because the activity would want to stay for eternetity if it can. A failure mode of doing so would be a selfish meme that says "don't move". Then you can have the same mind over and over.

Perhaps this happens with sleeping sickness (Small documantary about this, Speculations On Striatum, Behaviour Streams).


Society of Mind (Papert, Minsky), The Jungle of Ideas (emphasizing it's living entities), the idea ecosystem, magic banana matrix, idea multiverse, Hofstadter's cities of meaning, situations (they scale arbitrarilyi).

IIIb): The Langauge And The User, Elegance, Competence, Abstraction, Computational Cybernetic Psychology

Everything should be as simple as possible, but not simpler.

Albert Einstein

What are the principles that make good languages?

                                             +--+     user level
                             +----------+    +--+
 competence drivers          |          |
 abstraction drivers         |          |
 ------->  ------->            |           ------ ------- ------ the user interface
layers below,                  |            +-+
development                  +-+---------+  +-+
                             | v         |           'common sense'
                             | |         |  +--+      affordances
                             +-+---------+  |  |
                               |            +--+      language level

The point where the magic comes in I think. The science of abstraction and elegance doesn't exist yet, but we see it in software design and so forth: There is something about minimally explaining something, also called elegance.

This 'minimal but sufficient' description is the meaning of Einstein's everything should be as simple as possible, but not simpler. This is a way to do theoretical physics as far as I can see. It is to take a range of phenomena and find a fundamental plane of explanation, where a minimal but sufficient amount of causality structures explain, in terms of common sense perhaps, how the world is.

We see it again in the genetic toolkits of evo-devo and so forth: There is something about expressing a complicated thing in terms of a lower layer of building blocks, that is powerful. This power is to some extent subject to the philosophy of programming. See Magical Interfaces, The Principle of Usability.

But the real science that will talk about it will be something deeper. A science of design and explanation. I don't think it would be wrong to call it 'cybernetic epistemology' A core idea of cybernetics is the idea of 'the interplay' and the view from above. Saying how the world is from a counterfactual plane, what the purpose of the system is and so forth. This is an obvious thing for me when you consider biological systems, the organism has a purpose and a function, and its structure will approximate a solution to the problem. That is the function doesn't exist, it is a counterfactual, but it is the most relevant part of the explanation of the organism. I think this notion of 'how building blocks fit together', and this 'harmony' are cybernetic topics, too. Also, I feel that David Deutsch is an early contributor to this science, and he highlights the importance of knowledge. Perhaps Constructor Theory will help with coming up with some theories of this science.

What are the principles that make good languages? Brain software must explore the space of possible good languages and design. From this reasoning, we see clearly that the analysis brain software is a special case of an analysis of whatever this 'elegance' and 'succinctness' in explanation structures is.

Either way, we can use some solid common sense biological cybernetics reasoning, mixed with knowledge of programming and interface design to at least describe this layer of brain software. And we have in some ways succeeded, too. The problem of the mind turned out to be deeper, one that will unify all explanation-making and art (See David Deutsch: "Why Flowers Are Beautiful").

Names: 'Epistemological cybernetics', 'Constructor theory of elegance', 'The science of elegance', 'The science of harmonic building blocks', 'Design Theory',

I conjecture that brain software grows an explanation language layer, which I label 'common sense'. This common sense can talk about objects, places, people, causalities and so forth. Its implementation in the brain is of course the neuronal ensembles, the high dimensional data structures that want to be stable. The logic of stratified design creates roughly a polarity of user and producer entities. Where producer entities have memetic drivers to be more general, harmonic and useful, and user-level entities have a driver to be less concerned with the details, compose useful languages with elegance, and strategically commit to plans and actions which will help them stay "on". Note that they don't need to know how actions work in detail, that again is handled by producers of the magical interface.

Stratified Design: User, Producer, User, Producer, and so forth

  • The difference between programming languages and user interfaces is nominal
  • A good explanation language will be good because it makes a general producer layer, and a user layer, that composes it elegantly
  • This is allowed to go in layers down a hierarchy, or some heterarchies
  • So the 'user' of common sense that makes explanation structures is allowed to be a software-level entity in some deep middle layer of the cake. I guess the cognitive user simply has the interface "I look and I understand the world". Then they look at the world and the producer of the interface provides the contract. That is, the implicit theories of how the world works, for instance, that you are looking at a ball hanging from a string, are composed of something like this:
       |       |
       |       | user
           |      I understand when I look
       +----------+ producer
       | ^    ^   | user
         |    |         I compose causality structures   ^
         |    |                                          | This ball fits my narrative, I active them, too (?)
      <--+    +--->                                      |
+----+      ..        +----+   producer                  |
|    |                |    |                             |
+----+                +----+   user                      |
  string                ball                             |
    |                         <----+                     v
   ....                            |
                                 |    |   I am an object in the world
                                        I compose sensor-level expectation structures
                              ...       - this is what to expect when looking at me from different angles

                              |    |  I encode a position
                 +---+                            +----+
                 |   |  I encode a color          |    | I encode a shape
                 +---+                            +----+

                 |   | I encode a weight

                      - this is what to expect when you put me in your hand

The point is that user 1 doesn't know how the system makes him understand the world, she only looks and demands from the system that she understands. Producer 1 yields composed causality structures, i.e. they are using the common sense language. But they don't know how the language works, this in turn is handled by the building blocks of the language itself.

The fact that saying "I just say what I want" is somehow useful is a programming topic.

By saying what and omitting the how, we give a name to the what. And it is somebody else's problem to make the how work.

This is allowed to go all the way down. Because we get out of the infinite regress at the so-called primitive layer. Eventually, the process tasks are executed by the laws of physics. In an electrical computer, this works by arranging circuits artificially.

Memetic Engines Create Competence Hierarchies Up To User Illusions The Structure And Function of The Cell Assemblies Is Their ad-hoc Epistemology

What does it meam to give a name to something in a living software paradigm? Shouldn't this be messy and isn't symbolic AI disproven to work?

Symbols are recovered via high-dimensional computing [Kanerva]. Don't worry, hyperspace is so large that even with substantial noise, a point inside it is distinguishable from other points.

The means of abstraction are recovered by the notion of context and interpretation. My vague idea is that templates or situations are procedures. In this software paradigm, the procedure object is just made from the primitive data structures again! (I.e. a composed neuronal ensemble). First ideas: Context Is All You Need?.

The science describing the layer:

  • Minsky and Hofstadter analyze what I could call the software layer of the mind.
  • The science of abstraction and elegance:
  • Doesn't exist at the moment, perhaps a constructor theory subfield later?
  • philosophy of programming (underdeveloped)
  • This is necessarily a kind of programming, it is the analysis of a self-assembling program. (The space of analysis of programs is bigger than the analysis of our current programs).
  • Micheal Levin (the theory of how biological intelligences mine the competencies of their substrates)
  • Gerald Sussman (philosophy of programming)

The formalism that describes the layer:

  • A hypothetical advanced field biological software analysis and design.
  • I would like to label it computational cybernetic psychology (from Heinlein, inspired by McCulloch etc I suppose).
  • These are programming languages that do not describe the neurons and their wires, But they use the same abstractions that biological intelligence uses itself.
  • The elements, means of abstraction and means of composition would not be ensembles, although their implementation layer would be ensembles in the brain.
  • They would be situations (why the notion of a situation is sufficient to represent all of the elements, the means of abstraction and the means of combination in a computational system like the neuronal ensembles see Context Is All You Need?).
  • In these languages, I would be able to compose [ visualize burning toaster ] and whatever biological intelligence is running the code is visualizing a burning toaster.

    Whether this is even meaningful or useful remains to be seen. Implementing this for something like the brain would mean to play the piano of mental content.

  • There are partial successes in the electrophysiology of observing and manipulating ensembles, so the principle computational cybernetic psychology is not far-fetched. [Yuste is a inspiring thinker]
  • Observe that formalisms on this level are not necessary for making biological intelligences. This level emerges from the previous ones, which we have hopefully crafted carefully with the whole 'emergence stack' in mind.

Brain software Is a Technium

Technium is a term coined by Kevin Kelly, the founding executive editor of Wired magazine. He uses it to describe the greater, global, massively interconnected system of technology vibrating around us. Kelly suggests that the Technium is not a mere collection of tools; instead, it forms an ecology of devices and systems converging and co-evolving, cultivating each other and the planet in result. It shapes our thoughts, culture, and behaviors. In essence, Kelly sees the technium as the seventh kingdom of life following Plantae, Animalia, Fungi, Protista, Archaea/Archaeabacteria, and Bacteria Eubacteria.


I want to say the ideas are alive The Biology of Cell Assemblies / A New Kind of Biology, and we can also say the ideas are pieces of technology.

What does it mean that innovation drives innovation? In How Innovation Works Matt Ridley (2020) describes how the ideas seem to become ripe from the existing context. That 17 people had the idea of the light bulb right around the same time. Almost as if some yet undescribed force of nature is moving technology forward.

If the brain is an explanation-making device, then perhaps it creates a kind of ecosystem for good explanations to thrive. Explanation making and using the computer is a kind of technology, with little tricks and kludges. The mind is a programmer, programming itself. So it must discover and build upon the little pieces of the program it already found.

I guess something like the technium is happening in brain software. That there are fronts of technology, which enable higher layers of ideas in turn. Once a certain technological level is found, and makes a universal language (building blocks). And then other layers explore a new kind of technology in turn, using the existing building blocks and tricks.

This sheds light on certain 'pre-programmed' phases of development. For instance the language acquisition timeline of children.

I think before this is all over, we might find certain unifications of the fields of ideas, brain-software, technology, civilizations, epistemology and biology.

IV: The Cognitive Machine: A Simulated World With A Cognitive User Inside It

Turns out the best way to navigate the world as an animal, or be a human and make explanations together with other humans, is to simulate a world and a cognitive user inside it. This has the advantages of stratified design, a user level can compose a producer level. At each point of this cognition machine, the living software can improve itself. Leading to less leaky abstractions, a machine that just works, magically so.

the cognition machine

|  +--+   ++ |         user level entities:
|  |  |   ++ |         - confident, wizards, use magical interfaces, competent, confabulate, demand
|  +--+      |
|        +---+         producer level entities
+--------+---+         - elegant, general, easy to use, living software, can make explanation structures

                       situation analysis
                       -  the Darwinian brain still binds this system into to the world,
                          makes it care about food and mates and so forth

The science describing the layer

Note that it is explained in terms of the lower layers. Consequently, I don't think that a formalism on this layer would be meaningful, it would just be advanced computational cybernetic psychology, taking into account the user and their interface.

  • Memetics
  • grown-up neuroscience
  • psychology
  • cybernetic psychology, or 'robopsychology' when it is for robots (that is Heinlein and Assimov's names respectively)

'Activity That Survives' Spans Mechanistic, Software, User And Biological Function Layers

We easily see that coalitions of memes are implemented by associated or composed neuronal ensembles.

My hunch is that neuronal ensembles that are active together are symbiotic by default unless they are alternatives to each other. Then see Contrast And Alternative.

Note that you can always increase the time window, and you will have stretched what counts as single ensemble.

[ insert how the complete neuronal ensemble overlaps with the survival coalition we might call a self ]

[ not because we were looking for a self, but because we think about explanations of the mind and its biology and find entities that correspond, after the fact ]

The Living Language Problem

Musings That Are Bound To Be Wrong, The Living Language Problem

You wanted a banana but what you got was a gorilla holding the banana and the entire jungle.

Alan Kay

From the logic of harmony in a self-assembling software paradigm, we get a layer of generally useful, fundamental and abstract languages, which I label 'common sense'. They are hormonic building blocks of explanation structures, that fit well together and must allow us to express composite explanations with elegance and artistic swag. I.e. they must be powerful languages.

  1. Languages are powerful when they are general.
  2. Languages are general when they don't have ideas on their own.
  3. I.e. Good languages are perfectly obedient, which is the opposite of creativity in some way.

This is a deep property of explanation-making processes. It seems like everything possible to express must ideally be equally easy to say.

Why this must be the case, consider an alternative, a skewed language.

Imagine a hypothetical language, which imposes the constraint that the word 'banana' would have to appear in every sentence. This banana language would be less powerful than English. It would necessarily make some ideas harder to express, namely the ones that have absolutely nothing to do with bananas. Arguably, it would impose arbitrary constraints on the thinking process of the thinkers, who must find a way to incorporate the word 'banana' in every sentence.

In a software paradigm where the pieces are self-assembled and evolved and are ultimately selfish, this is a real problem. The ideas themselves have an agenda, and this stands in friction with the logic of the stratified design of programming.

You use the language and by how you use it, you change what is easy to say in the language. Because the ideas keep being alive; This sounds cool at first glance, but it means that the language is dancing on you - that is a leaky abstraction of its implementation detail of being a living language.

There are at least 2 memetic drivers for a language building block at play:

  1. You want to be harmonic with the memes you are on together often (i.e. you want to make a good system of building blocks).
  2. In order to be a good language, all ideas need to be equally easy to express.
  3. You are ultimately selfish, if you could you would make the language skewed.

1 and 2 conflicts with 3, what you might call the compositionality vs. living software conflict.

Observe that once the language is grown. For all the user care, the implementation of the interface can be replaced by any other version that adheres to the contract of the interface (which is fundamental to the notion of an interface). For instance, we can imagine replacing the 'common sense', physical explanation language with a compiled computer version, if it adheres to the interface contract. For all the user cares that might even be better.

There is an obvious way to mitigate this issue on the machine level. That is broadly the notion that freeze layers after a developmental phase, with the goal that a system of building blocks would develop in one layer and we subsequently 'freeze' this portion of the system. This way the language would grow, and once it is developed, we would drastically reduce its dynamism, changing its properties into a 'usage' mode, presumably by downregulating its dynamism.

This sheds light on Perceptual narrowing and the various critical periods of the visual cortex and so forth.

Names: The Banana Language Problem, the language is living software, general powerful languages don't dance on you,


WIESEL TN, HUBEL DH. SINGLE-CELL RESPONSES IN STRIATE CORTEX OF KITTENS DEPRIVED OF VISION IN ONE EYE. J Neurophysiol. 1963 Nov;26:1003-17. doi: 10.1152/jn.1963.26.6.1003. PMID: 14084161.

Explaining Cortex and Its Nuclei is Explaining Cognition

I am in the camp of people assuming that the interesting part of cognition happens in the cortex. In other words, it looks like modeling the function of the cortex is a path toward a model of cognition.


  1. Injuring the cortex makes a person lose specific capabilities.
  2. The cortex is the thing that explosively blew up in human evolution. 1a. Whatever is special about us is almost certainly special because of the cortex.
  3. The thalamus has 108 neurons; this is an upper limit for cortical inputs.
  4. Cortex neurons have 1010 inputs. -> This means that 10x maybe even 100x more connections to the cortex are from the cortex. (Braitenberg 1986) The stuff that cognition is is mostly stuff about the mind itself (reflexive). No wonder hallucinations are a thing.
  5. The cortex is more generic than other parts of the brain, it looks as if evolution found some basic building block (cortical columns?) which when duplicated, made more useful intelligence in the animal. -> Both the given that the cortex is generic/uniform and the given that areas are different is interesting.
  6. To explain cognition means to also explain altered states, out-of-body experiences, dreaming, mental pathologies, etc. (Metzinger).
  7. Everything with a brain also sleeps afaik. Roughly half of a good model of what the brain does concerns itself with stuff that happens during sleep. (we will see that this move makes us able to happily move this or that part of a cognition mechanism into a dream phase).
  8. The mind is self-assembled, without an external loss function.
  9. The mind is online and has resource constraints. Unlike a computer, there is generally not a pause when you ask a person something. The mind cannot stop reality in order to process some input. (But the existence of attentional blink shows us that a tradeoff in this space is being made).
  10. Whatever the basic building block of the cortex, it grows in the first 2 dimensions but not the height. Otherwise, we would have evolved more cortex volume, not surface area (Braitenberg …)

  1. Neurons can represent information states, neurons are universal [McCulloch and Pitts 1943, Church-Turing]
  2. Layering information representations leads to higher abstracted information representations [Rosenblatt]
  3. The same idea is the 'feature detectors' of computational neuroscience [Hubel and Wiesel 1960s]
  4. We sort of know that cognition machines are meme machines [Dennett 2017]

Reasonings / Intuitions:

The cortex is an information mixing machine - Braitenberg

  • When thinking about the cortex: a. We are allowed to use the rest of the system as an explanation. The cortex is about itself. -> The explanation is allowed to go in a loop.

    b. We are allowed to use an abstraction barrier and explain a part (a juggling ball) in terms of stuff

  • The cortex is like an ocean. This is part of the point.

|                                          |
|                      |    ^              |
|               ====== |    |      <-------+-----+
|               Cortex |    |              |     |
|               ====== |    |              |     | about itself
|                      |    |              |     |
|                      v    |              |     |
|                                 ---------+-----+
+--------------+                           |
|              |                           |
+--------------+                          |   | eyes
   rest of the brain                      +---+

Why is there a big fat cake of stuff that is about itself on top of the brain? And it seems to have to have to do with what we call cognition and intelligence.

What is [B], the brain, about? Easy to see for vehicles 1,2. There are simply sensors 1:1 about the world. With vehicle 5 we enter the world of being able to stick in more information processing units in between sensor and actuator. The interneurons of neuroscience. With that move, we have an internal world, and some pieces of our machine can be about that, instead of the sensor input.

The cortex is mostly about itself. If you look at a piece of cortex, you will mostly see neuronal units that are about… some other piece of cortex. Their world of a cortical neuronal unit is mostly the rest of the mind!33

Pyramidal cell activity - The Gasoline

I used to think hey this or that. Maybe Glia cells are important, maybe the neurons do computation inside themselves, who knows?

I have refined my model to simply take pyramidal cell activity as the center and move on.

  1. If something else than neuronal activity is how the brain works, then why is activity what drives muscles? It seems much more biologically harmonious that whatever is making muscles move is also what the brain is about. (Turns out that is pyramidal cell activity).
  2. If neurons do computation inside (microtubules? quantum computing…?, pixie dust at the synapses?) then how do you get the information from outside the neuron into the neuron and back out again? I see you need to solve the same engineering problem that the brain is solving in the first place, a second time. (I.e. how does the information go into the neuron and out?).
  3. Some sensor -> motor reactions happen in a time order that leaves time for only 1 or 2 action potentials through the brain. (Braitenberg 1977) Everything looks like there is nothing faster than action potentials that can move through the brain. The pyramidal axon, in general, has an evolutionary drive towards transducing action potentials as fast as possible. How would this fit with a view where something else than neuronal activity is doing the computation? Just the aesthetics of these 2 ideas don't fit in my opinion.
  4. Tissue with brain lesions is filled with glia, and this tissue does not substitute the brain function as far as we know. (Braitenberg 1977) This does not bode well for a model where glia calls are doing anything essential for cognition.
  5. The hippocampus has the most cells in the cortex [iirc], it is often the initiator of epilepsy. (Braitenberg 1986) Whatever the hippocampus is doing with all those cells then, it has to do with excitatory neuronal activity.
  6. With spiking neuronal activity you can put short-term memory in the units. Just make a circuit that goes in a circle. The simplest version of this is a pair of neurons going back and forth. This is a memory implementation. -> This all looks like neuronal activity can make short-term memory
  7. If I want to make a lot of short-term memory and have an evolutionary driver towards mid-term memory, I get something like a Hippocampus. Consider a cluster of cells with an evolutionary driver to make activity back and forth to store some previous state. a) Hippocampus has a lot of excitatory cells, so you can store a lot of 'activation'. b) If you want to abuse the circuit to simply make activity back and forth, it is useful to slow down the spiking rate. Hence, it is not surprising to me that the hippocampus has slow theta wave activity. c) You have another problem to solve, this ongoing memory activity should not spill to the rest of your (activity-based) system. So you curl up that piece of neurons and make it anatomically separate from the rest of your cortex. Related to this, you get the problem of epilepsy, you need to manage this immense activity of neurons now.
  8. We know some visual cortex circuits and they work with pyramidal cell activity.

Everything looks like and nothing does not look like the main substance of what is doing cognition is the activity of the (pyramidal) cells.

It is not the engine, the gasoline, the wheels or the steering wheel that is most important in a car.

If you say 'the synapses are the most important part'. I only need to make this analogy and I am at peace. The only thing that truly matters is that you don't bring in something non-essential because that is just wrong.

If the activity is like the gasoline, then what is the rest of the engine?

Neuronal Activity shapes the Neuronal Networks

The story of Hebbian Plasticity is a remarkable triumph of reason and imagination.

Let us assume that the persistence or repetition of a reverberatory activity (or "trace") tends to induce lasting cellular changes that add to its stability. … When an axon of cell A is near enough to excite a cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that A’s efficiency, as one of the cells firing B, is increased.34

Hebb [Hebb 1949] came with psychological reasoning and imagination about what neuronal tissue could be doing to explain what we observe from studying animals.

With something in mind to look for Eric Kandel went and found the biochemical basis for Hebbian plasticity, the work that made him win a Nobel prize.

It is important to note that fire together, wire together leads to a slightly wrong idea.

It is that every time I fire, I can look at who activated me - and those synapses I can strengthen. So there is fundamentally a causality in this notion, not mere juxtaposition or mere association. The rule needs to look at the activations of 2 time steps, not one.

It should be called Your fire is my wire, I wire if you fire, fire me and wire me, They who fire, make me wire, The more you fire, the more I wire, When you fire, I will wire, I am looking for some fire with my wire, I want to plug my wire in your fire.

[insert history, Turing, Lorente, Hopfield] [cool history part by Rafael Yuste here: Rafael Yuste: Can You See a Thought? Neuronal Ensembles as Emergent Units of Cortical Function]

Alternative Plasticity Models

Perhaps Cortex doesn't actually do Hebbian Plasticity.

Iceberg Cell Assemblies

we find increases in neuronal excitability, accompanied by increases in membrane resistance and a reduction in spike threshold. We conclude that the formation of neuronal ensemble by photostimulation is mediated by cell-intrinsic changes in excitability, rather than by Hebbian synaptic plasticity or changes in local synaptic connectivity. We propose an “iceberg” model, by which increased neuronal excitability makes subthreshold connections become suprathreshold, increasing the functional effect of already existing synapses and generating a new neuronal ensemble.

Intrinsic excitability mechanisms of neuronal ensemble formation

Intuitively, if many of the neurons in a locally connected group have more 'excitability', then this group has a higher chance to ignite.

All of these models are like having playdough in high-dimensional spaces. You are allowed to morph the 'attractor' landscape by activating neurons together.

;; ================
;; intrinsic excitability plasticity
;; ================
;; *Iceberg Cell Assemblies*
;; Paper: https://www.biorxiv.org/content/10.1101/2020.07.29.223966v1
;; quote:
;; we find increases in neuronal excitability, accompanied by increases in membrane resistance and a reduction in spike threshold. We conclude that the formation of neuronal ensemble by photostimulation is mediated by cell-intrinsic changes in excitability, rather than by Hebbian synaptic plasticity or changes in local synaptic connectivity. We propose an “iceberg” model, by which increased neuronal excitability makes subthreshold connections become suprathreshold, increasing the functional effect of already existing synapses and generating a new neuronal ensemble.
;; So instead of Hebbian plasticity, we can try to model a cell-intrinsic `excitability`, `intrinsic-excitability`.
;; The easiest model coming to mind is the same as the attenuation above. (but inverted).
;; A cummulative excitability, with a decay.
;; `excitability-growth`: Could have been called learning rate, to make it sound like machine learning.
;; `excitability-decay`: Relative decay each time step.
;; `excitabilities`: You could imagine pre-allocating this, probably this is somewhat random in biology.
;; 0. Grow excitability for all active neurons, (add excitability-growth)
;; 1. Decay excitabilities multiplicatively by excitability-decay
;; 2. excitabilities multiplicatively on the sum of the inputs for each neuron.
;; 0-2 could also be more complicated functions
;; An excitability of 0.0 means your inputs are at baseline.
;; An excitability of 1.0 means your inputs count double and so forth.
;; In principle, negative numbers are allowed. (but not utilized by this step model here).
;; In this case, this would flip into an attenuation or depression model.
;; -1.0 is the minimal meaningful number, saying that synaptic input is 0, beyond that you would
;;  get negative synaptic-inputs, which are not defined behavior.

(defn intrinsic-excitability
  [{:as state
    :keys [excitabilities excitability-growth
           excitability-decay synaptic-input n-neurons
  (let [excitabilities
        (or excitabilities
        ;; decay
        excitabilities (mathjs/multiply
                        (- 1 excitability-decay))
        ;; accumulate the excitabilities on everybody active
        (.subset excitabilities
                 (mathjs/index activations)
                 (mathjs/add (mathjs/subset
                              (mathjs/index activations))
    (assoc state
           ;; if you have excitability, your inputs count more
            (mathjs/add 1 excitabilities))
           :excitabilities excitabilities)))


Figure 4: You can make ensembles like that. What I find is that the overall network will fall into a single stable attractor. Which is of course useless.

From considering this relatively static aspect of the substrate at that point, I came up with adding a skip-rate.

Combining intrinsic-firing-rate, attenuation, skip-rate, excitability-growth, a simple threshold device that goes random between 25 and 50 neurons, and geometry:

(Note these are without Hebbian plasticity).


Figure 5: Combining a few mechanisms, toy neuronal area with a sensory field. Effector neurons pull sensor balls into the sensory field. Sensor neurons are statically allocated per color and fire when a ball is in the field. The neuronal area resets every 5 neuron ticks. So ensembles need to re-ignite.

This is a much more dynamic substrate. I feel like it finds a ball at times and with some luck, this ball will be stable in the sensory field. In a way the basic conjecture of neuronal ensemble memetics. Given the chance to be stable with luck, the memes will be stable. It is the job of the substrate to allow for this luck to occur in useful ways.

The network connections (the underlying graph) are static still. My current explorations of dynamic connectivity is here: Synapse Turnover.

Geometry makes Cell Assemblies

Without plasticity.

Here I have a model of this:

Sensor input (move mouse) activates a random projection in a neuronal area. There is no hebbian plasticity rule. The network has a geometry (topology). Neurons that are next to each other have a higher chance to be connected. This makes neuron time steps, takes the top (cap-k) neurons at each time.

This network is static so to say. It represents the inputs with a cell assembly.


Cell assemblies and Memetic Landscapes

The brain is playing with fire.

Tery Sejnowski35

The meaning of this is that the brain works with positive feedback, and excitation, not negative feedback like the control systems we build. The fundamental problem coming out of this organization is epilepsy; Which unsurprisingly comes in many different forms.

The fire is igniting the network, and the biology of the system must keep this fire in check.

The second part of Hebbian Theory is the Cell Assemblies. The cell assemblies are well-connected sub-networks that support their activation together.36

In order to make something interesting, we need to limit the amount of activation, which Braitenberg called a Thought Pump 37. Another plain name might be inhibition model. (But the term thought pump is so daringly far-reaching, I love it).

This fire; survives, or it doesn't.

Epilepsy is a memetic problem: The simplest meme says 'activate everybody'. This suddenly gives us the perspective from one abstraction level further up: The machine that shapes the memes.

Imagine those cell assemblies, supporting each other, evolving strategies of survival etc. Survival means I keep on firing together with the people that activate me. They are replicators, they represent knowledge about how to replicate across neuron timesteps. They are memes [Dawkins 1976] made from neuronal substrates.

Note: These are somewhat different memes than the cultural unit of imitation. These memes here, the cell assemblies are the replicators of computational neuroscience. They have to do with cultural memes, but only because they are part of the functioning of the cognitive computer the cultural meme is infesting.

An inhibition model is a memetic landscape shaper. It says 'I make it harder for memes to exist'.

If you shit on all ideas evenly, only the good ideas will survive. This is the basic notion of a thought pump. And a doorway into neuro-physiological conceptualizations of memes and their substrates.

This activity flow needs a strategy now, in order to win out against other activity flows - else it will not win in the competitive environment that the thought pump creates.

Of course, it does never have strategies the way we do, its strategies are free-floating rationals and it has competence without comprehension [Dennett 1995].

The activity flow of neurons that support their activity together, is subject to the laws of natural selection. They are replicators, abstract entities with an agenda.

The computational substrate I have in mind is designed to support memes:

  1. Make activity scarce
  2. Make the activity mean things by connecting it to sensor and motor destinations and sources.
  3. At each time step, choose the best-connected subnetworks. I.e. make the subnetworks compete for connectedness.
  4. With 'online' plasticity, you get immediate associations between cell assemblies. Like playdough that sticks together.
  5. Mutal self-excitatory pockets of activation emerge from this arrangement. The Cell assemblies can be seen as a data structure in a high-dimensional computing framework2.
  6. We can see the cell assemblies as replicators[Dawkins], replicating across neuron timesteps. How well do they replicate? It depends on their connectivity, i.e. their meaning.
  7. Use memetic landscape tricks, that bias the system towards useful memes.38
  8. Optionally: Prune away everybody who is not active. [Literature on 'perceptual narrowing', and Neural Darwinism]

[This is all biologically reasonable]

The working hypothesis:

The structure and function of Cortex is to implement an assembly calculus.2

Memetic landscapes?

Ideas so far:

  • Control all sources of activity well, they make the meaning of the meme space.
  • If you make the prime source of activity the sensors, you get memes about the world.
  • If you reset the memes in a short amount of time, you get memes that are competent in a short amount of time which is a memetic driver for abstraction and hierarchies.
  • Now you have memetic drivers that make memes spread into areas/ nuclei that are not reset. If you make it hard after some criterion to spread into such nuclei, you select what kinds of memes you get.
  • If you make a slow meme-place, everybody wants to spread into that place, that is an evolutionary driver for midterm memory. Simply make a place where memes survive for long periods, they will compete to get there.
  • You can implement reward by activating a sub-network that makes up a meme, it is tempting to assume that some gamma activation would come from such a rationale.
  • You probably want to hook up to some Darwinian wires, that tell you 'this has to do with survival'. This way you can reward all the memes that the Darwinian brain approves of, i.e. you change the memetic landscape to favor memes that are useful for survival.39

Consider the simplest network of 2 neurons A and B.

[ A ] [ activate? ] [ B ]

The meaning of the network is the connections. It can mean A activates B, B activates A, or both, or none. Disallowing self-connections for the moment.

Consider the basic optimistic meme: 'I will stay alive', or 'I will be active'. We can draw the success matrix from above again.

activated! not activated
|   S     |   X    | competent
|         |        | (the meaning of the network means I stay active)
|   X     |   X    | incompetent
|         |        |

You need to be optimistic and competent in order to have success, or else you are discarded. The only meme that will survive is the one that says 'I will stay active'.

From memetic theory then, we can see that this network will stabilize into expressing the notion that 'A and B activate each other'. I.e. those are the connections that will survive and strengthen, everything else is discarded.

Kinda work in progress reasoning.

Let's assume for a moment that we can support memes with temporal structures. That is a cell assembly that waxes and wanes and represents a succession of states and so forth.

Here is a wonderful strange inversion of reasoning, it is that the meaning of memes happens before there is the meaning of memes.

Let me explain:

Suppose for the moment that virtually all activity must flow from the sensors. We see the first layer of memetics, which says 'Sensor active'. Since the simplest meme says 'I will stay active'. The simplest meaning is 'This sensor will stay active', in other words, 'The world will stay the same'.

Let's imagine a memetic unit (whatever that is, maybe a cell assembly) saying "I survive", in some derived meaning space. I.e. it is a space of possible meaning space, that is about the sensor memes, about the spatial and temporal structure of sensor meanings. The memes don't know yet what they mean, it is that the only memes that will survive are the ones that have competent connections I.e. they connected to the sensor memes in such a clever way, that they represent something about the world that stays the same, even though the sensors change!

To see why this is true consider that each meme competes for sensor activity with alternative memes. We can also imagine competing with a null hypothesis element. That says 'If you are not connected well enough to some people that activate you, the null hypothesis will be on'. These are thought pump considerations, making a more competitive environment for memes.

We can further imagine another layer of memes still, that simply stays on, because they connect to other memes with a temporal structure. Thereby representing the essence of some ongoing situation.

  1. These memes have a reason to find essences in situations.
  2. This search is memetic:

    First, you have many possible memes, all saying that they are eternal truths.

    Second, the wrong ones are discarded. And the ones that mean something useful about the world stay.

It is tempting to muse further. What about selfish memes that simply activate from all over the network? Maybe the answer is that they exist, and their meaning then is 'The mind exists', or 'The self exists', or 'The world exists'.

Maybe it is those memes that can give us the sense of something vaster and greater and eternal existing in the universe, our minds, our social realm, etc.

Maybe it is unallocated meaning units, that look for a reason to exist; Maybe it is some of those meaning units that a person can attribute things like god, or the fundamental spirit of nature or something.

Why is the basic meme 'I am eternal truth'? It is because of the optimism-drive reason from above. A pessimistic meme will be less successful than an optimistic one. It is the holy overlap between optimism and competence, that is favored by natural selection.

To relax this one further, it is useful to consider the basic meme 'I will be thought again'. This opens the door for what I call strategic memes (below, future). Strategic memes are memes that bias the system to be expressed again, not by staying active but by playing longer, cleverer games. (It was of course obvious from memetic reasoning that they should exist. We see that cell assemblies like this would exist, given the properties of the network above).

The basic possible rules of meme machines will create an environment that biases everything towards useful memes, that is the problem that the basic wiring of the brain somehow needs to solve.

One of the most important problems a meme machine has to solve is selfish memes that don't mean anything useful. So we can expect much of the brain's functioning to be solutions to this problem.

I would submit that describing, categorizing, fabricating, and transforming memetic landscapes is one of the basic problems of cybernetic psychology or 'synthetic memetics'.

Similar to this you might say all memes are a mix of static and dynamic activity flow. This idea comes up in Neural Assemblies by Guenther Palm.

A cell assembly implementation (online) A random directed graph with geometry(neighbors have a higher chance to be connected), without plasticity. At each time step, the top k neurons are allowed to be active. You can move the mouse to produce different inputs. (inputs in red). Only one cell assembly will 'win out' generally in such a setup.

With such an inhibtion model, each cell assembly implicitly inhibits its alternatives, since only one is expressed.


Figure 6: Ambiguous picture of either a young woman or an old woman.

A striking aspect about such ambiguous scenes, like this or the necker cube, is that you flip between interpretations, but you never see both interpretations at the same time. The system somehow 'decides' on one interpretation.

The basic meme strategy is the one that makes cell assemblies in the first place: Activate your activators.

Since meaning-level memes and 'sensor level' memes stand in a relationship, and since we introduce the inhibition model, which makes the system decide between some cell assemblies, you get these prediction/interpretation shapes:

         \             /
          \     A     /                   meaning-level
           \         /
            \       / <----->   B
             \     /   inibit
              \   /
                X                        sensor-level

A - Stable cell assembly

We can observe:

  • You can produce perception states by stretching cell assemblies across the meaning level and sensor level.
  • The meaning level parts of cell assemblies will bias the system towards 'perceiving' the world a certain way. (Activate your activators).
  • If meaning-level cell assemblies compete via something like the inhibition scheme of the model above, it is simply the nature of the mechanism that one wins out, never multiple.
  • I submit that this fits well with the emerging view of cognition called 'predictive processing', [Andy Clark, Anil Seth, etc.]
  • The inverted pizza piece, or the inverted ice-berg:
    • Perception is a meaning-level construct.
    • The meaning level is much larger than the sensor level (i.e. most connections in the cortex come from the cortex).
    • The meaning level can take larger situations into account. Perhaps the basic reason for having a cortex in the first place.

Perceptions are stable memes, everybody saying 'I am true'. But only the ones that get enough activation flow support from the rest of the system, including the sensors, win out. I.e. the meaning of a meme is its connections. The connections are allowed to be random at first since we can select the meanings afterward via natural selection. In a system where most activation flows from the sensors, this will select the meanings that represent true things about the world.

Cell Assemblies Lit

Tübinger Cell Assemblies, The Concept Of Good Ideas And Thought Pumps

Paper Cell Assemblies in the Cerebral Cortex, V. Braitenberg 1977 40.

I am using the term neuronal unit interchangeably with neuron. And the term neuronal area interchangeably cortical area.

Cell assemblies are subnetworks of well-connected neurons that activate each other. In an environment where activity is kept in check (via the inhibition module), only the cells that activate each other well will stay active. The inhibition element might work by setting a threshold of synaptic input, below which neurons will not be active. (Threshold control). Note that a simple inhibition model would be 'cap-k', saying that the top k neurons are allowed to be active.

The fundamental property you get from this is what you can call 'pattern complete'. We say a cell assembly can ignite from activating a subset of its neurons.

A cell assembly lives via the excitation of its elements.

Via Hebbian plasticity, 2 cell assemblies happening at the same time, will associate, we will find a subnetwork that is connected to both.41

From one perspective, we can say that there is only ever 1 cell assembly in the whole brain. Intuitively, you can imagine lowering the threshold maximally and getting all neurons to fire. This doesn't have to be this way, it depends on the connectivity of the network.

This would be the 'maximal cell assembly'. Presumably, that is the whole brain (that would be epilepsy).

The notion of a cell assembly only makes sense if you take the inhibition into account. Otherwise, it is hard to get to the notion of a piece of activity that survives on its own, because the elements activate each other.

A cell assembly might be composed of sub-cell-assemblies, that is, there are well-connected subnetworks of active neurons, which might form a well-supported cell assembly on their own. We say that the cell assemblies have multiple centers.

Braitenberg mentions homogenous and non-homogenous cell assemblies.

A homogenous cell assembly says that each neuron is connected with the same weight as everybody else, this is a theory-only construct, I think, to point to the non-homogenous cell assemblies: In a non-homogenous cell assembly, there are subnetworks of neurons that are better connected to themselves than to the rest. Again we say that such a cell assembly has multiple centers. Where a center is a subnetwork that would be able to support a cell assembly on its own.

Excitability is simply the inverse of the threshold; (I also called it eagerness before encountering the word).

If my threshold is low, I am eager to ignite.

If my excitability is high, I am eager to fire.

This means that you only need a little bit of input (synaptic inputs) to 'ignite'/fire.

This is simply to make it easier to think because sometimes I want to have that perspective.

If you imagine a cell assembly of mutually supporting neurons (a cell assembly center) and now you increase the excitability of the neuronal area, you will see that additional neurons are firing, the more you increase the excitability. These are not necessarily part of the center. They don't need to support the cell assembly at all, they simply ride on the existing activity, without feeding back to where it comes from. (Note that you only get this kind of activity with a low threshold. If the threshold is extremely high in the system, only the core of the best-connected subnetworks will be active - i.e. they all activate each other strongly).

We call this the halo of a cell assembly. The halo is like the outskirts of the cell assembly city. Where the center is the city center. It supports the activity of its halo, but if you increase the threshold, you will narrow the assembly down to its center.

For some reason, this makes me think of Arthur C. Clarke's 2nd law:

The only way of discovering the limits of the possible is to venture a little way past them into the impossible.

From this, you can craft a hypothetical mechanism that finds well-connected ideas, ideas that fit with the rest of the situation of the brain.

Say you have some inputs coming into your thinking goo (neuronal areas), that activate a subset of your neurons I. You will form a cell assembly from this. Call it FI. Note that the neurons that are part of the cell assembly don't have to be the ones that listen to the sensors in the first place. (With a random graph they are not, see my programmatic versions of this).

What you can do now is increase the excitability, activating a cell assembly E(FI), this is the union of FI and FI-halo. This represents everything, more or less, that is sort of associated with the inputs your sensors pick up. Maybe that stands for the 'excited' FI.

E(FI) will almost certainly be a non-homogenous cell assembly, containing multiple, maybe many, many [depends on the network] centers. One of them is the initial FI, others are BI, CI, … From cybernetic intuition, it is very likely that FI is not the strongest of these centers. I.e. there is some other sub-cell-assembly in E(FI), that has a stronger mutual activation. I am not sure yet and there many perspectives to take to make this super plain and obvious, but the intuition here is that there is a 'better idea' available; That there is a solution to the overall situation so to say, a meme (cell assembly) that has the right connections to be supported in the current situation (since the meaning of the world is in the connections of the network - it's quite wild, yes).

One thing to consider is that the rest of the cognition machine is making all those top-down cell assemblies.

Since those participants in the system are activating their activators, the shape of the lower level - E(FI) is determined by the larger situation at hand.

I.e. whatever is well supported by the rest of the system is activated. In other words, you bias what is possible by taking the larger situation into account. In other words, the system has a vague idea about what is true, you will not be a good meme, your connectivity will not 'make sense', your cell assemblies will not be supported, if your meaning is something completely else than what the system thinks is roughly true.

If I am a cell assembly that says 'Here is a face', I go and activate the lower perception people to represent inputs that look like a face. So E(FI) has a shape where all the activity that fits with the notion 'here is a face' are more active. Note that 'good idea' here means very specifically a subnetwork of neurons that happen to be connected in such a way that they get activated by whoever is active right now - stronger than alternative sub-networks.

We can observe that this is true by imagining 2 competing centers in E(FI), say BI and CI. BI might be slightly better supported by some top-down connections from the network. In other words, BI has friends higher up that support it.

The second step, as you might have guessed, is that we increase the threshold again. This will select the best connected sub-assembly from all possible centers inside E(FI). Finding for instance BI. [I can show programmatically how you get the best in a simple model. This is quite intriguing. And fits with the notion that something immensely parallel is happening in the brain].

This mechanism is what Braitenberg called a thought pump. (Note that this is a slightly different one from the one above; In "The Vehicles" the thought pump just meant a threshold control. In the 1977 paper, 'thought pump' means a very specific dynamic threshold control that searches for well-supported cell assemblies).

Now you can imagine a process where you increase and decrease the threshold in quick succession - perhaps creating the alpha, beta, gamma EEG waves. I.e. perhaps beta or gamma frequency represents the 'upper' step of a 2-step oscillation.

You will get a sequence of cell assemblies, I imagine a ball moving around in meaning-space. Perhaps it is especially interesting to have the ball move around then then suddenly stay. If it stays, you have found connectivity in the network that is the 'best-supported idea' of the network.

Or you move around the ball and it suddenly grows. This is a 'good idea that fits many things'. G. Palm is musing that this is maybe a mechanism for good jokes, too. Note that a good idea is an idea well supported by the current situation.

Another parameter would be the number of time steps you wait until you declare something 'the best interpretation of the situation'.

And another one would be the amount of activity that you declare a 'good idea that fits well'.

It looks like it is possible to shape a network memetically to represent the world (see sensor flow musings above, and predictor mechanisms, coming soon). Perhaps you build a network that represents the meanings of the world, and then you use that network to find further meanings again.

The thought pump can ask the existing network 'What fits?'. If the network grew from the sensors, then it has to do with real things. 'What fits' will be something that 'makes sense', that is realistic from many angles.

It is interesting to consider that there are 2 sources of time and causality with those cell assemblies.

1 is that I connected A->B via Hebbian plasticity, which means that if A is active, it will flow activity to B, in the next neuron time step. This is the fastest time representation available with that substrate, then.42

2 are the thought sequences of the thought pump. It seems like this needs at least 2 neuron time steps. 1 to increase and 1 to narrow down the activity flow. This doesn't mean that 2 is always faster than 1 because you have different neuron frequencies, varying by more than 2x,4x,6x or something.

The idea that you can lower the eagerness and get more fine-grained thought is quite intriguing. Consider that only the cell assemblies that fit the situation especially well will survive.

What you can do now is find an especially well-connected assembly. That one is still large, even though the threshold is low.43

Perhaps Hofstadter's abstraction ceiling (see his Strange Loop talk) is a brain feeling created from the situation of increasing the threshold very tight, without the wonderful release of a good idea.

Perhaps dopamine modifies what counts as a good idea. The bar for connectedness is lower so to say, biasing the system towards action, not deliberation. Perhaps psychosis is that you take the first good-looking idea and believe it, even though it was not the best idea you could have come up with at all.

Interlude: Zebrafish Larva

Zebrafish larvae are translucent for the first time in their life.

Scientists at the Howard Hughes Medical Institute studied live zebrafish larvae that had been genetically encoded with a calcium indicator called GCaMP5G. They suspended the larva in a gel and then beamed it with lasers. Just before a neuron fires, its action potential is expressed via a spike in calcium ions, so when one of the genetically modified larva's neurons reached its action potential, it glowed. This showed the researchers the firing of the neurons without them having to attach a bunch of electrodes to the fish. Over the course of an hour the researchers used laser beams to scan the larva every 1.3 seconds, exciting the retina of the zebrafish with each scan. This microscopy method allowed the researchers to record up to 80 percent of the fish's 100,000 neurons at single-cell resolution. This is the first time scientists have recorded such a high percentage of an organism's brain activity at such a high resolution.

Funny, there is a meta-phenomenon to this video: There is a huge spike in youtube 'most replayed'. At 00:18, hard to miss, a huge flash of activity through whatever that big middle part is.

Maybe it means nothing, but our imagination is pumped. It thinks! The small flashes are just as interesting as the big flashes. Neuronal flashes would make sense in light of thought-pump mechanisms.

It is perhaps counterintuitive if this is a thought pump we are looking at, a big flash might be more something like a re-orienting. It is that the current interpretations are not so interesting, useful, or make enough sense, then it makes sense to lower the threshold a lot of a moment. This corresponds to searching in a much wider meaning space radius for a new interpretation, presumably because the current one is insufficient.

Whether those are thought pump oscillations I cannot say. Since we see a frame every 1.3 seconds, this might flash like this all the time but we just did not have luck seeing it.

Perhaps thought-pump mechanisms are layered and come in local and global varieties?

Funny, those zebrafish guys analyze the activity in terms of cell assemblies: Neural assemblies uncovered by generative modeling explain whole-brain activity statistics.

They get these subnetworks, quite interesting. It has some philosophical overlap with the cell assemblies of Braitenberg and Palm that I describe here. But the methods are different. As far as I can see, they don't have an inhibition model; Which is one of the requirements of getting to these pockets of self-excitatory activation I talk about here.

Modeling a threshold-device

See Wilson, Cowan 1972. It is intriguing to consider the 'population dynamics' of inhibitory and excitatory neurons. Maybe all you need is a 'bouncy' excitation control,


Figure 7: From here, googled damped oscillations. This is the curve a spring makes, bouncing and then settling on some set point.

Maybe you get away with modeling such a bouncy threshold.

;; Imagine [I], the population of inhibitory neurons,
;; and [E], the population of excitatory neurons.

 ;;                                     P
 ;;                                     | (input from sensors etc.)
 ;;                                     |
 ;;                                     |
 ;; +--------+ <--------------  +-------+-+
 ;; |        |                  |       v |
 ;; |   I    |---------------|  |   E     |
 ;; +--------+                  +---------+
 ;;     ^
 ;;     |
 ;;     |
 ;;     |
 ;;     |
 ;;   inhibitory-drivers

If there is a lot of (sensor) inputs P coming in, your threshold control might automatically go into an oscillation mode.

Like pulling on a spring and then it bounces around. This might already be the mechanism to get thought pumps in the cortex.

I get an inhibition model with a few parameters:

  • The bounciness, how much I oscillate.
  • The set point of excitation, that I make the system go towards.
  • Maybe a min and a max of excitation/threshold.

This Would Like Event-Related Potentials And So Forth

It would go like this:

  • ++ sensory input, from the periphery or attention
  • Relatively high excitation makes the threshold device bounce
  • Then bounce is damped, or the activity is wiped or something.

The threshold device might also be cortico<->thalamic inhibitory interneuron things.

Alpha, Beta, and Gamma Could Be Thought Pump Steps

The interpretation at hand is that each of the oscillations is then the upper of the 2 steps through the pump mechanism.

An alternative, but perhaps both are happening is that they are composed of synchronous ensembles: How To Make Use of Synchronous Activation.


(Idea also from Braitenberg when he considered the thought-sequences).

Another thing you can put in the inhibition model is attenuation, this is an (inhibition) rule that says, 'if you were active recently, it's harder for you to be active now'. I.e. fresh neurons are eager.

This is one of the known properties of biological neurons, and it has a use in a thought pump mechanism.

In effect, we memetically select for 'freshness', at least temporarily. Attenuation is a memetic landscape concern.

Another way to put it is fresh memes are eager.

Consider: Verbal attenuation is very quick. I think I can say a word 4 times in succession and it becomes weird. 44 You can do this experiment yourself by speaking aloud and repeating any word at random. Unless you cheat in some way and imagine speaking the word in different contexts or so forth, after a short amount of repetition the word will sound weird. You might even have the feeling 'How is this even a word?'. 'This is strange'.45

Almost as if we would experience now the syllables and the sound, but the meaning is gone. See Semantic satiation.

The neurophysiological interpretation at hand is that some neurons that represent the meaning of the concept, are now becoming attenuated. I.e. the meaning is hard to re-ignite, sort of gone.

There is a satisfying reason why the attenuation would be high in some language cortexes. And why we would model different neuronal areas with different attenuation parameters.

With high attenuation, you get 'guaranteed' thought-pump sequences. Since we make it harder for the same cell assembly to stay ignited. In a thought pump jump, we heavily bias the system towards settling down on a fresh idea.

If we consider that language is virtually useless, unless it has some kind of syntax. It seems like there would be an evolutionary driver then, to make language cortex with especially high attenuation.

[My vision is the activity flowing like an ameba, or the 'ball of highest connectivity' moving from place to place].

More conjectures:

I think perhaps the opposite driver would be found in the cortex that represents 'meanings that stay'. This might be the visual scene - objects stay around without their meaning changing from looking at them. I think I can look at an object for more than 10 minutes and still don't feel like it becomes meaningless. This is in a way the opposite challenge to models that simply say 'attenuation makes satiation effects'. You need to also explain why there are no satiation effects somewhere else, then.

The cell assembly configurations in a way are the attractor states of the system. (I am not a math guy, but it sort of sounds like something like that).

Either way, I want to give this a name conflagrant modes, which comes from the Braitenberg paper.

Names for the possible configurations of activity:

conflagrant mode, ignition modes, supported activity configurations, attractor states, activity flow states, interpretations, representations

"My conflagrant modes tell me this is a bad idea". (When you have a hunch, meaning that some of your meaning spaces support the idea this is a bad idea).

"My current interpretations are joyful".

Names for the cell assemblies that are around a lot:

stable meme cities, well connected meaning cities, stable ideas, ideas that fit many situations

"This just gave me a spark for an idea."

"You just ignited some interesting ideas in my networks."

"These ideas are so out there, I have to build an ignition infrastructure to support them first."

"Your perspective just made me flip into a completely different conflagrant mode."

Here is a 'rolling malus' implementation in the browser.

;; Here, I do 1 simpler that is just carry over an attenuation malus
;; 'Rolling malus' implementation
;; 1. Every time you are active, your malus goes up.
;; 2. With every time step the malus decays.
;; 3. The malus is applied to the synaptic input divisively
;; kinda simplest thing, 1-3 could also be more complicated functions.

(defn attenuation
  [{:as state
    :keys [attenuation-malus attenuation-decay
           attenuation-malus-factor synaptic-input n-neurons
  (let [attenuation-malus (or attenuation-malus
                                  #js [n-neurons])))
        ;; decay the malus from previous step
        attenuation-malus (mathjs/multiply
                            (- 1 attenuation-decay))
          ;; accumulate the malus on everybody active
          (.subset attenuation-malus
                   (mathjs/index activations)
                   (mathjs/add (mathjs/subset
                                 (mathjs/index activations))
    (assoc state
      :synaptic-input (mathjs/dotDivide
                        (mathjs/add 1 attenuation-malus))
      :attenuation-malus attenuation-malus)))

Code Here


Figure 8: neuronal area with low attenuation, looks like the same cell assembly stays active.


Figure 9: neuronal area with more attenuation, cell assembly is forced to have temporal structure, later sensory overload I suppose. (That is perhaps the sensor input neurons become the most viable cell assembly, perhaps I should play with more attenuation decay).

Ah, the joy of seeing one's ideas inside the computer.

This playground is up here: Assembly Friends #2.

That is a random directed graph with some geometry. The geometry wraps so that at one end the connections go to the other side (like a torus).

There is a threshold device, too:

(defn rand-cap-k-threshold-device
  (fn [{:as state :keys [synaptic-input]}]
    (assoc state
           (ac/cap-k (rand-nth numbers) synaptic-input))))

;; ...

                             [(* (:threshold-device-high
                              (* (:threshold-device-low

That is a cap-k, going between 5 and 10% of the neurons or something; Randomly.

It is one of the simplest threshold devices you can come up with I think. There is a chance it already gets the job done.

It would not find G. Palms good ideas though. Those depend on a threshold, not a neuron count cutoff. Because a good idea would mean that you have a high threshold and a lot of activity at the same time.

The white box is a sensory field. Whenever one of the stimuli (colored balls) is in the white box, a subset of neurons in the neuronal area are activated (unconditionally) - the projection neurons of the stimulus.

Finally, the connections in the area are updated via Hebbian plasticity.

Input Circuits And Latent Spaces / The Games of the Circuits

I am a Murray Sherman fan; There is some cool work they did to establish these things empirically [Talk 1, 2, 3 these are all super dense and elucidating].

Also, I'm only picking the 50% that land on my views here; And probably leave out the most important parts.

An Input Nucleus To Rule Them All

  • All neocortex (vs. Allocortex, the evolutionarily preserved, old cortex) receives driving inputs from Thalamus nuclei in layer 4.
  • All sensory inputs to Neocortex go through the Thalamus. Note that olfaction is the exception, it goes to Allocortex.
  • Thalamus is in a strategic position to modify the inputs.
  • LGN is the visual relay nucleus, MGN auditory and so forth.
  • Relay nuclei make driving inputs to Cortex (v1,…)
  • modulatory inputs are shaping activity, driving inputs carry information and are the key inputs for activation. (this distinction is a key contribution by Sherman and collaborators).

Cortical hierarchical arrangments:

conventional view, *wrong*:

                       cortico-cortical connections,

        cortex primary                  cortex secondary
    +-------------+                    +------------+
    |             |                    |            |
    |   ^     ----+----------------->  |         ---+-------------> ....
    |   |         |                    |            |
    +---+---------+                    +------------+
    |^  |  |   Thalamus, "Relay / Gate"
-----+  sensors

In this naive view, you say that the cortico-cortical connections would make information flow inputs up a hierarchy of (perhaps 'feed-forward') cortical areas. Say (v1->v2->Mt->…).

As far as I understand, Sherman and collaborates primarily established the concept of driving vs modulatory input, which leads us to this update:

updated: Thalamus makes driving inputs to *all* cortical areas,

                                         +--  cortico-cortical are not driving inputs
                                         |    (their role is something more subtle)
          cortex primary                 |         cortex secondary
     +-----------------------+           |      +---------------------+
     |                       +-- ---  ---+  ----+>                   -+ --- - - > ...
     |          ^          | |                  |  ^             |    |
     +----------+----------+-+                  +--+-------------+----+
                |          |                       |             |
                |          |                       |             |
                |          +-----------------+     |             +------------> ...
                |                            |     |
           +----+------+                  +--+-----+-+
           |    |      |                  |  v     | |
           |^          |                  |          |        Thalamus
           ++----------+                  +----------+
            |   first order nucleus         second-order nucleus
            |   LGN, MGN, ..                Pulvinar, ...

driving inputs

- - ->
modulatory inputs

Note that this importantly includes most of the Thalamus, which is left out in the above 'model'.

  • Relay nuclei make driving inputs to the cortex.
  • A relay nucleus is in turn characterized by its driving inputs.
  • First order relay nucleus: Receives driving inputs from the periphery
  • Higher order relay nucleus: Receives driving inputs from the cortex (layer 5)

So this goes in a zig-zag with thalamus.

It is crucial, I think, to realize that the information flow is in a zig-zag, but the processing flow is parallelized across the relay nuclei of the thalamus.

'Bottom-up' vs 'Top-down' is somewhat misleading from this view, they label a virtual information flow that goes across elements, but those compute in parallel.

Similarly, 'ventral stream' and 'dorsal stream' are virtual information flows, but the computation is parallelized.

'Conventional' view:

                 cortical areas

             +------+ +------+        |
             |   <--+-+--    |        |
             |      | |    ^ |        |
             +------+ +----+-+        |
                           |          |
  +-------+----------+-----+-+        |
  |       |          |     | |        |
  |  <----+--     <--+---  | |        |
  |       |          |       |        |
  +-------+----------+-------+        |

                                      bottom up

    top down

A series of (probably feed-forward), serial cortical areas. Where information processing goes in a bottom up fashion from area to area. Presumably, each area represents higher derived feature detectors. And this might be modeled by neuronal nets, although this is contested.

At the same time, there is a top-down processing with big open questions, presumably constraining the information processing.

Updated view:

             cortical areas            thalamic relay nuclei    ^ |
                                                                | |  Virtual top-down and bottom-up
              +--------+                                        | |  information flows
              |     c4 | <------------------- d                 | |
              +--------+                                        | |
              |     c3 | <------------------- c                 | |
              +--------+                                        | |
              |     c2 | <------------------- b                 | |
              +--------+                                        | |
              |     c1 | <------------------- a                 | |
              +--------+                                        | |
                                                                | |
                                                                | v

a  - input relay nucleus
b  - second order relay nucleus, ...
c1 - primary cortical area, ...

The information processing flow is parallelized over the thalamic relay nuclei, i.e. information goes from a->c1, at the same time as b->c2 and so forth.

Perhaps it makes more sense than to say there is a virtual information flow, and that goes top-down, bottom-up. Note that the information processing arrrows are allowed to be shorter, this is the point of parallel processing. It means that you can do more in shorter timesteps.46

This model applies to ventral and dorsal processing streams, too.

My reasoning:

It's not clear whether it is the whole story, but the base problem of the (fMRI) so-called 'cortical areas' is explained by this. Cortex activation, presumably including the BOLD signal, would be driven by whatever thalamic nuclei they look at.

This fits with the observation from the infamous ferret routing study [Nancy Knawishers Lecture is nice]. If the primary auditory looks at LGN, you get vision and orientation columns and so forth in the primary auditory cortex.

The most parsimonious explanation I can think of:

  1. Neocortex is relatively generic.
  2. The neocortex will represent inputs, depending on what inputs come from the thalamus in layer 4.

It makes sense that evolution would hit on a more general-purpose computer and repeat its architecture over and over. Our computer chips are general purpose and there is a reason. It is easier to program. General purpose is more powerful than narrow purpose.

This doesn't rule out that some cortex is different from other cortex (see musings on attenuation above for a reason). What might be true in a weaker form is that all sensory cortex is relatively the same.

The strongest claim here would be something like: If you could experimentally swap say MT and Fusiform thalamic input circuits*, you would simply swap movement and face representing "cortical areas". Presumably without much loss of function.47

*) pre-development


(Got some of my questions answered by M. Sherman!)

  • Do relay nuclei ever branch to multiple areas?

Yes, it’s common got thalamocortical axons to branch to innervate multiple areas.

  • If the arrangement is relay nucleus -> cortex -> relay nucleus and so forth, isn't there one Cortical area at the end of this arrangement that doesn't have a relay nucleus as a target.

Good question. The arrangement is cortico-thalamo-cortical going up a hierarchy. This is feedforward. But we just published a study showing that there is a feedback version, so the highest cortical area would still project to thalamus. Evidence suggests that all cortical areas have a layer 5 projection to thalamus.

  • Do relay core inputs ever converge on a cortical area?

Yes, but it’s not clear what this means. For instance, one example is LGN and pulvinar innervating V1. The LGN input is basically driver, and the pulvinar input, modulator. Whether this is typical of other convergent examples is unknown.

  • Why have a V1 when the LGN is already representing the information?

LGN may crudely represent information, but it has very limited computational power. That’s why cortex evolved. A better question is: Why have an LGN? or Why not have retinal input go directly to cortex? We have tried to address that.

  • You might wonder if that sheds light 'functional connectivity' [Dynamic functional connectivity]

    Does it mean that the data is replicated? Or does it mean that multiple different processing streams are interpreting the data in different ways? Perhaps it means that tightly synchronous cell assemblies form?

    Perhaps it is hard for architectural or memetic reasons to have multiple large cell assemblies in the same area. (Would on the inhibition model the brain implements, if it implements an assembly calculus). (Perhaps multiple cell assemblies in 1 area simply don't make much sense, since they merge). Perhaps that would be a reason to have multiple processing streams on the same information.

  • Perhaps another element in the basic circuitry of the cortex is needed to explain it?

Note that the neocortex has 6 layers, allocortex has 3 layers. This implies that the neocortex evolved by duplicating the cortical layers, and by hooking up to the thalamus<->cortical organization. (Allocortex doesn't have layer 4-6).

This means that layers 1-3 are the evolutionarily preserved layers. Making it obvious again: You need to understand the thalamus; Neocortex is a thalamocortical concept.

Thalamic connection kinds

  • Thalamic nuclei are heterogeneous, we differentiate at least a matrix and a core system [Matrix-core]
  • Matrix is less well understood and projects diffusely to the whole Cortex. It receives inputs from the striatum and subcortex [got this from another talk]
  • Core is making the driving activation to the Cortex, projecting to layer 4 neurons.
  • Core neurons receive glutaminergic inputs from the periphery

My musings:

Matrix would be a candidate for an Hyperdimensional vectors [hd/vsa] implementation. If this is true, you would be able to read out a hypervector from the matrix nuclei.

At the very least, the interpretation that Matrix is implementing some kind of 'perspective', 'context', or 'landscape shaper' mechanism lies at hand. Surely, it must globally modify the attractor landscape of the system.

One might muse that such an element would help create mid-term memory. In that case, some yet unspecified process would somehow make the hippocampus store hyperdimensional meaning points of a holographic cortex encoding. (see Statistical considerations on the cerebrum / Holographic encoding? for why). Another unspecified process would recruit neuronal activity from the hippocampus to the matrix, giving the cortex a context/perspective. From there, the cortex could then fill in the rest of what we call the memory of a memory. (more below).

TRN the gate

  • Thalamic Reticular Nucleus (TRN) is like a blanket around the thalamic nuclei
      |     ^  C              |  Cortex
      +-----+-----------------+                     3)
      |     +->B    |         |  TRN  <-------------+
      +-----+-------+---------+                     |
            |       | 2)                            | drives TRN and inhibits Thalamus
      +-----+-------+---------+                     |
      |  ^  | A     _         |                     | Serotonin++
      +-----------------------+                     |
         |                  Thalamus   |------------+
---------+                                          |
 driving inputs                                     +---[brainstem etc.]

 modulatory inputs, cortex, subcortical

A - Relay nucleus driving neuron, makes projections to layer 4 in Cortex.
B - All relay neurons leave collaterals in TRN
C - layer 4 input neuron

2) - TRN inhibits back to Thalamus
3) - Neuromodulatory inputs from the brainstem and so forth are making strong inputs to TRN,
     also, Serotonin modulates the TRN

The TRN is a regular sheet around the thalamic input nuclei. The evolutionary drivers for TRN are easy to understand. It is an input gate. With low inputs from 3 (brainstem neuromodulator makers), in awake states, the gate is open. With high 3, in drowsiness [this is established experimentally with cats and so forth], the gate is closed.

TRN is controlling 1:1 how much activity flows into the cortex. This was one of the basic requirements you have for an activity management device, it controls its inputs.

So I am imagining a blue (blue are the sensors on my whiteboard) input nucleus ball, the ball receives inputs from the sensors, and represents the faithfully (drivers), there is a dynamic lid around the blue nucleus, that allows the machine to say how much of the activity flows into the thinking goo around.

Serotonin: At first glance the role of serotonin is ambivalent, it has to do with attention (cognitive neuroscience term), but also with drowsiness and sleep? [Lecture 8. The Thalamus Structure, Function and Dysfunction].

It lies at hand that closing the gate (activating TRN) is a mechanism happening during sleep. Shut down the inputs (mostly). This way the thinking goo can do something else, presumably it is relaxed from analyzing sensor data and performing as a behaving animal (broadly speaking).

Speculation: If we activate TRN, we shut down the inputs; Making the internal activity in the cortex less about the sensors and more about its 'own' context.

You can make this tradeoff presumably every time the sensor data is not that useful for whatever we want to do. Where the sensor data would somehow mess with the kind of information processing you are doing.

Note that it would be more useful if the default setpoint of such a gate is somewhere in the middle. This way, you could open the gate, and flood the cortex with inputs (awake, surprise?), or close the gate, and starve the cortex of inputs (drowsiness, perhaps thinking and imagination?).

I think we can assume this is a rather general element and operation in the system, useful for many kinds of mechanisms.

Of course LSD is a serotonin receptor agonist, and what is being reported from such altered states broadly maps onto what you might expect from the gate being closed, making the cognition be about itself - making the cognition about something like imagination perhaps.

From this, you will see that a maximally large dose of LSD will make you blind and deaf since the inputs going into the Cortex are maximally low, similar to a sleep pattern presumably.

It is ironic when substance users talk about opening their minds, while the mechanism that produces this state comes from shutting down the input gate to the world.

I suspect much more interesting and ultimately trippy thoughts are thought by building once understanding; Using pencil, paper, whiteboards and code editors as thinking aids;


  • Does TRN have any segments?

    [1 and 2 deleted for brevity]

  • TRN is activated sparsely and allows fine-grained control by cell assemblies spreading into it.

    (Perhaps the fact that layer 6 neurons innervate TRN speaks for 3.).

Contrast And Alternative

The relay nuclei of the Thalamus have inhibitory interneurons [I], which receive inputs from Cortex [layer 6]. Layer 6 cells also make branches into TRN and they make branches back to input neurons at layer 4 - quite the puzzle! Let's concentrate on the [layer-6] -> [I] part for a moment.

At first glance there is something strange in the light of memetics, this circuit violates the basic tenant activate your activators. It inhibits its activator!

This is not so if we assume that all activity is sparse [Valid assumption afaik].

Consider 2 memes now, being activated by some input neurons A. Both memes go and activate some random I inhibitory interneurons, which meme is more successful? It is the meme with the random wiring that makes it inhibit all its alternatives.

     |   [Ac]  [Ac1]   ^   | Cortex
        |       |      |
        |       |      |
        |       |      |   Bc is an alternative to Ac
        |       |      |
        |       |      |
     |  |       |      [B]  |
     |  |       v           |
     | [A]|----[I1]     [B] |
     +----------------------+  , where I inhibits [A] and [B]
                                 [I1] - inhibits [A]


     |   [Ac]  [Ac2]     ^ | Cortex
        |       |
        |       |        |   Bc's support is being inhibited
        |       |
        |       |
        |       |        |
     |  |       [I2]---|[B] |
     |  [A]                 |
     |                  [B] |
     +----------------------+  , where I inhibits [A] and [B]
                                 [I2] - inhibits [B]

Ac1 - hypothetical meme inhibiting its own activator
Ac2 - (actual) meme inhibiting its alternatives

If there is only so much excitability to go around (threshold device, see above), the memes need to compete for well-connectedness. You only need to be the best-connected guy in the network, in relative terms, not absolute ones. (Consider cap-k as an inhibition-model).

Therefore, Inhibit your alternatives is the second basic memetic move, and reified in this cortico-thalamic arrangement.

Murray Sherman mentioned this, too:

There is a problem here…

Please imagine that this circuit is based on this [cortex cells innervating relay cells, and neighboring inh. interneurons].

And we have lots of cells that are doing this at a single-cell level. Whenever one of these cells is active, some relay cells will be excited, others will be inhibited.

However, the techniques we use to study this pathway are limited because we don't activate single cells and record from single cells. Instead, we excite or inhibit whole groups. Now if you look at the way this is organized, if you were to excite all of these [many in layer 6], even if you use things like channel rhodopsin, the result is that all of these [thalamic neurons] will be excited or inhibited.

… If you had pattern activation, you'd get a very different response down here.

This is an important proviso for most of the experiments being done. When we talk about knocking down a pathway, lesions or chemical ways of reducing all these cells [whole populations of cells all at once] and then even with modern techniques. We are not seeing the actual way this circuit is activating/behaving. And it's for that reason I think that it's very difficult I think to get clear evidence for what the effect of this pathway is on relay cells. Because all the experiments I know of with very few exceptions either excite or inhibit large groups of these cells.

And you wipe out the beautiful, detailed single-cell circuitry that really is playing the role.

M. Sherman about layer 6 cortico-thalamic circuits

If we activate some patch of cortex neurons, there is a problem. It is like making all the pistons go up and down together in a car, you don't see the actual arrangement of what this thing does in vivo so to say.

In this case, you will see the inhibitory neurons being activated, inhibiting Thalamus->Cortex inputs. But the important thing in this arrangement is the fine-grained flow of what the activity does, you cannot make a mass action circuit analysis and expect to not see a distorted picture.

You cannot look at a circuit without considering what happens when the activity is flowing inside it. And the activity is sparse.

Parallel Processing Flows

Let me bring back the inverted pizza slice:

         \             /
          \     A     /                   meaning-level
           \         /
            \       / <----->   B
             \     /   inibit
              \   /
                X                        sensor-level

A - Stable cell assembly

Let's map this onto thalamocortical circuitry:

                                                      ....     .... n-relay-layers
                                                   +--------+                       +-------+
                                                   | H    1 |  secondary cortical   | H   2 | , .... 30+
         -----------------                         +--------+                       +-------+
         \             /                           +--------+
          \     A     /           meaning-level    | k      |  second order relay
           \         /                             +--------+
            \       /                              +--------+
             \     /                               | H      |  primary cortical
              \   /                                +--------+
             --\-/------------                     +--------+
                X                 sensor-level     | k      |  first order
                                                   +--------+  relay nucleus

A - Stable cell assembly

k - low dimensional neuronal area
H - High dimensional neuronal area


There is an entity one abstraction layer up from the neurons and the circuits: The cell assembly. A cell assembly can stretch meaning and sensor-level neuronal areas. We can say it is a data structure, created by the fundamental operation of pattern completion.

From biological reasoning [could have been otherwise, but consider the anatomy], it makes sense to locate this sensor level at the first-order relay nuclei of the thalamus.

Assuming that 1 thalamic relay nucleus can have multiple cortical area targets, the information flow would fan out. [Reasoning from the existence of ventral and dorsal streams, how could this be made except by splitting the paths somewhere?].

Considering that there is much more cortex that is not primary sensory areas, we can assume that there is roughly something like the inverted pizza size shape; If you map a cell assembly to neuronal areas, the 'meaning level' parts of it are necessarily larger than the sensor level parts.

Further, this circuit supports one of the fundamental notions that we get from memetics. Via the inhibitory interneurons of thalamic relay nuclei. (Described at Contrast And Alternative).

Inhibit your alternatives, on every level of meaning.

         ----------------- -- ---
         \       \     /        /
          \     A \   /  B         ....
           \         /        /    k
            \       /        /     H
             \     / \             k
              \   /   \    /       H
                XA ---| XB       thalamic relay nucleus
            A inhibits B on the sensor level

A - Stable cell assembly
B - A possible cell assembly, without enough support from the dynamic network to be active

You see from the cell assemblies clearly, whatever interpretation the system is expressing at the moment at large, i.e. which assemblies are stable. These will modulate and create contrast at the level of the sensor inputs coming in - down to first-order thalamic relay nuclei.

In this view, the thalamus is far from a simple relay. It is an essential component of the memetic landscape of the cortex. The memes in the cortex modify the inputs to the cortex. To fall into stable interpretations, which shape the inputs.

Why do they shape the inputs? To reproduce themselves.

The purpose of activity flow is to reproduce itself.

Parallelism and Multisensor Integration

An assembly calculus will integrate multisensor information immediately, just as well as 'uni' - modulatory inputs. All you need are areas that mix inputs from multiple input streams.

This is a very important requirement for brain software.

A Curious Arrangement

From Sherman:

  • driving inputs are glutaminergic excitatory synapses.
  • We can say there are first-order and higher-order Thalamic nuclei, they are defined by their driving inputs.
  • Every cortical area receives driving inputs from thalamic nuclei in Layer 4 (Neocortex).
  • An example of a higher-order nucleus is the sub-nuclei of the pulvinar, they go to the 30-something visual areas in the cortex []
  • A higher-order thalamic nucleus receives driving inputs from the Cortex itself, from layer 5 neurons.
  • Curiously, all input neurons to higher-order thalamic nuclei make branching axons. The other branch seems to go to motor* centers. Asterisk because this was preliminary. Perhaps it is 'lower targets', not 'motor targets'.
  • Why have this branching axon? Sherman speculates this might be the biological implementation of efference copies. Consider when moving one head, almost everything in the retina/sensors changes, but the brain somehow should keep track of this 'self-movement' and factor it out in its perception.
  • You can see this yourself by pushing gently on the side of your eyeball. The world moves a little. The fact that this is not the case when we move our eyes shows that some processing is canceling out the changes in the world coming from one own movement. In order to do such an information processing, efference copies were postulated very early.
  • Observe that the efference copy in this model is the one going into the higher-order nucleus. Since the other branch goes to motor centers.

      cortex 1                           cortex 2
      e.g. v1                            e.g V2

+----------+---+ 1-3               +---------+-------+ 1-3
+--------------+                   +-----------------+
|              |                   |                 |
| ^            | 4                 |        ^        | 4
+-+------------+                   +--------+--------+
| |          | | 5                 |        |       || 5
+-+----------+-+                   +--------+-------++
  |          |                              |       |        pattern repeats, up the higher order nuclei
  |          +---------------+ 5a)          |       +------------------------+
  |                          |              |                                |
+-+---------+                |    +---------+--+                             |
| |         |                |    |         |  |                             |
| |         |                +-+--+-->      |  |                             +------->
|           |                  |  |            |                             |
+-----------+                  |  +------------+                             |
   first-order relay nucleus   |   higher order relay nucleus                |
   e.g. LGN                    |   e.g. Pulvinar (1 of ~30 subnuclei)        |
                               |                                             |
                               |                                             |
                               |  5b)                                        |
                               |                                             |
                               |  motor* centers                             |
                               v  colliculus, red nucleus, etc.              v

5a) A layer VI neuron making inputs to a second order 'relay' nucleus, branch a.
5b) The branch b of the same neuron, going motor- or similar kinds of targets.

This is challenging to the conventional view, that cortical areas are somehow functionally specific. Here we have a circuit that implies that all cortex does at least a little bit of movement.48

This fits again with the view that the cortex is made from relatively generic, general-purpose elements. Additionally, such an element would affect animal behavior. So it would be immediately useful evolutionarily. The hypothetical proto-neocortex ('single area') would go to behavior targets, presumably already from layer 5. The input organization of this cortex would evolve together with the cortex. The logic of the efference copy would be a strong evolutionary driver for making the layer 5 output neurons branch, leaving collaterals in the input nucleus. Eventually, the targets of these collaterals would have an evolutionary driver for specialization, making a new nucleus. The whole organization would now have a reason to evolve a duplicated cortex element, which analyses the latent space, the output of the first area.

Something like that, perhaps the order is swapped somewhere.

What is the role of the cortico-cortical connections? Probably they are important, too.

From the lens of the cell assemblies and the memes:

To analyze a circuit from the view of memetics, we can imagine a few different 'possible memes', let them play the game of the circuit in our imagination, and see who has the most success.

Making the biologically plausible assumption:

All wires are kinds of wires.

The kinds of pre-and postsynaptic neurons, how you branch, to which nucleus you go and so forth are allowed to be determined darwinistially, but that the actual fine-grained connectivity is simply random (at least at the start).

Let's say we are some memes that are active in v1, looking directly at some sensor inputs. So the activation that makes us active is from a first-order nucleus, coming from the sensors.

The prime goal of the meme is to stay active. If it can bias the system to keep the sensors the same, it will do so.

From the circuit above there are 2 games these memes need to play:

  1. I spread into some layer 5 neurons, which make the appropriate motor output that keeps the sensors the same because this is where my activity comes from.
  2. I want to spread into the higher-order nucleus because this is my primary way of influencing the rest of the Cortex to keep me active. I.e. a meme that spreads to the higher guys and has friends up top will be selected compared to memes that don't do that. I.e. cell assemblies that stretch across cortical areas have more connectivity.

So 1 and 2 are two different memetic drivers, 2) makes you want to spread into as many layer 5 neurons as possible, but 1) gives this a twist: The behavior stream you are outputting shouldn't change the situation away from the situation that made you active in the first place.

The cell assemblies now have a memetic driver to output a stream of possible behavior (at layer 5 neurons). A stream of possible behavior that keeps them active.

You can look at a line in your visual field, and the cell assemblies in your primary visual areas make outputs at layer 5 that correspond to eye movements, staying on that line in the visual field.

Whoever is looking at the higher relay nucleus now (the efference copies of those movements), sees an encoding of things like lines, color blobs, edges and so forth. In the language of possible eye movements.

This would be to a low-dimensional latent space between the high-dimensional cortical areas.

It would mean that the latent space of the cortex is encoded in behavior streams, which keep the sensors/situation the same.

Let's imagine some red sensor memes in v1 when we are looking at an apple.

Now this circuit forces the memes to play with open cards in a way I think. You want to spread into the higher relay nuclei and influence the rest of the system, but the circuit forces you to make some motor output at the same time.

Imagine meme 1, which has a few random connections to the motor output, making the animal move its eyes for instance. This is not a better meme compared to meme 2, which makes the eyes stay in the same spot -> My sensors stay the same, so I stay active.

But meme 3 is even better, it spreads maximally into layer 5, and it outputs a range of movements; A range of movement that will keep the eyes inside the patch; Stopping exactly at the edge of the patch.

I am imagining the following experiment:

  • Make a monkey look at a red circle.
  • Somehow read out the activity coming from v1 to motor targets (the branching axon 5).
  • If you would run a decoder (or whatever you do), you would be able to read out a range of eye movements.
  • For a large circle there is either a correspondingly small or correspondingly large movement range signal, depending on whether this is encoding the freedom or restriction of movement.
  • Or, if it is a mix of such signals, you would have to decode more cleverly. Or perhaps setpoints are communicated, not procedural movements. Or a mix of those.

Alternatively, you would knock out all other motor pathways except v1; I think you would see eye movement. Small eye movement inside small patches of color, large eye movement for large patches of color. Line tracing movements for lines and so forth.

The memes that hit on the right connectivity to make movements that keep the sensors the same, will simply be selected in favor of the ones that don't. But the game is multilayered, the overall cortex context comes into play, too.

The memes that find the widest possible range of movement, which keeps the sensors/inputs the same, have a greater selection advantage still. Because it is those memes that spread most efficiently into the higher thalamic nuclei, and thereby have a chance to form cell assemblies with the higher cortical areas.

Ever since I came up with this idea, when I look at objects I wonder. Isn't there sort of a wall you hit when you move the eyes inside the edges of an object?

There is one other thing that comes to mind; Which is tactile mitgehen.

Alien hand syndrome Alien hand syndrome has been shown to be prevalent in roughly 60% of those people diagnosed with CBD.[6] This disorder involves the failure of an individual to control the movements of their hand, which results from the sensation that the limb is "foreign".[2] The movements of the alien limb are a reaction to external stimuli and do not occur sporadically or without stimulation. The presence of an alien limb has a distinct appearance in CBD, in which the diagnosed individual may have a "tactile mitgehen". This mitgehen [1] (German, meaning "to go with") is relatively specific to CBD, and involves the active following of an experimenter's hand by the subject's hand when both hands are in direct contact. Another, rarer form of alien hand syndrome has been noted in CBD, in which an individual's hand displays an avoidance response to external stimuli. Additionally, sensory impairment, revealed through limb numbness or the sensation of prickling, may also concurrently arise with alien hand syndrome, as both symptoms are indicative of cortical dysfunction. Like most of the movement disorders, alien hand syndrome also presents asymmetrically in those diagnosed with CBD.[7]

Corticobasal degeneration

Let's assume for a moment that in CBD the usual (frontal) motor pathways are impaired. Perhaps this is freeing up the memes of S1 and so forth, to make movement that makes the sensors stay the same.

If this is true, you would say that mitgehen comes from s1.

This idea doesn't depend on the branch b of the layer 5 neurons to go into motor targets. Could be 'chemical balance' or whatever nuclei, too.

I feel like cell assembly memetics can elucidate all mysteries of the circuits of neuroscience.

Consider the circuit, consider the memetic games of the circuit. Imagine a few alternative memes.

Memes know no boundaries, they will spread into whatever circuitry is available. If the circuitry forces memes to play a certain game, they will do so.

Claustrum speculations:

My reasoning: Claustrum looks like an element that implements some machine-level operation. It is orderly connected to everything else in the neocortex, stimulating it leads to loss of consciousness [Koubeissi et. al (2015] 49. It is at least interesting that this would overlap with what we expect from the hypothetical threshold device (further up). Stimulating it would either confuse this element to think that there is a lot of excitation, which needs a high threshold to keep in check. Or it would directly excite the inhibition neuron populations - In both situations, we would expect the activity of the cortex to drop. Perhaps leading to unconsciousness. But this is just speculation.

Further, there would be a class of epilepsy that would all be some kinds of claustrum pathologies. The opposite might also happen if you have too much activity in the claustrum - epilepsy in the claustrum. If the claustrum is the threshold device, this would lead to profound inhibition of Cortex.

It might be otherwise, but what if the mechanism of Cortical spreading depression is some kind of epilepsy in some inhibitory nucleus? Whatever that nucleus is, it is powerfully inhibiting the cortex. But what in neurobiology travels at 1.5-9.5mm/min? Some kind of offending activity, spreading somehow in a direction that is usually prohibited by the circuit perhaps. Perhaps something usually goes 1000 that fast, but in the other direction?


There is a literature on the 'microcircuits' of layer 5 output neurons.

…modulation of corticostriatal activity was shown to alter behavioral output in an auditory discrimination task (Znamenskiy and Zador, 2013), suggesting that layer 5 subnetworks may be differentially engaged by sensory inputs on the basis of current expectations or environmental demands.

Layer 6 cells


The Biology of Cell Assemblies / A New Kind of Biology

The cell assemblies then, provide a perspective on the structure and function of the cortex and its nuclei. (1 out of 12 required).

The cortical network is the meaning, but the ideas live inside the network.

Pieces of mutual excitatory sub-networks compete for activity sources. These can also be seen as stable data structures in a high-dimensional computing framework [Dabagia, Papadimitriou, Vempala 2022]2, representing the inputs.

The biology of the cell assemblies is a biology of hyper-dimensional, agenda-having, knowledge-representing entities, operating in subsecond timescales.

The basic tenants of this cell assembly memetics (what I have come up with so far):

  • A random directed graph with an idealized inhibition model, discrete time steps and Hebbian Plasticity [Hebb 1949, Kandel 1965] implements assembly calculus [Vempala 2022].
  • After the presentation of inputs to the network, a stable representation (called cell assembly) of the inputs forms rapidly.
  • (Note that we are allowed to require on-the-fly plasticity [from neurophysiology])
  • Cell assemblies are hyperdimensional: They are made of many small data points. Pattern completion is the basic operation of Cell Assemblies.
  • The timescale is sub-second. We only need a few neuron time steps to form stable cell assemblies. [per theorem, Vempala 2022]. (For igniting existing cell assemblies from partial inputs, I suspect you get a substantial part of the cell assemblies after a single timestep and the full one after very few. That is just from me hacking around with an assembly calculus model.)
  • Cell assemblies compete with their alternatives, with the best connected sub-assembly winning.
  • Assuming an idealized inhibition model, we can select the best connected cell assembly at each time step. [Braitenberg 1977]
  • Cell assemblies are subject to natural selection, cell assemblies are replicators with tiny agendas. I.e. assuming many possible connections initially, well-connected sub-networks compete for well-connectednes. In the context of the driving input activity. In other words, stable cell assemblies are the ones that exist. We can see that cell assemblies are replicators, replicating themselves across neuron timesteps. (But strategic cell assemblies exist, too. Biasing the rest of the system to be active again, somewhere in the future, not necessarily the next neuron time step). Note that gene-centered and extended phenotype views [Dawkins 1976, 1982] apply to memes, too. Giving us a large explanation lever on their structure and function.
  • Activate your activators is the basic memetic move. (The basic property of a cell assembly).
  • Memes are selfless: Memes will merge, given the chance. [from assembly calculus]. With assembly calculus, we can activate 2 cell assemblies together, and find an overlapping cell assembly. Another way to put this is that symbiosis is a primitive operation for the memes. Mutual activation is literally how the cell assemblies are formed in the first place.
  • Causality is the most fundamental concept, (see Hebbian rules, it is a 2 time step concept)
  • Cell Assemblies have temporal structure, or "causality flow" unless their activity goes in a tight mutually activating subpopulation.
  • I.e. association without direction is when the activity goes back and forth.


  • Inhibit your alternatives is the second basic memetic move. (and supported via thalamic inhibitory interneurons, innervated from cortex layer 6, see Contrast And Alternative)
  • The meaning of a meme is the connectivity of its sub-networks. (This is only useful if the activity comes from the sensors and ultimately moves effectors).
  • The basic agenda of a meme is I will be thought again, or I shall stay active.
  • Memes don't know boundaries (memes spread into all wires available).
  • Neuronal ensembls only care about some other part of the brain, if that helps to make them more stable.

    Stability Before Spread, No Stability, No Spread, No Gain, No Chain.

    I.e. If I don't gain stability from you, then I don't make chains with you. Or I don't spread my ensembles.

  • This effect is excerbated if memes are competitors, and another area is gain
  • All wires are allowed to be 'kinds of wires'. The actual connectivity is allowed to be random, because in the second step, the memes will populate the network, and non-sensical wires will simply not be active.
  • All circuits in neuroscience can be understood in terms of the game of the circuit, the memetic games and the drivers a circuit creates for cell assemblies to play.
  • Because of the circuitry (see Contrast And Alternative on this page), whatever meme wins out fastest will go and inhibit its alternatives. Whatever meme is fast wins. (But perhaps during dreaming only parts of cognition are tried out, reducing the grip such a 'best meme' has on the system).
  • The computational paradigm of cortex is simply high-dimensional interpretation or representation, pattern complete, "template filling-the-blanks"; Or 'ongoing situation analysis using best guesses of explanation'.
  • Perhaps the memes can be said to participate in the interpretation game. Whatever memes are representing an expectation structure that 'fits' the current situation, are on.

The inference that can be drawn from this is that the relevant "things" of our experience are represented within the brain not by single neurons but by sets of neurons. Just how many neurons are involved in the internal image of a thing like "my house" or "the neighbor's dog" or "the tune of Greensleeves" is very difficult to say, but it is likely that their number is too large for any neurophysiologist ever to be able to record their activity by multiple-electrode recording. The sets of neurons that stand for certain events, or objects of the outside world, the "cell assemblies" as they were called by Hebb50 must have a certain internal coherence, since it is one of the basic observations of psychology that partial evidence of a thing tends to make us perceive the whole thing. This is best explained by supposing that the elements of a cell assembly are coupled by excitatory synapses, so that the excitation of some of them ignites the whole set and leaves all of them in a state of excitation until their activity is again extinguished by external inhibition. If we want to localize cell assemblies more concretely in the cortex, we notice that they are of two sorts - restricted to particular areas, or diffuse. The cell assembly "my neighbor's dog" represents within my brain a very complicated bundle of properties in various sensory modalities and must occupy almost the entire cortex. On the other hand, the cell assembly "Greensleeves" may be localized in the auditory region of the brain and may indeed persist undisturbed by any other activity my cortex may be involved in.

Some more properties of cell assemblies come to mind. Some are essentially synchronous, like motionless visual images, others have a temporal structure, like the cell assembly representing Greensleeves. The synchronous ones are composed of parts any one of which may recall the others, while the asynchronous ones lack this symmetry, since obviously it is much easier to recall the tune of Greensleeves starting from the beginning than from the end. However, some temporal structures, such as the words of a language, seem to be represented almost in a synchronous way, since it is about as easy to find words that rhyme with a given word as it is to find words beginning with the same letters: the asynchronous cell assemblies representing the words can be activated in the direction of time as well as in the opposite direction.

(Braitenberg 1977)

Note that a model that talks about using the whole cortex for single concepts is very much out of favor in current cognitive neuroscience. But until they have a model of cognition that talks about what areas are doing and why concepts don't span Cortex, I just assume that the anatomy and the functioning of the Cortex come across like it is mixing notions of local with global information processing flows. Perhaps the ultimate notion of global processing on this page is one of the holographic encodings of the cortex.

Either way, when thinking with cell assemblies, we are freely assuming that cell assemblies simply span the cortex. We can say there is a global cell assembly, made from all the current sub-cell assemblies. This is simply a matter of terminology.

The Structure And Function of The Cell Assemblies Is Their ad-hoc Epistemology

Audio diary (stream of consciousness evolving ideas, better at 2x speed):

  1. Magic Gem Epistemology of Meaning-Level Wizards
  2. Musings On The Epistemology of Cell Assemblies
  3. Small Summary And Musings How Context Is Implementing Procedures In Assembly Calculus
  4. Speculations On The Mechanisms Of 'Good Ideas'

We can assume that evolution hit on a knowledge representing substrate. This substrate is populated memes - pieces of knowledge that replicate. This is only useful evolutionarily if this knowledge is useful.

It is fair to say that humans inhabit the cognitive niche (Pinker). Or something along the lines of the purpose of humans is to create knowledge (Deutsch).

The brain must support what I call an ad-hoc epistemology.

Alternative names: biological rationality layer, animal epistemology, living epistemology

Memetic Epistemology

In The Stuff of Thought (2007), Steven Pinker draws a very broad and deep picture of language and children's language acquisition. (also Hofstadter and Sander 2013, for a similar perspective).

I think the nature of cell assembly epistemology must be Popperian. It must make models, and conjectures of how things work that are simpler than the things they explain (maps, not territories). Because they are represented in the brain's network, which is only so big, in a world that is that large.

Roughly, there must be first the ideas, and then a pruning algorithm that leaves the ideas that make sense left over.

But this is allowed to be meta, the processes that make ideas are also allowed to be selected and grow themselves.

Natural selection with a parallel high-dimensional computing flavor. It is that there are many ideas possible at each time, and the best ideas are found, in parallel (so fast), across many possibilities.

The cell assemblies must go forth and grow into explanation structures of the world. How to represent an explanation of the world? By making expectations about the world. If the fundamental operational paradigm of the system is 'make situation analysis until the interpretation does not change anymore'. Then it is the memes that find the fastest explanations, which fit the rest of the system the fastest (sensors, motors, rest of the cognition), that will have the most success.

Consider these two versions of the meme B:

+------------------------+  thinking machine
| A -> B -> C -> ....    |  keeps going to new states

+------------------------+  thinking machine
| A ->[B]                |  done thinking, B is stable

The second meme B is the better because it keeps being active thenceforth.

Every meme has the driver to make the machine stop thinking. In other words, the system has a bias towards falling into stable attractor states and staying there. All the neuronal activity will go out of its way to shape the network into falling into attractor states (their attractor states) fast.

How do you make it stop thinking? By doing the hard to fake [Deutsch] thing perhaps, be connected to the rest of the network in a way, that you simply are the best connected interpretation available to the network (see thought pump).

If B says "I see a white gold dress", you are done. This is a temporarily allocated meaning structure, spanning meaning-level and sensor-level sub-interpretations. That says high-level "There is an object there it is a white gold dress". And low-level "I expect to see white when I move my eyes here". This allows you to hook in other meaning structures, like the notion of the photograph of the dress, how you would describe it in words and so forth.

Note that this expectation structure will use other tricks I talk about on this page. It will inhibit its alternatives via Thalamic layer 6 neurons. It will make your eyes stay as much inside the dress as it can.

For all it cares, it wants you to explain the world now in terms of white gold dresses. Because if it is implicated in many things, it is on.

[to be continued].

Cell Assemblies Have Drivers For Generality and Abstraction

Cell assemblies want to be active. There is an obvious memetic driver for generality and abstraction, then.

Consider how children will over-generalize rules of language and pieces of knowledge. Pieces of knowledge are alive, they have an agenda. They want to reach as far as possible; Because if they do so, they are active more. And that is the 1 thing that memes want.

I was imagining high-meaning level memetic wizards. The wizards can use the magic of their matrix (see Memetic Engines Create Competence Hierarchies Up To User Illusions) to create little gems.

A gem is a piece of the network that receives more or less activity flow. (It's of course a cell assembly again). A gem will glow if the current context supports its activation.

If I have a piece of network that is supported by yellow and long-round shapes and the smell of bananas, that is allowed to become a gem, representing the abstract notion of a banana.

Higher-order memes will care about associating with useful gems. They will have an agenda for the gem to be as general and abstract as possible. To reach into many meaningful places.

The wizards have 2 new memetic drivers: 1: Make useful gems and 2: Make the gems you have more general.

A banana gem was glowing 3 to 4 times right now in your brain, and at least 1 wizard was happy about his gem glowing. That wizard will go out of its way to play the game of the matrix (virtual meme world) to grow a banana city of meaning. A city of meaning that makes associations with his gem.

Not because he knows what the gem means, not because he cares, but because it is a source of activity. And all activity that reproduces itself is the activity we see around.

At the user level of this meme world, some processes indulge in collecting many, beautiful, abstract gems, and giving them labels and pronunciations, the words of our languages. A gem is useful to the wizards that associate with it, and even more useful if it has a pronunciation.

They don't know it, but the gems contain knowledge. Knowledge that is represented in their connectivity to the network (the wizards don't know about that either).

A wizard simply sees a gem that is active sometimes. If the wizard receives activity flow from such a gem, he will have an agenda to make the gem active more. If that is a banana gem because it's connected to the taste of banana and the color of banana and so forth, then the wizard will have the agenda to make the abstract notion of a banana, which is active for all bananas. Wizard and gem are symbiotic memes.

In other words, it seems like there are memetic drivers on all levels of hierarchies to build Hofstadter meaning-cities bigger [2013].

The Computational Substrate of Cell Assemblies Makes Them Merge

The cell assemblies are strange. The substrate [assembly calculus], forces cell assemblies that are active together to merge. Creating a new cell assembly which is neither one nor the other. (At least temporarily, depending on the plasticity rules and so forth).

That is cell assemblies leave their identity behind to become something new. They are selfless in this way so to speak.

We can observe from replicator theory that the memes that merge in useful ways with other memes are the ones we see around. A memetic driver for merging with the right crowd, then.

There is no other replicator that I know that can do this; Except maybe ideas and thinking. Perhaps this is a hint that we are looking at the computational substrate of ideas.

Further, from the same logic; The cell assembly doesn't care about its neurons. It will simply leave neurons behind if that makes it have better connectivity. We see this a little in the logic of how cell assemblies form from input projections. It is not the initially activated neurons that form the cell assembly. These neurons are only a stepping stone for the activity to flow into its stable subnetwork.

This is another hint to me that considering the neurons is one abstraction layer too far down. A Darwinistic perspective takes the neuron as the selfish replicator agent is in principle viable, but less useful than the cell assemblies.

Context Is All You Need?

What is a hypothetical programming language on top of assembly calculus? What are the means of combination, what are the means of abstraction? [Sussman, Abelson 1984].


  • Which cell assemblies are active? The ones that are supported by the current activity flow in the network.
  • Assume for a moment are 2 main branches of activity flow top-down - the context, and bottom-up - the sensor inputs.

If the context is lines, edges, three-dimensional objects, or a few of them, then the system can 'go in harmony' with all the inputs it has. It is interesting to speculate how mechanisms like the thought pump above can steer neuronal activity into some 'useful' interpretation. An interpretation that 'fits well'.

There were these schemata from psychoanalysis. They are similar to the small situations of Hofstadter, too. The schemata are templates, which you use to build your psychology around or something. I think they are a good idea after all. We can make templates out of cell assemblies, they can be a piece of context and a procedure. It says 'pattern complete this'. This is what we can give the rest of the high-dimensional, parallel computing framework. The answer is then

  1. Whatever comes out after n neuron time steps


  1. Whatever comes out after n thought pump steps (oscillations)


  1. Whatever the system will stabilize with. Perhaps this state is reached quickly, never.

We can observe that there is a memetic driver for 'making the machine stop thinking'. Presumably by simply being well-connected enough so that even thought-pump steps don't change the set of active cell assemblies anymore.

This is the ideas getting into harmony with their context, their niches.

This is an extremely powerful idea because we make the means of abstraction in this programming framework the same kind of stuff as the fundamental data structure!

In other words, what we need to make arbitrarily interesting computations in this framework is simply situations, or templates, or contexts, or programs, or schemata.

The templates are represented by the same primitive data structure, the cell assembly.

So Cell assembly procedures. Those procedures should be parameterized, and I have a hunch how that is done. Since we are hyperdimensional, you can always express a mix of vague or I don't know yet information states. We can allow the notion unspecified until further data is gathered, but here is the direction of my hunch. These hunches, niches, and contexts, are allowed to be filled dynamically.

For instance, there would be a procedure that says I recognize objects in a scene. It doesn't say which objects, and it is good if it doesn't say. That is filled dynamically by some more sensor-driven parts. But it doesn't usually fill it with something else than a 'parsed science with objects', because that would not fit the context. (That would lead to major confusion I suppose, more below).

Something else is useful in this arrangement: The vague ideas are already useful from the beginning. This is a gradual information processing paradigm. A vague hunch I see x is useful already, even before details of x are supplanted.

Biologically, this is a very elegant design because now all the replicator theory/memetics of the cell assemblies would apply the procedures, too.

It is like having a programming language made from living play-dough. The substrate and its properties come out from the high dimensional computing framework the brain implements. If it is something like assembly calculus, then your fundamental operation is pattern complete. You get infinite malleability, just juxtapose and mix different information, and the computing framework will fall into an interpretation, which fits (a.k.a. it finds cell assemblies that have a lot of support from the currently active network).

The functions of brain software are simply memes, too. They are living procedural, parameterized, abstract data structures.

They want to be general, they want to be harmonious, and they want to make good languages, and good building blocks (see below). They are like clay, ready to be merged with other ideas, ready to be juxtaposed to get dynamic new stuff out. They want to be magical to use, this is how they play the memetic game of having friends higher up.

A computational framework where situations are the primitive means of abstraction.

I am imagining the high-meaning level wizards, putting together magic gems and playdough. The gems glow with activation, if they are supported by the sensor activation (thalamic input organization). The wizards that find general explanation structures, will be able to find powerful interpretations of the world. I.e. they will find meaning-level representations that are supported well by the sensors (Perhaps programming such a system would right now yield a mind on the spectrum, I'll talk more about the circuits that would support far-reaching imagination and superstition later).

Note that cell assemblies are allowed to have a temporal structure. The need for this is self-evident from considering how to represent motor movement. (More on such things from G. Palm 1982). In case you wonder if such a pattern complete can live up to rich, temporal sequences and so forth. In principle, that is covered by cell assemblies. Though perhaps there is more circuitry to build into one model. Perhaps some special areas or nuclei facilitate the temporal structures of cell assemblies.

I feel like I found Hofstadter's situations [Hofstadter and Sander 2013]; He went from the top 1 layer down and started talking about the cities of meaning. I am (attempting to) look from the bottom and here they are. Living networks of meaning with the agenda to grow.

Building Blocks / Harmony / Selfish Memes

From the selfish replicator view, there is a funny twist: They make harmonic teams, if they can. Dawkins explains this as the rower boat teams, it is those rowers that are useful together when thrown into a boat that are successful.

If the cell assemblies make ad-hoc epistemologies, then the ones that work well together, symbiotically so to say, are the ones being selected. This is a memetic driver for specialization, harmony, and elegance.

Elegance in the sense that if some memes make a good language, they have a memetic driver to be a good language. This is a deep topic of software engineering and design.

In other words, memes have a driver for making elegant ad-hoc languages, which are so easy to use and put together for the higher meaning-level memes in the system, that they are magical.

Confusion and Socratic Wires

Socratic Wires, Champaign Bottle Memes, 1 idea about confusion mechanisms; Via absence detectors or null hypothesis element cell assemblies in intermediate meaning levels.

The absence of an explanation structure can be represented by cell assemblies, representing I know that I don't know. This is very useful to a system that (somehow) rewards good explanations.

  • Socratic sensors on intermediate meaning level

    Let's pretend we have a network laid out roughly like so, 1-3 are neuronal areas, or virtual neuronal areas (the layout would depend on the topology, these things).

    For some reason, if you have a network that represents sensors, then you make another network representing that representation, in turn, you get higher meaning representations [Rosenblatt].

        derived-meaning-level                   sensor-level
         +-----------+  +-----------+  +------------+
         |           |  |           |  |            |
         |           |  |           |  |            |
         |           |  |           |  |            |
         |   B       |  |           |  |            |
         |           |  |  M  M     |  |         A  |
         |    -------+--+>      <---+--+-------     |
         |           |  |   M       |  |            |
         |           |  |           |  |          ^ |
         +-----------+  +-----------+  +----------+-+
           3              2             1         |
                                            sensor activity
    B - high-meaning level cell assembly
    A - sensor-level cell assembly
    M - hypothetical intermediate-level cell assembly

    Case 1: There are some cell assemblies M active, they get support from both A and B, in effect forming a large cell assembly AMB, which spans all meaning levels. I speculate that such an arrangement corresponds to some kind of fitting explanation structure. By forming AMB, the system can predict its sensor input and so forth.

    Case 2: A and B do not "agree", their connectivity into 2 is not overlapping enough, and M is not ignited.

        derived-meaning-level                   sensor-level
         +-----------+  +-----------+  +------------+
         |           |  |           |  |            |
         |           |  |    ? <----+--+--------+   |
         |           |  |           |  |        |   |
         |   B       |  |           |  |        |   |
         |           |  |     ?     |  |         A  |
         |    -------+--+>          |  |            |
         |           |  |    ?      |  |            |
         |           |  |           |  |          ^ |
         +-----------+  +-----------+  +----------+-+
           3              2             1         |
                          M-box                   |
                                            sensor activity
    B - high-meaning level cell assembly
    A - sensor-level cell assembly
    ? - A and B try to ignite potential cell assemblies in 2, but there is not enough support

    I'll give 2 the name M-box, for 'middlebox' for a moment.

    The ? in M-box is interesting. It would represent the absence of an explanation structure that fits well from top to bottom.

    How to represent this absence in terms of cell assemblies again?

    Idea 1:
         |  0S  |
         |  ^   |
         |      |
         |      | null hypothesis nucleus

    Make a null hypothesis nucleus (presumably you put that element or instances of this element everywhere in the whole organization). If the null hypothesis nucleus is making an intermediate, random input to each area, then all cell assemblies have to compete with it. If the thresholds are balanced well, you would have the situation that A and B from above need to provide enough overlapping connectivity/activation, to win out agains the null hypoth

    The presence of the 0S cell assemblies in M-box is then allowed to represent the absence of a 'good explanation that fits everything'.

    Idea 2:
          M-box                   next cortical area
         +------+               +-------+
         |   ?  |               |       |
         |     -+ --            |   0S  |
         +------+   |           +-------+
                    v               |
               +--------+           |
               | _a ----+-----------+
               |        |
               | _a     |
               higher thalamic nucleus
    _a - absence detectors via inhibitory interneurons
    0S - Socratic cell assemblies /I know that I don't know/

    It is useful to use inhibitory interneurons to sprinkle in absence detectors, (Bratenberg talks about this in the biological notes on the Vehicles).

    One strategically useful place for this would be thalamic relay neurons again. Then everybody in the cortex can modulate relay neurons (via layer 6 circuits) and activate the absence of something else, too.

    If there exists a thalamic nucleus, which receives projections from M-box, then absence detectors in that hypothetical relay nucleus represent the absence of an explanation structure that fits everything.

    Whatever is activated then from this absence (0S), is allowed to represent the notion I know that I don't know. That is the basic notion of a Socratic wire.

    This model predicts that it is the 'intermediate meaning level' parts of the network that mediate "critical thinking".

    Now you might wonder if you would find 'confusion' neurons, perhaps everywhere, but perhaps especially in the parietal lobe, where the information processing flows seem to represent intermediate levels of analysis.

    Perhaps there is something about how mechanistic explanations, click the most? Perhaps it is real explanations, that stretch across all meaning levels.

    Perhaps there are m-boxes between everything. Perhaps Socratic cell assemblies are a generic signal to the system, that it is not 'done thinking'.

    What would be the m-boxes of social cognition? Something like person-mental-state analyzer pieces of the network perhaps? Funny, that TPJ is active during social cognition tasks, making the parietal look again like an m-box. Whether that means something I don't know.

    This idea in general says, you need middle-layer networks to understand, that you do not understand.

    This model maps onto things like hemispatial neglect; That would mean that the high-meaning level memes naturally simply confabulate. If the m-box is gone, the signal of not knowing is gone, too. Then the system would simply not have the interpretation I don't know anymore. This would mean that the parietal regions affected in hemispatial neglect patients are m-boxes.

    If this would be true, then…

    The prediction from this model is that you would find 'confusion' neurons in parietal regions. If you had a way to pattern activate those, presumably, you would bias the animal towards criticial thinking.

    Perhaps you could disrupt the m-box in strategic moments, and by doing so you would either create the feeling of a 'fitting explanation', or disrupt the feeling of a 'fitting explanation'.

The Sensor Critics

How to get a rational [Deutsch] memetic environment? You need some critics [Minsky], that can say whether something is a good idea, or not.

One answer I have for this is that the activity flowing from the sensors can be the critics.

If your explanation predicts x, but there is no x in the sensors, this is where the system has the chance to revise the explanation.

Rational Memetic Drivers (they must exist)

These critics then should create a new kind of memetic environment, where actual, good explanations are better than bad explanations.

If everything is about predicting the sensor inputs [see literature on unsupervised learning] if predicting the sensors is a game that memes need to play; That is useful because it is hard to fake [Deutsch]. Explanation structures then have to represent something about the world in order to predict the sensors.

Perhaps the answer is already here: Make the activity come from the sensors and "Memes want to stay active". The memes will represent true things about the world, then. Because it is the memes that are connected well, in the context of the active sensor inputs.


Hofstadter analogies, these are again situations.

There is a memetic driver to use the existing explanation structure to explain new things.

Why? The same thing: Memes want to stay active. The ones that already exist will make clever reasons don't you see that is the same thing! I should be active and no further alternative should be searched!.

You might even go so far as to say this looks like analogy is the basic memetic epistemological move. And this all checks out because analogies are situations, which are allowed to be represented by cell assemblies again. And those are subject to the memetics of the cell assemblies in the system.

Symbiosis by reward narrative

Why Flowers Are Beautiful, Elegance, Explanation, Aesthetics

Reflections On The Notion of Objective Beauty, Speculations On Elegance And Its Relationship To Brain Software, The 'Quick To Grasp' Hypothesis For Elegance Detectors

What is a good explanation? This must be a fundamental question when it comes to understanding brain software; Its main purpose is to create explanations. David Deutsch calls it creativity, creating new explanations.

David Deutsch has a philosophy of objective beauty, which ties together with explanation structure making. Talk Why Are Flowers Beautiful?. That is also a chapter in The Beginning of Infinity.

His theory is that flowers make hard-to-fake beautiful structures, by expressing objective beauty. They need to do so because they co-evolved with insects, across species boundaries. I think the key point is that each flower has many pollinator species, flowers must find some kind of smallest common nominator of beauty.

Objective beauty must exist in the same way that objective truth exists. Real art is more like science then, you can be more or less right. But you can never be perfect, objective truth is open-ended. Just as in science, we will always be able to find a deeper explanation.

This is a far-reaching view. It says that there is an unimaginable, infinite future of art there to be discovered.

What exactly it is remains somewhat elusive, but there would be a unification of elegance, scientific parsimoniousness, and artistic beauty.

Flowers seem to speak to something in our minds, the way that other things in nature do not. And it is the flowers that co-evolved with insect pollinators I think. Grasses for instance have flower structures, but:

Grasses, a group of plants in the family Poaceae, generally do not rely on animal pollinators. Instead, they are predominantly anemophilous, which means that they are adapted to wind pollination. The flowers of grasses typically have feathery stigmas to catch pollen grains carried by the wind, and their pollen is usually light and abundant to facilitate dispersal through the air.

And the grasses don't have this quality I think.

Once you think about it, is almost as if flowers glow with some kind of pleasantness, you would say it is magical.

In software engineering we know, there is something about creating a system of general kinds of explanations, and then explaining our problem in terms of that language, which is for some reason an immensely powerful thing to do. Software like this is beautiful and elegant, or a nice hack.

There must be something that says an explanation is elegant. Intuitively, some kind of 'minimalism' is part of the answer. Perhaps Kolmogorov complexity or recently "Assembly Theory"51 are attempts to make a theory of this. Perhaps some subfield of constructor theory might be talking about 'the biggest amount of explanation, given a small amount of time'.

Now trying to tie this into my cell assembly memetics:

Consider (assume bilateral) symmetry:

In order to explain, in order to 'expect' the right things when interpreting a symmetrical object, if the explanation machine has 'symmetry' as an operation (and that operation is cheap), then after explaining half of the structure, the explanation is done. It need only add the additional info that the object is symmetrical.

In terms similar to Kolmogorov's complexity of the explanation program I need to write, if the object is symmetrical, in principle the length of this program is cut in half.

If the brain is making a situation analysis until the system is stuck in an attractor state of best guesses of explanation upon which no further improvement is found mechanism, it would mean that whatever memes are explaining a situation in the shortest amount of time, are the ones we see around.

Good explanations, like good computer programs, will use abstract building blocks of explanation pieces, they will compose those building blocks in elegant ways. These building blocks are memes themselves, of course. With a good strategy because they are re-used across many situations. The composers of explanation, the methods of explanation making, are memes themselves in turn, too. It is the capacity to appreciate elegant ways of creating explanations, that is perhaps the ultimate step in making an explanation machine.

This would memetically grow a set of abstract explanation building blocks, what you might also call common sense.



  • Perhaps when a physical structure like flowers is described easily in terms of "common sense", it is especially pleasing to the mind.
  • You could, for instance, count the neuron timesteps until a solid interpretation is found.
  • it would mean elegant structures are interpreted faster, that humans would understand flowers faster than apes do.
  • It would be useful to build in elegance detector wires Darwinistically, which would help with growing an appreciation for good explanations.
  • This explains somewhat why there would be flowers that look like water drops. Alternatively to symmetry, they would exploit other pieces of common sense, in order to be explained easily.
  • Perhaps as an alternative, whatever flowers are doing is maybe fitting especially well with much of the network.
  • G. Palm calls it a good idea. That would be a thought-pump-associated situation where you have a high threshold and still a high amount of activation. (that means that the idea fits the network especially well).
  • Perhaps this way flowers can touch something deep in our minds that has to do with feeling, too?
  • (But perhaps, if we have elegance detectors there is already an implication with the feeling stuff going on).
  • flowers are beautiful because they fit the network well?
  • flowers are beautiful because they fit the language of explanation well?
  • Some art might then be the attempt at finding other kinds of structures with land well on the explanation structures of humans.
  • It is necessarily the case that an elegance detector in the human brain must be some kludge-riddled hack made by evolution, that doesn't measure true elegance, but some randomly thrown-together approximation of it.

Note that it is not my view that elegance is explained as Kolmogorov complexity.

My view is that an analog to Kolmogorov, the brain interpretation ease, i.e. how well something lands in the network, and/or how fast it settles into a satisfactory interpretation, are proxies for elegance. Additionally, perhaps evolution would have hit on making the wires that will create a reward situation for situations where this prox is on. That this reward signal would be part of what we call the beauty of flowers.

But the deeper important move is that such a mechanism makes us appreciate hard-to-fake elegant explanations, and the meta processes of making elegant explanations. Which is perhaps the move that makes a machine an ultimate explanation maker. (Everything else is simply a recursive improvement, not a fundamental additional move).

If this is true, you would expect that you could measure some activity flow (EEG? not sure how fine-grained). For persons looking at flowers, you would

  1. find a stable representation faster
  2. And/Or find that more of the network is active (I realize not the case, if the threshold device goes up. You would need to measure the threshold device, too)

There would be golden elegance wires, too (I call reward wires golden wires). These would be on, every time some "new stimulus" reaches a stable interpretation fast.

I could speculate now on how the striatum could dishinhibt a thalamic relay and have stability-detector wires listening to the cortex at the same time. How you could tune a time window would be a 'surprise of quickly reached stable interpretation' but I will not indulge. Because I would be wrong about most of it.

But this is the kind of scheme one could build into a vehicle.

How to build a stability-detector?



+--------+              +-----+            +---+
|        |              |     |            |   |
|  --O---+--------------+ off | ...  off   | O |
|        |              |     |            |   |
|  ---X--+--------------+ on  | ...  on    | X |
|        |              |     |            |   |
|--X----X+--------------+ on  | ...  on    | X |
|        |              |     |            |   |
+--------+              +-----+            |   |
    neurons            ... n-wires         +---+

                          t0,       t1,     t2

You make random wires through the neurons s-listener-wires. At each time step, you can look at which s-listener-wires are on. The set of hot wires is a proxy of what ensembles are active in the neurons.

If the ensembles move around, your wires will certainly change.

Make stability-detector a population of neurons, make it postsynaptic of your s-listener-wires. Use biochemistry or equivalent to say that the detector neurons need some amount of inputs in a wandow of time steps.

Then, your stability detector is a population of neurons that represents the current amount of stable activity (of ensembles) in the neurons.

(In absolute terms).

Note that the stability detector scheme can also be used to simply observe a little what ensembles are active in the neurons.

Visit this playground: assembly-friends#4.

I have added a stability detector, the Navajo-white neurons between the cyan neuronal area and the white sensor box.

n-wires is 50, threshold is 2, and a 'tick-window' is 4. So if the detector sees its neuron on 2 out of 4 timesteps, it's on. And there is a beep, too. This setup is not guaranteed to make beeps.

The beeps are wonderful because they are a second way of perceiving the ensembles. It's like the composition of the activity on top of the neurons.


Some hyperparameters and properties of the system would change how fast the system stops thinking. For instance, if everything has high attenuation (see above) the system will always need to find circular patterns, or never stop thinking. If everything has high Hebbian plasticity, it would fall into an interpretation quickly and get stuck there.

Knowing about these properties, and knowing how to make the system balance itself in those regards, is one of the finer points of psychological cybernetics. And perhaps sheds light on some human psychological disorders.

Deutsch goes one more specific and says that human explanation structures have the same far-apartness as species boundaries, too. In his view, only humans (at the moment) have enough explanatory creativity to appreciate flowers, then.

Something confuses me though, why do flowers appeal to all kinds of insects, then? Because insects have evolved to understand the kind of hard to fake beauty of flowers? Not because insects have so much common sense, but because they had an evolutionary driver to be attracted to the objective beauty of flowers?

(Getting Philosophical For a Moment)

I think there was/is a tension in neuroscience and computational neuroscience - that as a biologist you see a thinking machine, but what is the structure and function of this? The answer was it does computation. So now you say this thing does this computation and so forth. But this is like looking at a car engine and saying this does acceleration. I would say that you can replace computation with wizardry in a neuroscience textbook and you get the same information content from it.

What is missing is having a shape of the function in mind, because it is this shape, the rationale for what the structure should be, that allows us to satisfyingly explain biological systems. If the shape of the function is not perfused with an explanation model of what this should be, then you will always only see single pistons turning and gasoline flowing and so forth, you will see the trees but not the forest.

Now the cell assemblies are sort of a little bit 1 perspective on the intermediate level, the level of the language and primitives of the cognition software.

And this computationally intermediate plane is understood in terms of something living. The cell assemblies have a biology to them.

My philosophical view and agenda is to make the ideas come alive in our brains so to speak. Now these ideas have quite different evolutionary environments, landscapes, and pressures, they run on a computer, and they create software, that competes with other software creating agenda-having entities. They live on subsecond timescales in vast meaning spaces. Building an intuition around the properties of this world and its inhabitants is an aspect of explaining cognition in my view.

The mechanics and evolution of these little entities, their structure and function, are allowed to dissolve in our minds, the way that the structure and function of biological systems do.

But they are computational constructs, with the malleability to represent computation, knowledge creation, psychologies, societies of a kind and so forth. One perspective on them is how they construct building blocks, very chemical. Another perspective on them is how they make harmonious societies, very system-biology, ecology thinking and so forth. Another perspective is how they create knowledge, very epistemological, practical philosophical and so forth.

So from biological cybernetics we get this substrate, that is allowed to be put together so beautifully complicated.

This then is one perspective and candidate intermediate land and language between us and the neurons. This would consist of a computational framework (assembly calculus, thought pumps, hyperdimensional computing and so forth, something that says what kind of neuronal areas and nuclei you have, something that says what kind of attention mechanisms are available). The behavior of this system is described in terms of memetics. That is sort of alive, sort epistemological, sort of ecological, sort of software, sort of building-block making. It is psychological only in the sense of making sense of large-scale processes inside the system. The entities themselves don't have psychologies, they only have tiny agendas.


  • How big are cell assemblies in this view? Tiny, or is there 1 at each time step in a brain?

    I suppose the answer might be it depends on how you want to parse it, and both ideas are good sometimes.

  • How do cell assemblies relate to interpersonal (cultural) memes? (The ones from Dawkins and Blackmore).

    Cell assemblies are a brain computational construct, interpersonal memes are abstract replicators representing some knowledge of how to spread between brains.

    Both of these things are pieces of information that replicate and are subject to natural selection. Both of these can therefore represent useful information. This is interesting in light of the epistemology of the memes and memetic landscapes. (See David Deutsch's rational memes.) I think an analog of rational memes is the ad-hoc epistemology of the animal brain.

    It is intuitively obvious, how a piece of knowledge can spread interpersonally. Dawkins's tunes and habits of walking.

    It is required for such a meme to spread and be 'expressed' by the computer it spreads into. Any creative cognition machine is susceptible to such memes. (It must support it in the first place to count as a cognition machine in my book). This works, regardless of whether that cognition machine is implemented in terms of cell assemblies or something else.

    (But the landscape of available brains is the landscape of the social memes, so incorporating truths about certain animal implementation details of the brain you spread into makes sense. For instance, a meme might work by making you angry, but this same meme might not work on a Spok brain. In a world of Spoks, such a meme would be a bad meme. In the world of angry apes, it is.).

    The cell assemblies out of which a 'brain-phase' substrate of an abstract meme is made are different between persons (just to be clear common sense and neurophysiology apply).

    A social meme must navigate the messy biology and cognition of many brains in order to work.

Implementing Perspective, p-lines

You can make a few random wires through the neurons.

| X            X           |  neurons
| |     X      |           |
  |     |      |
--+-----+------+-------------------------------     perspective-lines
------------------------------------------------    [ p-nucleus ]

Single neuron activation can pattern complete to an ensemble [Carillo-Reid, Yuste 2020].

We can see intuitively how a hypothetical perspective line would modify the attractor landscape of the neurons dynamically then. This would be an immensely useful operation to have, from the cognitive machine itself, too. (Hinting at something that sounds like the actuation branch of an attention schema [Graziano 2015]).

I like to call this mechanism and its operation a perspective mechanism.

You can make a perspective organization by selecting random lines and activating them. How something random is useful to the system is the topic of the memetic engine and the living software of the rest of this page.

So you make some buttons (via the usual affordance mechanisms) to modify the landscape in random ways.

A perspective organization (or orchestrator), enables internal behaviors. That is simply saying, modifying the current perspective landscape to make these other things more likely. (This would be labeled attention in cognitive neuroscience, but my hunch is that at least 3 other things are labeled attention, too). With such a mechanism at hand, the system would be able to move from perspective to perspective, where each is a high dimensional point in meaning space, giving context to what kinds of content are likely to occur in turn.

This to me looks like the thought-jump, mental zooms, and perhaps the query lookups to memory that are available to us as cognitive users. Also Random Perspective?. A perspective mechanism is a generally useful thing A toy idea for a meme-pruning algorithm / A Candidate Dream Mechanism.

When one looking for a word, is it not the case that one has a vague idea of a 'place' in idea space that would help finding the word? When I am at a loss for a word, I might go through several of such places, bringing to mind all kinds of contexts, for instance, memories of where I believe I might 'find' the word. For instance, I remember that a word came up in a book I read, and I have the operation available the 'move my mind into the place', or 'bring to mind the situation' of some memory of how I imagined the dialogue in the book. This in turn modifies the mind to make all kinds of follow-up, 'highlights of memories' of when I read the book or something.

In other words, it is a bit of a temporary ecosystem, that makes different kinds of ideas be able to live inside the world.

Striatum Looks Like a Perspective Orchestrator

Speculations On Striatum, Behaviour Streams (perhaps should be called affordances streams)

Note that machine learning attention is a perspective mechanism (they gave it a slightly worse name in my opinion).

It is remarkable that via neuronal ensemble memetics, we see that something (initially) random would be so powerful.

Some Memetics On Eye Movement And Maybe How That Says Where

With the layer 5 movement output from A Curious Arrangement in mind, how does the brain achieve:

  • Having a model of the relative places of objects in the scene.
  • Have a stable representation of the scene. It stays when I move my eyes. (Even though the pixels on the retina all change).

I move my eyes and I see an oval shape of color and shapes that I label the visual field. I can move to the edges and there the shape of this field changes somewhat, scued to one side. But in general, this field just stays - it strikingly stays the same.

Presumably, Cortex can offset the change in retina sensor input, taking into account the eye movement data, presumably coming from the efference copy from A Curious Arrangement The layer 5 circuit tells us that this efference is then represented in higher-order thalamic nuclei, (pulvinar..).

It seems parsimonious to assume that the higher-order visual areas represent the visual scene and that all those representations stay the same when moving one's eyes. When I look ahead I see an object. I can move my eyes and I feel the object stays in the same place. There should be a stable cell assembly that represents the location of this object somehow if we can be so naive. It is stable across eye movements.

The stuff that changes is then pushed down, to the primary visual areas and retina, everything else is allowed to (and usefully so) to be stable.

I was thinking of some wizards sitting together in a city, they are looking at the clouds (retina data) and are trying to predict how the clouds move. They have a magic sun deal (eye movement information), which says north-east, east, and so forth (with an encoding of degrees and distance or so forth).

Something meme-centered struck me; What if every object meme simply outputs the behavior 'move your eyes to me'? (At layer 5, presumably at the cortex that is usually labeled as the object where cortex).

                    visual field
        |                                    |
        |   +---+                  +---+     |
        |   | O1| <-------X------> | O2|     |
        |   +---+   x->o1   x->o2  +---+     |

X -  current gaze
x->o1 - eye movement encoding, how to move gaze to O1
x->o2 - same for O2

Then, all where information is allowed to be encoded in the eye movement that would make this object be in the center of the gaze. From memetic reasoning, an object-representing meme should benefit greatly from being looked at. (I am mixing the internal world and the external world freely, see Macrocosm and microcosm in the first part of this page why we can make this move when talking about cell assemblies). I.e. it would make sense if every object meme would play a game of the gaze trajectories.

I think there is something about looking at things like the stars in the sky. But this can be any arrangement of objects. I did the following this morning when sitting on the terrace, where we have some happy plants coming between the stone tiles:

My visual scene:

          |                   |
          |                   |
          |     V1            |
          |                   |
          |              V2   |
          |                   |
          |                   |
          |      V3           |
          |                   |

V - plants growing between cracks

Suddenly I experience straight lines between the points of interest in the visual scene:

          |                   |
          |                   |
          |     V1---X---V2   |
          |      |       /    |
          |      |     -/     |
          |      |   -/       |
          |      | -/         |
          |      V3           |
          |                   |

V - plants coming between cracks
X - the center of gaze
| - the feeling of the lines between the points

If I move my eyes between the plants, this effect is intensified, I think. And then I think, do I not feel like I know the distances between those things, as the lines between the points?

There are very ghostly faint (purely virtual so to say) lines between the objects.

Perhaps those lines are the potential eye movements, expressed at layer 5 of objects and 'object group' representing cell assemblies. That says 'Move your eyes like this, please'. Because they want to be looked at. This is useful information to the rest of the system because the rest of the system can say from this exact eye movement information what the distances are.

Isn't this part of why people of old have looked at the stars and saw the patterns, the constellations?

If the game of the circuits is for some reason laid out in a way that forces many eye movements, it would be useful for the memes to output potential movement trajectories with multiple stops. I.e. if we for instance build in a restlessness in the circuits, the memetic landscape changes. Because every meme would like to say look to me and then never away. But if we force moving around, then the memes play the game of move to me, then move to this associated thing, which will lead you back to me.

If the brain uses the output of these trajectories to pattern complete some prediction states from this, i.e. move 5 degrees to the left and you see the banana, move 10 degrees to the top and you see the apple. And so forth, then the relationship, the where is encoded in this composition of possible movement and prediction.

Something else that falls out of the model:

If object A says, move to me!, it would also like to output /move to A1, move to A2, … / where A1, A2, … are sub-points of interest inside A.

I can't come up with this idea without having the layer 5 circuits in mind, quite humbling. Because who knows, maybe there are 5 other things like this?

How does that predict the retina sensors when moving the eyes?

This is very speculative… perhaps this is just a starting point and reality is crazier and simpler than this.. :

Perhaps it goes something like this:

     higher-order visual cortex
     |         |
     |     A   | stable representation of A
     |     +   |
           | layer 5
           |                               primary visual cortex
           |                             +----------+
 +---------+                             |          |
 |         |                             |   +--+---+
 |         |                             |   | X| A |   <--------- [ retina data ]
 |         |                             |   +--+---+             3.
 |  +------+---+  1.                     +---^------+ + --+ -+       The overlap of expectation and sensor data ignites
 |  | ----     |                             |        | A |  |       the interpretation "A in the center of visual field".
 |  | -------  +----- ----------- ---------- +        +- -+  +
 |  | ----     | Expectation: A will be in the center of the gaze
 |  | x->a     |                               ^
 |  +----------+                               | ^
 |    pulvinar                                 | |
 |    (all kinds of possible eye movements)    | |
 |                                             |
 |    _                                        |
 |    | says go (by inhibiting the inhibitor)  |
 |    |                                        |
 |  +-+--------+                               |
 |  |  ?       +-------------------------------+
 |  |          | 2.
 |  |          | Striatum says 'go' to the movement and to the expectation at the same time
 |  +---------++
 |   striatum |?
 |            |
 |            |
 +------------v-----------> [ eye motors ]

                            virtual visual field
                |                                     |
                |               X------->A            |
                |                  x->a               |

X - the center of gaze
A - Object
x->a - eye movement that puts A into the center of the visual field

This would mean that there should be projections from pulvinar to v1 because the eye movement data from higher order cortex needs to go through a higher-order relay nucleus (from Sherman's thalamocortical circuits).

Something is weird in this model. If layer 5 goes to motor areas and higher thalamic nuclei, why does the go need the striatum? As far as I know, the striatum says to thalamic relay nuclei how much they are inhibited, not to motor targets whether they should go. Does that mean there is another element that is missing from this? It would be between the pulvinar and motor targets, it would be on when striatum says go.

A principle of the efference copy is at play here: It is exactly the power of splitting the message in 2, that there are different interpretations of the message possible. Movement by the motors, and the expectation of what to see at the primary visual in this case.

You could make the following experiment:

  1. Make a monkey look at a visual field with nothing but a red blob in it.
  2. Read out all wires that go from pulvinar to v1.
  3. When the monkey has a blob in the periphery, you see wires being hot.
  4. When the monkey moves its eyes to the blob, the wires are really hot for a moment (bursting?)
  5. If you could, you would read out from those wires red blob data. You would see that these wires change the attractor states in v1 to see a red blob in the center of vision.

The fascinating thing here is that eye movement data is red blob data for somebody else.

  1. ? Perhaps now the same wires say make little eye movements that stay on the blob. And More red blob and its edges or something.
  • The cell assemblies of A want A to be in the center of vision.
  • The cell assemblies in v1 presumably are happy to represent sensor data. There must be something in the logic that makes the sensor data be relatively even memetic players I feel like. Otherwise, how can we have this smooth transition when moving the eyes? I.e. the center of vision neuronal area is happy to swap from white cell assemblies to red cell assemblies.
  • Perhaps this sheds light on hemispatial neglect, if the higher-order area I talk of is gone, nobody represents the objects to that side anymore.
  • If you would experimentally knock out the layer 5 of where cortex, you would

    1. Prevent some forms of eye movements
    2. Disrupt the animal from understanding where, distance, spatial relationships

    (I realize this doesn't say much when we say disrupt where cortex).

A fundamental tool of replicator theory is asking Cui bono?.


  • What is binding the object representation with the where information?

    The answer might stare us in the face here: It is perhaps partly encoded in the combination of eye movement and expectation. If I move my eyes like this, then I see a red apple This represents the red apple and where it is located, the information is bound via the nature of the cell assembly prediction representation. (The continuation of this idea is below, Getting A Visual Field From Selfish Memes)

  • What is the nature of the inner eye?

    Can I not zoom and move around in imagination states?

  • Why are eye movements spared from sleep paralysis?

    Assuming the primary role of sleep paralysis is to prevent moving because it is dangerous. I think the answer most at hand is that eye movements don't matter. I.e. when you sleep in the trees, then moving your arms is deadly, but moving your eyes is not.

    Now one might wonder if eye movements are perhaps fundamental to cognition, attention, mental moves, and guided imagination. Then one would think it is obvious you should be able to move your eyes during dreaming.

    I don't think it needs to be this way. Why not implement a gate nucleus right between the eye motor muscles and the output nuclei? (Because eye movements are not dangerous I think).

  • Whatever the hearing attention moving effectors, do they have all the same logic?

    I.e. those hearing attention movers, do they also originate from virtually the whole cortex? Are they spared during dreaming? Are blind persons using a hearing attention movement encoding to locate where information?

  • Why do cell assemblies not cheat by representing the same object multiple times in the visual field?
  • Is there any visual illusion where
    1. You see more of the same kind of object than is there?
    2. Do You feel that the same the object is in multiple places?
    3. You feel that you move your eyes but you see a different object than expected.
    4. You move your eyes and suddenly the distances between objects are different than expected.

I don't know of any of such illusions. So I think whatever the circuitry is, it doesn't make it worth it for the memes to trick the system in this way. Perhaps except 1, then I feel like interpretation is many of x. Like when you look at ants and you feel I see many ants.

Zooming and the 'local' relationships of parts of an object.

This made me think of a new kind of user interface arrangement. Instead of submenus that you flip through with page buttons, you have a zoomed-out view. Then you zoom into one part of the interface, and you get a higher detailed view.

The submenus align with the notion of places we visit.

The Zoomer menu aligns with the notion of the global scene and the local scene or something.

Getting A Visual Field From Selfish Memes

Pretend for a moment we are looking at a scene with open eyes, without head movement (the same logic will apply to head direction and so forth). For present purposes, imagine there is a where cortical area that encodes an eye position, in absolute terms. (It says eye coordinates).

Let's say there is a primary area to represent retina data, and some what areas representing color and object identity and so forth.

Observe the simple case:

  • The user looks at a certain position in the visual scene, which means there is a cell assembly A active in the where cortical area. (The motor encoding for the position is active)
  • [?] Auto associate the cell assemblies in primary (encoding sensor data) with A. This would for instance work with cortico-cortical connections and plasticity.
  • Further, what represents an interpretation of what is being looked at in primary.
  • All these 3 things what, where and sensor data are active together, and have a chance to associate in cell assembly stretching all 3, you might call it A-wws, for A where, what, sensors. I just call it A, even though it is not the same A from step 1.
                         |  A                  |
                         |                     |
                         | ^ A                 | layer 5
                         +-+-------------------+ eye position
 what                      |
     +------+              |
     |      |              |
     |      |              |
     |  A   |              |
     | A <--+------------->v^ (? thalamocortical wires? cortico-cortical wires?)
     +------+               |
     |color | ...           |
     |      |               |
     +------+               |
      ..                  +-+---+
                          | v   |  primary
                          |     |

primary - primary visual v1
A - stable cell assembly, stretching multiple cortical areas

Should the user decide to look away from the object, the cell assembly A will simply stay, in where and what. In a meme world, A (and its meme friends in the rest of the network) will want it to stay active. And A will spill into primary again, given the chance. Perhaps if the user closes their eyes and pays attention to A, presumably A would have the chance to spread into primary a little again.

A says 'You can expect to see this if you move your eyes here'. Note that the where part of A, namely the activity of the where cortex, is a sub-part of the data structure of A. And parts of information ignite the whole; So should the user pay attention to 'this position', then the rest of the ensemble will re-ignite in what, and presumably a little in primary. Should the user move their eyes to this position, A-wws is re-ignited in full all its glory.

This logic will create a competitive environment of 'object memes'; All saying 'Look at me!'.

                          visual scene (virtual)
        |                           |
        | +----+                    |
        | |    |                    |
        | | B  |                    |
        | +----+                    |
        |                 +------+  |
        |                 |      |  |
        |                 |  A   |  |
        |                 +------+  |

A, B - object memes

Note that an object meme that does not say 'how' to look at it, is bad.

Memes want to be good interface entities.

That is, since there are user-level entities in the system (they are a necessary part of explaining brain software), every meme wants to be easy to use by the user. That is a little bit like A and B want to be great buttons, advertising what they can do to your mind.

Click me and your mind will change in this way….

How to click an object in the visual scene? It is hard to describe, but it is very easy to do. It comes across like deciding to move one's eyes, or deciding to look at an object, or perhaps giving in to the temptation to look?

So the object memes have memetic drivers to be looked at easily, which firmly encodes where, via an eye movement motor encoding. The memes say what and what to expect via their associations. That is simply what is part of the cell assembly. Remember that the cell assemblies pattern is complete; if the cell assembly looking at position x is pattern completing the redness of the apple and a little bit the apple. Then this is an expectation structure, and I think this is all we need in order to represent the notion that there is an apple at position x.

Binding is an 'expectation coalition' of temporarily associated, compositional data structures.

Further, memes will go out of their way to mean many things, to be implicated in more aspects or situations. If a meme can advertise a hunch of some potential interesting sequence of mental state, if only you would pay attention to it, it will do so.

Note how the dynamic part of the system is pushed to edges as far as possible via memetics:


|             |                  |
|   assoc.    |    where         |
|     ?       |                 -+------  stable
|             |                  |
|                         |      |
|  what         |         |  V1 -+-------- dynamic

Note that the memes in what and where can stay on even though the retina data changes. It is only V1, representing retina data, for which staying on across eye movements would constitute major confabulation. Presumably, such an immense top-down influence on sensor-level representations is the mechanism of hallucination.

So V1 should be very impressible somehow by the sensor data coming from LGN and the motor movement messages, presumably coming from pulvinar.

As a healthy person, the edges of such a mechanism happen for instance when walking amidst trees. Some 'person' representing memes may be activated in the higher visual areas, giving the vague impression of a person in the corner of the eye. This problem is remedied with the help of some 'startle' or 'surprise' situation, which makes one pay special attention to the spot in question. Presumably resetting the system, to pay attention to the sensor data coming in. (Counterintuitively, resetting the whole thing might slow down how fast you recognize the visual scene in such a situation). These trigger-happy 'person' memes are thereby kept in check because the new sensor-level representation can now facilitate the real what interpretation for the position: Just a tree.

Presumably, this process is disrupted in a hallucination situation. It might be that hallucinating persons are somehow less surprised by out of place interpretations, but this is speculation.

The higher visual areas want to stay on, and that is useful in the system, too. That allows us to represent a stable visual scene.

In principle, there is a memetic driver to push dynamic parts to the furthest edge possible.

Note that cell assembly memetics gives a fundamental, high-level neurophilosophical conceptualization of top-down and bottom-up information processing. The memes (cell assemblies) want to be stable and active, across as many meaning levels as possible. Top-down and bottom-up are thereby explained in a unified model of 'stableness across levels'.

Why is the visual scene colored in, even though there is only a small spot of color vision on the retina?

Via the set of counterfactual expectation structures, if you look here, then you will see this color.

Perhaps the inverted pizza can help.

Case 1: Looking at an object A:

         \             /
          \     A     /                   meaning-level
           \         /
            \       /                    (higher visual areas)
             \     /
              \   /
                X                        sensor-level  (LGN,  V1)
                                                      ('retina data')

A - Stable cell assembly

A nice happy cell assembly, stretching all available cortex, making an expectation structure that says what to expect from the retina data.

Case 2: not looking at A, looking somewhere else.

         \             /
          \     A     /                   meaning-level
           \         /
            \--------                    (higher visual areas)

                                         sensor-level  (LGN,  V1)
                                                      ('retina data')

A - Stable cell assembly, cut off from sensor data

A is not supported by sensor data anymore right now, but it's a meme. And it is the chance to stay stable in the system, by getting enough support at the meaning levels.

This says there is an object A at this position. This includes its color.

Then, in reality:

         \             /\     \             /
          \     A     /  \ B   \     C     /   ...          meaning-level
           \         /    \     \         /
            \-------/      ------\-------/                  (higher visual areas)
                                  \     /
                                   \   /

                                                        sensor-level  (LGN,  V1)
                                                        ('retina data')

A, B, ... - Set of stable cell assemblies, cut off from sensor data
C - Stable cell assembly, stretching meaning and sensor level

There are many object memes (cell assemblies) representing what to expect when looking at positions. At any given time, there would be roughly 1 cell assembly, the one whose position is being looked at, that stretches into the sensor level.

Meaning-level ensembles have the chance to stay active, because of the arrangement of cortical connections, where there is a large amount of cortex receiving input from itself. The sensor-level representations form some kind of peak-hole of meaning that moves around, supporting the large meaning-level representations. Keeping track of this moving around is fundamental for representing where.

Why is the blind spot colored in?

Because the object memes were active in the meaning levels all along. Memes are saying 'Look here and you will experience x'. That includes the position that is currently covered by the blind spot.

Seems like the lack of peripheral sensory data support is a problem for the object meme. Such that small objects will vanish from the scene. Not because the small object meme would not fight for survival I think.



It depends on how you look at it

The beautiful thing in this arrangement is that the memes active in the visual areas are simply playing the basic 'look at me game'.

On some user level in the system, this creates a beautiful, magical-to-use, interface. Where every interface entity (like a hyperdimensional button, an affordance) is advertising what would happen, if you looked at it.

From looking at this data from a new angle, the one where you can decide between many things to look at, the higher-order meaning-level software entity we call visual field is constructed.

This idea meshes with Micheal Levin's and Joshua Bongard's notion of overloaded computation: There’s Plenty of Room Right Here: Biological Systems as Evolved, Overloaded, Multi-Scale Machines. As a principle of biological computation.

The idea is that a computation can mean different things, depending on the perspective. A general principle for evolved intelligence might be that a computation evolves to solve 1 task. Subsequently, a new observer evolves to interpret the computation from a different perspective.

This notion is mirrored in the anatomy of the Neocortex and Thalamus52. First, evolve a cortical area, which makes situation analysis and outputs motor commands at layer 5. Then a secondary cortical area, looks at the motor output of the first, allowing a new interpretation.

The Principle Of Usability

Explaining brain-software entails explaining magical user interfaces. In the context of user entities, all memes want to be usable interface entities. Like buttons that explain themselves self how you can use them. So they represent James Gibson's Affordances, too.

Perhaps then all cell assemblies will represent what to do in order to make me more important at their layer 5 motor outputs.

For the visual object memes that is look at me. For some more cognitively derived memes perhaps that is pay attention to me?

Memes can then be a constellation of promises pay attention to me and your mind will be x.

There is a hunch that the triangle of where, what and sensor data is very important. That the interpretation this object has this color is only able to be constructed by having the sensor level in the loop.

Speculations/ Questions:

  • Can you quantify the amount of where slots or something?
  • Presumably object memes need to compete for position encoding in where cortex?
  • How do these mechanisms make sure we never perceive multiple objects in the same spot?
  • This yields a cute story of merged (associated), symbiotic sub-object memes. Which forms a temporary object coalition. Giving up their identity in order to be represented.
  • Conversely, we also don't see the same object multiple times.
  • Perhaps higher-order situation analysis cell assemblies provide a context, which encodes the notion objects don't jump around, objects only exist once, objects don't overlap in space. Perhaps if the brain was wired to a different physical world it would be happy to represent alternatives to these notions? I can only speculate at the moment.


  • Presumably, there is some dynamic 'zooming' where one considers an object as whole, then sees the subparts of the object and so forth.

    What are the mechanisms to support such zooming and so forth…?

  • Perhaps we can imagine this inverted pizza slice then going up and up some meaning levels; Some say/this is my situation in life/, this is my personality on timescales of years. Then the 'sensor-level' from that perspective so to say are the daily and moment-to-moment ongoings, call them behavior-level situations.
     +---------------------+                    |
      \      ?            /                     |
       \                 /    personality level | months, years
        \---------------/                       |
         \             /                        | weeks, months
          \     A     /                         |
           \         /         behavior level   | days, hours, minutes
            \       /                           |
             \     /          situation level   | seconds-minutes
              \   /                             |
             --\-/--------                      |
                X             sensor level      | seconds

A - a stable cell assembly, stretching meaning levels

I like the term 'situation analysis' exactly for the reason that 'situation' doesn't say how long or large of a situation.


Also called 'perisaccadic visual remapping', or 'forward receptive fields'.

'Remapping' is upside down from my perspective. The retina data is a dynamic thing.

The eye position/eye movement is not a resource to the visual brain in order to say what bottom-up sensory things to what higher level object things. The eye position/eye movement in my view is part of the interpretation structure in the first place. It is simply 1 data structure (1 cell assembly) per position and object, that represents what to expect when looking at this spot.

What is the difference between this and 'predictive brain'?

I consider 'predictive brain' [Andy Clark, Anil Seth, …] a neurophilosophy. It makes sense how perception is constructed, via a series of improving guesses. It is a framework that helps thinking of top-down and bottom-up processes.

This is somewhat aligned with the view I have here. After all, the basic operation 'pattern complete' is a prediction mechanism.

I see the biggest difference is in how the memes represent knowledge. They are approximations to an explanation structure, which is a counterfactual.

This emphasis on 'invisible', counterfactual entities deserves its own section on its own. It is that the memes represent models of the world. The are not merely some information processing means for prediction. They are the whole point of this software.

Speculations On Striatum, Behaviour Streams

This is bound to leave out 80% of what the striatum is doing.

Update: Thalamus Matrix System Looks Like a P-Line Implementation

Note this was with the idea in mind that striatal (Gp) outputs go to the thalamic core system, but it turns out they go to the thalamic matrix.

  • 'Behaviour orchestration' is updated to 'perspective orchestration'.
  • 'behavior streams' is updated to 'affordances streams', to make clear that perhaps most of this is representing 'acts of attention'.
  • Except for the too-narrow conceptualization of 'behavior' I think these notes still make sense.

Here are the old notes:

Orchestrating Behaviour?

Speculations On The Puzzle Of Tourette's, Musings On The Hypothesis Of Behaviour Encoded Situation Analysis


  • Input to the striatum is from 2 types of neurons in layer 5 of all cortex. [insert sources]
  • Striatum is implicated with 'generating' 'movement patterns'

    Other expressions of emotional states, such as smiling, also originate in the striatum and pallidum, and we are very sensitive to the emotional states these movements represent. ‘Faking’ a smile with conscious movements uses cortical motor control instead, and produces a result that looks false to others in most cases.


  • Striatum makes modulatory, inhibitory inputs to thalamic relay nuclei [M. Sherman]
  • Consequently, this doesn't look like the pathway carrying the information for how to move. It is much more parsimonious to consider the role of striatum as some kind of selector, filter, or orchestrator.

    (This is counter to the current mainstream thinking about striatum).

  • Striatum by default inhibits thalamic relay nuclei (via globus pallidus int.)
  • Parkinson's is a degeneration of Substantia Nigra Pars Compacta, which makes dopaminergic innervations to the dorsal striatum etc. Pars compacta
  • Sleeping Sickness,

    The disease attacks the brain, leaving some victims in a statue-like condition, speechless and motionless

  • Oliver Sacks in Awakenings (1973) talks about the sleeping sickness patients that were rejuvenated by L-DOPA. (Small documantary about this).
  • But this had the effect that the patients had compulsive behaviors.


  • Let's assume that layer 5 is the 'behavior output layer' for all Cortex.
  • Further, layer 5 behavior streams are the latent space encoding of higher-order thalamic relay nuclei (see A Curious Arrangement on this page)
  • There would be 'behavior streams' that simply participate in the information processing of analyzing the situation.
  • This would only be useful if the layer 5 neurons that go to thalamic relay nuclei are correlated with the ones going to the striatum. [which I don't know whether that is the case or not. Considering that column activity is correlated, the question is whether those neurons are co-located in columns, and that sounds likely to me].
  • Striatum is an element that listens to all "latent" behavior streams of Cortex.
  • What exactly is relayed at those thalamic nuclei? The answer is part of the puzzle of the striatum.

But here is sort of one idea that makes some general sense:

Say that the striatum is a selector, it looks at all latent behavior streams being expressed at the cortex and takes a value judgment from some Darwinian wires [Braitenberg 1984, also called Darwinian Brain]. And removes the brake from the streams that get the most approval. Perhaps it is selecting only a tight subset or a single one. This would solve Minskies 'pulled by 2 spirits' challenge.

Consider an animal that is roughly equally thirsty and hungry, it stands just between a water hole and a pile of hay. By the time it walks over to the hay, the thirst dominates and it goes over to the water. But by the time it is at the water, hunger dominates. You probably want to select one behavior and stick with that for a while.

     what striatum sees:

                  /--                     /-|
                /-                     /--  |
             /--                     /-     |
          /--                     /--       |
        /-                      /-          |
      +----------------------+--            |
      |                      |             /+
      |  +---+       +-----+ |           /-
      |  |   |       |     | |         /-
      |  +---+       +-----+ |        /
      |        +---+         |      /-   time flow
      |        |   |         |    /-
      |        +--++         |  /-
      |           |          |/-
                  behavior streams


                            cortex II
                       |              |
                       |              |
        cortex I       +----------+---+
     +---------------+ |          |XX |   layer 5
     |               | +----------+-+-+
     |               |            | |
     +----------+----+            v |      /----+-
     |          | XX |              +->  /-   /-|
     +----------+-+-++                 /-   /-  |  (pretend striatum takes all inputs into account
                  | |                +-----/    |   at once for simplicity right now)
                  | |                |     |    |
                  | +------------->  |     |    |
                  |                  |     |  /-
                  v                  |     | /     |
            +-----------+            +-----+/      |
            |           |             striatum     |
            |         1 |                          |
            +-----------+                          |
            Thalamus, relay nucleus    |-----------+ inhibition via GPi

            +-----------+                          |
            |           |  |-----------------------+
            |         2 |                      |
            +-----------+  |-------------------+   |
            +-----------+                          | n per nucleus?
            |           |  |-----------------------+
            |         3 |                          |
            +-----------+  |-----------------------+ n?
                ...                     |
             n-relay nuclei       behavior orchestration organization
                                  perhaps 'the piano of behavior'

XX - active layer 5 neurons, correlated, with targets in the thalamus and striatum

In this model:

  • GPi is like an array of gas pedals that are 'off' by default, 'the piano of behavior'.
  • The striatum is removing the brakes, orchestrating what behavior is expressed.
  • The striatum is a brain, concerned with selecting, switching, and committing to behaviors.
  • It would make sense if this selection takes a Darwinian value judgment into account.

    |      +---+   |
    |      |   |   |   possible behavior streams
    | +-+  +-+-+   |
    | +++    |     |
       |     |
       |     |
       |     +---------                               +---------| THA 1
              commitment assemblies?  --------------| +---------  THA 2
                                                      +---------| THA 3
            'commitment organisation' ?               +---------
                                             behavior orchestration
                 | value judgment  ^
                 |                 |
     +--------+  |                 |
     |        |  |                 | dopamine ++
     |    D   +--+                 |
     |        |                    |
     +--------+                    |
                               [VTA?, SN ]

D - Darwinian Brain
THA - Thalamic targets
VTA - Ventral tegmental area
SN - Substantia Nigra

Note the failure modes of this arrangement:

  • Switching commitment too much: Erratic behavior?
  • Switching commitment too little: Failing to multi-task?
  • Committing too little: Parkinson's? Sleeping sickness? Catatonia? Basal Dementia?
  • Committing too much: Overeating, violence
  • Selection too permissive: Ticks, Tourette's?

Why the tremor of Parkinson's? Perhaps something tries to offset the lack of commitment?

Why swearing and ticks with Tourette's? In the model, the striatum is too permissive. (But the striatum is not encoding behavior information, it only selects). The logical conclusion is that there are behavior streams that output 'swearing behavior', virtually constantly.

But why is this useful for the information processing the brain is doing? Who says 'Thank you this is the kind of information I needed to do my processing'?


  • Swearing is a more primitive form of language (see Pinker 2007).
  • From biological cybernetics: When wondering how to build language into Braitenberg vehicles. It is a useful first step, to make vehicles make sounds for the situations they are in.
  • (semi-wild, but parsimonous I think) An information stream representing the notion 'I am dissatisfied', could be encoded in terms of swearing behavior.
  • With the bonus hunch that the insular/cingulate cortex would represent dissatisfaction (hedonistically, socially,…), and so forth.
  • If the striatum is too permissive, it could accidentally commit to expressing this behavior.

Not because the person wants to swear, but because part of how we analyze situations includes an encoding of dissatisfaction, virtually always.

I believe this dissatisfaction is a deeply animal aspect of our psyche. It puts us as an animal in our situation. It is simply part of the whole arrangement. Part of what gives it its flavor is hooks for memory lines and so forth.

Why ticks?

Memetics says there exists a meme, which is partly made from the tick behavior. If the striatum is too permissive for some reason, the memetic landscape is changed. A meme can strategize to span across striatum and express behavior which will, in turn, activate its activators, and so forth in a happy behavior loop.

This shows us that the healthy memetic landscape probably has a bias to not make the same behavior over and over and things like that.

Internal Behaviours

Elegant, nice hack: Use the same organization to orchestrate internal behaviors, 'cognitive moves'. Stuff like having a train of thought, deciding to remember something, guiding one's imagination and so forth.

Why is there no name for this, what the hell?

Names? 'internal behaviors', 'thought moves', 'guided cognition', 'stuff like imagination', 'stuff like using working memory', 'internal muscles', 'muscles of the mind', 'mental avenues to take', 'though-muscles', 'mind movement', 'mind behaviors', 'internal storylines', 'narrative muscles', 'narrative moves', 'internal cognition', 'reflexive cognition', 'attention', 'stuff like remembering', 'moving one's inner eye', 'moving in thought space'

This then sheds light on basal dementia (Corticobasal degeneration). Roughly, if the striatum is gone but perhaps the gas pedals (GPi) are still 'off' by default. You stop being able to think and move.

It was always very dubious to frame the basal ganglia purely in terms of 'behavior' as in muscle movements.


  • Other input outputs of striatum?
  • If the medial temporal is a Darwinian brain, what is the relationship between the striatum and medial temporal?
  • What makes the tremors of Parkinson's?
  • What is the detailed circuitry of the striatum->thalamus?
  • What is the relationship between the cortex, thalamus, striatum and cerebellum?
  • What is the nature of the 'commitment organization'?
  • What is the nature of all elements I have left out, GPe, subthalamic nucleus and so forth?
  • Is there 1 organization of cortex-striatum-thalamus repeated across the whole neocortex?
  • What is the nature of nucleus accumbens and reward pathways?

A High-Level Story for Midterm Memory Retrieval

You don't know how you remember something - and you don't want to know.

Let's consider an alternative to the action-hero meme thought experiment for a moment. Imagine we implement a cognition machine software, where the fundamental data structure is a 'narrative unit'

|                   |
   narrative unit

In order to remember something, the cognitive machine can pull a move like this:

    (mix I-know-what-I-remembered random-noise))


 +----+--------+-----+  narrative unit vague

X - concrete "I start remembering"
O - unspecified random noise
RO - Semi specified "I remembered"

This narrative unit doesn't have much substance you might think. Isn't there so much random noise?

Vague states are great templates, they are context to the meme-machine.

So you are some competent meme that knows if there is a situation 'I start remembering'. Then you can go and activate some other dude that will provide some half-baked memories to the system, and so forth. You don't know how it works either, but you know some buttons you can try in order to make your cell assemblies go in harmony with the overall situation. And the overall situation is that the cognitive user is remembering something right now.

Perhaps the thought pumps will not settle until some satisfying connectivity is reached in the memes. (some details are still left out from my story here).

But something happens that makes the narrative unit go in harmony with the rest of the network and tada - you have a filled in memory:

 run meme-machine,


 +----+--------+-----+  narrative unit concrete

X - "I start remembering" (concrete)
C - filled in memory states (concrete)
R - 'I remembered' (concrete)

Funnily enough, I think that is the one you remember afterward, too.

So the cognitive user will only ever see this thing working crisply and clearly.53

What happens when it doesn't work? Here is a pet hypothesis on tip-of-the-tounge from this perspective:

 run meme-machine,


 +----+--------+-----+  narrative unit - very concrete

X  - concrete, "I start remembering"
OM - mix of vague states and failed attempts
RT  - 'I remembered', 'I speak',  concrete (prediction)

The 'I remembered' part never really goes into harmony with the rest of the narrative unit, because the meme machine failed to deliver the memory details. You start having very concrete story snippets about failed attempts and 'search process' information states. They all didn't help. Leaving you very unsatisfied. The release of a 'good idea that fits well' (see More Cell Assemblies Thoughts) is missing - presumably creating a frustrating feeling.

I think what happens now is that the machine usually also has some tricks for delivering. Perhaps make the whole narrative unit (or situation) more clear/salient/important/central in the mind. Presumably, you do this by activating all the cell assemblies with high frequency, and increasing the threshold so other memes are drowned out. (this is not clear to me yet and very interesting topic of thought-mechanisms). (Such things are labeled extremely broadly as 'attention' in cognitive neuroscience).

Or perhaps you take the narrative unit states, put them into the thalamus and make it the inputs to the rest of the mind, but I'm not sure yet. Perhaps that gives you a fresh perspective on things.

Either way, you sort of sharpen everything in the unit, including the RT part. That is the part that is right now 'in the future', where you have remembered and you speak, using your tongue muscles. Since you activate the whole thing you activate all the memes strongly here - including the memes that have to do with moving your tongue. You want this story to work out, in a high level of detail.

Perhaps this feeling also tells us something about how speech is represented in the mind in the first place. Something that has to do with moving the tongue, that is for sure. And meaning feels like it can 'sit on the tongue'. It's funny that it is the tip of the tongue.

The bottom line is with vague narrative units you can build a lot of cognition. (If you have a nice meme machine at hand, too).

The fundamental notion of the cell assemblies is 'pattern complete'. We can get the whole out of a part. This is useful, it makes vague ideas meaningful.

Perhaps then 'confabulation', 'filling the blanks', 'everything a little bit until something fits', and 'auto-completing to success' can be the fundamental operation of a cognition machine.

This is simply to wonder how far you get with this main notion in mind. (See The Software Engineering Approach. You need to consider what the building material you have in mind can already do in a wider context. So that you can see what it cannot do yet or where it doesn't fit at all.)

'Harmony' here means what kinds of activations are supported by the network. These could be described as the 'resonant' modes of the system.

Braitenberg had the idea to call these 'conflagrant modes' maybe. To separate it from physics.

I wrote this before thinking about some more derived topics on this page.

Contrast And Alternative gives a mechanism of how the system 'stops thinking'. The cell assemblies that manage to ignite in each neuronal area can inhibit their alternatives via the thalamic input organization. Thereby making the system stop thinking.

A signal that 'keep thinking' is required might come from Socratic wires, which can represent the absence of a 'fitting' interpretation. Or perhaps the absence of some 'fitting' situation makes the thought pump oscillate.

Also, all Cortex activity is input to the thalamus already. It would not be a special move, but it is the default architecture.

What To Do With A Dream Mode

On Development

  • After an injury, CNS tissue is filled with glia, not neurons.
  • I reason from this that simply adding fresh neurons to the network is not useful. (and yes neurogenesis happens very locally in the hippocampus, the exception proves the rule).
  • It might be positively detrimental to add new neurons.
  • 1: why did not a little bit of neurogenesis evolve during wound healing and so forth?
  • 2: From ensemble reasoning, perhaps fresh neurons would simply make selfish activity.
  • Or there are rules of development, that must hold for the whole network at once for some reason.
  • Perhaps the thalamus or some other element is making special inputs that you cannot recreate for healing purposes.
  • development phases:
  • 1. peak synapse production, this is earlier in sensory areas, latest in the prefrontal,
  • 2. synapse pruning: over for sensory areas at 4th-6th year, prefrontal till adolescence.
  • If dynamism is so great, then why not cycle every x months between a new synapse production phase?
  • Clearly, something must say pruning required, or more plasticity and dynamism would not be useful.
  • The Living Language Problem: Brain software must solve the leaky abstraction problem of being made from living software. You don't want the elements of a language to have an agenda (or you want to minimize that somehow) because languages need to be obedient.
  • In general, I would say this pruning goes from high possibility -> lower possibility spaces.
  • (This is mirrored 1:1 in the empirical observation of perceptual narrowing )
  • This mirrors a little bit the general notion of natural selection: Step 1: many possibilities, Step 2: Narrowing down.
  • Why I think that from many possible software pieces, you get magical user interfaces is this: Memetic Engines Create Competence Hierarchies Up To User Illusions
  • I would put out there, that it is safe to assume the pruning rules are 'whatever is active, stays'.
  • It would be too strange if this was otherwise.
  • So this mechanism finds (over many years), ensembles that are more stable than others. And then it gets rid of the rest.
  • Unsurprisingly (because of the living language problem), the sensory areas 'freeze' or 'prune down' first.
  • This makes sense if we consider that the producer level of a language should acquire some generality, but then freeze (not living anymore in one sense, sad but that's the mechanism).
  • Then, you can give the user level, or next interpretation level, the chance to stand in relationship to this general language, freezing it after a while in turn, up the chain.
  • Idea: make random sensor-motor inputs and movements during development. Perhaps this primes the network to be a network that can represent the physical world.
  • Idea: traveling waves do something similar for a temporal structure.
  • (In case this wasn't obvious if you have something like Hebbian Plasticity, then you can shape the network a little by making things active together. If you make a traveling wave, you will get a network that makes synapses in that direction).
  • Idea: Generate neurons in cohorts, and make fresh neurons have high intrinsic firing rates, they are together so plasticity rules make them a subnetwork.
  • This would be a way to pre allocate ensembles. In a network.
  • The idea that 'the mind grows during dreaming' is just a cool notion I feel.
  • Children grow their bodies, so they also grow their minds.
  • How does a mind grow? Whatever it does when it dreams.

The Boltzman-Memory-Sheet

Here is a thought experiment,

Imagine we build a Braitenberg Vehicle with neurons, aka something similar to a Cortex and now we have decided to give the vehicle a memory implementation. Or, imagine an early animal evolving some cortex.

+---+          +----+
|A'<+----------+ A  |
|B'<+----------+ B  |
|C'<+----------+ C  |
+---+          +----+

bm-sheet       neurons

Make a companion neuron for each neuron in the brain. Say neuron A has A' as a companion neuron and so forth. Say that the companion neuron has a way to stay active infinitely.*

So we have a virtual sheet of neurons next to our neurons.

For any state of the neurons, we can make a snapshot (assume you have infinite sheets available, or you override the same sheet). If we feel like it, we can take the snapshot and put it to disk. This is a 1:1 representation saying what neurons are active.

I call this a Boltzman memory representation because it is a 1 moment in time Boltzman Brain.

If you have a wire going the other way, you can now re-instantiate the neuronal activity faithfully, simply first reset all neuronal activity and then activate all neurons from their companion neuron. (Which will be either on or off).

Observe that this memory scheme is useless, precisely because it is perfect. I.e. the vehicle doesn't gain knowledge by re-living an old situation, the power of (mid-term) memory and remembering must come from considering the current context together with a memory resource.

  1. We can expect the evolutionary drivers for mid-term memory to follow this context and memory resource logic.
  2. We observe that making a midterm memory is conceptually trivial, once we consider the notion of stable activity.

This ignores frequencies and so forth. Which is no problem, simply imagine you have a population of neurons for each neuron representing frequency.

*) We can do this simply by having an autapse (it activates itself).

The Slow Place: Evolutionary Drivers For Mid-Term Memory

  1. Imagine some early cortex, simply representing some olfaction data. [Braitenberg 1977 mentions this, I guess he wasn't wrong. Also makes sense given that the olfaction cortex is the least derived cortex.]
  2. Now make some neurons extra sluggish, or make some Autapses (neuron simply activates itself),
    • make some slow neurons that store activity patterns longer than the other neurons.
  3. There is an evolutionary driver for 'memory' in biological cybernetics. Memory is fundamental to computation. Biologically intuitive: Memory must be one of the main reasons to have brains in the first place.
  4. If you have some slow neurons, the cell assemblies spreading into the slow neurons will live across longer periods. This way we have something in between short-term and mid-term memory.
  5. It makes sense we have evolutionary drivers for mid-term memory. For instance: Make a population of neurons, add things from 2.
  6. You have a problem now: The activity of staying around is great, but always remembering everything is useless. I.e. I don't want to remember constantly the banana I looked at 1h ago. So another driver to isolate the slow neurons.
  7. Now you have an evolutionary driver for storing and retrieving memories from that slow place.
    • A load/retrieve organization.

You have another reason to isolate the slow place, that is epilepsy from all the activity.

The hippocampus has been called a flash drive. I will just take over this nomenclature and call this hypothetical slow place the flash drive (area) implementation.

Flash drive substrate

Hypothetical neuronal substrate that stores activity over longer time spans. The evolutionary driver for this is to keep cell assemblies alive for longer.

  • make many small neurons
  • slower neuron tick rate, because the activity just needs to stay around
  • make autapses or tight self-activating cell assemblies

Load, Store, Retrieve, Query, Result, Remember

From simple software engineering reasoning, we can be sure about some aspects of a flash-drive memory:

Data must be loaded, in a way that enables retrieval later on.

Data must be stored, frozen for all we care. For all we care that is allowed to be implemented by a USB stick (flash drive).

There must be a useful retrieval operation.

You don't want to remember all the things all the time, it is useful for the storage to be relatively isolated, and have some regulated way of saying 'Give me a memory… with criteria a,b,c …'

Retrieval can be further split into a query and a result.

Finally, the rest of the cognition machine can use the result for making a remembering situation. See A High-Level Story for Midterm Memory Retrieval for a cognitive-level idea about remembering.

I think it will always have to work like that if the data is not accessible in the network (long-term memory), then the network will have to load the data. The cognition machine and the flash drive stand in a similar relationship as the cognition machine and the sensors.

A-box flash-drive communication

It seems parsimonious to assume that the flash drive simply evolved as a sub-population of an existing conceptron (assembly calculus neuronal area).

Evolutionary early:


   |                  |  'online' cognition area
   |  +------+        |   ('fast' area)
   |  |    A |        |
   |  |  A   |        |
   |  |   A  |        |  slow sub population neurons
   |  +------+        |
         neuronal area whole

A - Cell assembly spreading across the whole area


   reset the online cognition area or new sensor inputs



   |         +---v--+ |
   |         |    B | |
   |   ^     +------+ |
   |   |              |
   |  ||  A  |        |  slow sub population neurons
   |  +------+        |
         neuronal area

B - Cell assembly with support from the sensors
A - The slow place part of A survives the reset,
    flash drive A part is now another input to the online area

Evolutionary derived:

|                  |    online cognition area
|                  |    (conceptron)
|                  |
|  |         ^     |
|  |         |     |
   |         |
   |         | Load and retrieve organization
   |         |
   |         |
|  v         |    |  flash drive
|                 |  (formerly slow neurons)

Note that online cognition area and flash drives are allowed to specialize. The online cognition area doesn't have a driver to implement mid-term memory on its own. We can expect this element to be a good 'fast' cognition area, then.

So 'automatic' load and retrieval would be the evolutionarily preserved operation.

The evolutionarily more derived specialized organization (where the flash drive is more isolated) has the job of regulating this load and retrieval.

It is like we have a substrate that is eager to spread. And our job is to regulate how it spreads. You might imagine this with water pressure and valves. But in some ways, it is closer to imagining this in terms of a wildfire. The wildfire just wants to spread into the available forest. This makes load and retrieve trivial, you only need some stretch of forest between 2 places and cell assemblies will spread.

We need some kind of network valve, then. To say what goes between the flash drive and the conception. Note that this valve is in principle the same kind of stuff again, a piece network that is ready to be populated by cell assemblies.

Here is a speculative idea, called association-box:


             |          | conceptron
             |          |
             |         -+-------------------+
             +----------+                   |
 +------------+  +---------------+  +-------+-----+
 |            |  |               |  |       |     |
 |            |  |               |  |       |     |
 |            |  |    A?         |  |       v     |
 |   F        |  |         <-----+--+--- Q  Q     |
 |     -------+--+-->            |  |        Q    |
 |            |  |               |  |             |
 |    F       |  |               |  |             |
 |            |  |               |  |             |
 |            |  |               |  |             |
 |            |  |               |  |             |
 +------------+  +---------------+  +-------------+

  flash drive     association box      'query area'
                  a-box                'q-wires'

Q - Query cell assemblies
F - Flash drive cell assemblies
A? - Potential overlap cell assemblies of a-box

This is very similar to the m-box of the explanation structures above. There is some intermediate level between the 2 areas, and with enough support from both areas, you get cell assemblies stretching all 3. But without it, you don't.

Result case:

             |+---+     | conceptron
             ||   |     |
   result    ||^  |    -+-------------------+
             +++--+-----+                   |
               |                            |
               |                            |
               +-----+                      |
                     |                      |
                     |result-wire           |
                     |                      |
 +------------+  +---+-----------+  +-------+-----+
 |            |  |   |           |  |       |     |
 | +----------+--+---+-----------+--+-------+---+ |
 | |          |  |   |           |  |       v   | |
 | | F        |  |      A  <-----+--+--- Q  Q   | |
 | |   -------+--+-->            |  |        Q  | |
 | |          |  |               |  |           | |
 | +----------+--+---------------+--+-----------+ |
 |            |  |               |  |             |
 |            |  |               |  |             |
 |            |  |               |  |             |
 +------------+  +---------------+  +-------------+

  flash drive     association box      'query area'
                  a-box                'q-wire'

FAQ - A flash-drive-association-query spanning Assembly

Whatever cell assemblies are on inside a-box are the results, so then it is a matter of having some a-box -> conceptron wiring to get the a-box results into the cognition machine (for remembering).

We see that would be useful for a-box to have a very tightly controlled inhibition.

  1. If a-box is active too much, flash drive activation would leak into the result wires. And presumably, there is so much flash-drive activation that this is a liability, potentially causing epilepsy. It is not surprising then that hippocampus and related structures are associated with epilepsy.
  2. A-box is useless if it is active too much. It would be equivalent to confabulating mid-term memories.
  3. A-box would be evolutionarily built to be some sort of valve.
  4. Note that there doesn't need to be that many special things about a-box, it should be a relatively generic citizen of the rest of the conceptron.

Having frequent resets of a-box perhaps, and regulating it very tightly, would be evolutionary drivers.

Note that these are high-dimensional spaces with sparse activity.

      |         |Sit|                                               |  conceptron (cortex)
                |   |
                |   |
      |         |FFF|                                               |  flash drive

Sit - Situation cell assemblies
FFF - Flash drive cell assemblies

In this drawing, we pretend we can collapse the high dimension into a single dimension. Also, we align them topographically. In reality, these are allowed to be distributed cell assemblies in the network.

How would load work with an a-box?

Perhaps it is allowed to be the same thing.


             |          | conceptron
             |          |
             |         -+-------------------+
             +----------+                   |
 +------------+  +---------------+  +-------+-----+
 |            |  |               |  |       |     |
 | +----------+--+---------------+--+-------+--+  |
 | |          |  |   A           |  |          |  |
 | | F      --+--+---->    <-----+--+--- S  S  |  |
 | |          |  |     A         |  |        S |  |
 | |          |  |               |  |          |  |
 | |  F    <--+--+-----          |  |          |  |
 | +----------+--+---------------+--+----------+  |
 |            |  |               |  |             |
 |            |  |               |  |             |
 +------------+  +---------------+  +-------------+

  flash drive     association box      situation area

FAS - A flash-drive-assication-box-situation cell assembly

If we have a hypothetical situation area, could span cell assemblies across flash drive, a-box and situation area.

Just one alternative, example:

Situation areas, part of conceptron.

                    | ^     ^  | conceptron
                    | v     v  | situation areas
 +------------+  +------v--------+
 |            |  |               |
 | +----------+--+-------------+ |
 | |          |  |   A         | |
 | | F      --+--+---->        | |
 | |          |  |     A       | |
 | |          |  |             | |
 | |  F    <--+--+-----        | |
 | +----------+--+-------------+ |
 |            |  |               |
 |            |  |               |
 +------------+  +---------------+

  flash drive     association box

FAS - A flash-drive-assication-box-situation cell assembly

This situation area would be useful if it represents a hyperdimensional point in 'time-space-conception' space.

As the situation of the animal changes, the situation area represents new cell assemblies, and the a-box activity moves forward.

The activity in a-box would open a relationship between flash drive and conceptron. And a-box would not differentiate between the current situation and a memory query.

This would mean that every time you retrieve from a flash drive, you have the chance to modify the memory in the flash drive. In the limit, this would mean you are re-creating the memory in a flash drive for each retrieval. Presumably, this depends on implementation details like the amount of plasticity in flash drives and so forth. This seems to map to empirical observations from cognitive neuroscience of memory, afaik.

Pointing to the fact that there is indeed some element in the system that doesn't differentiate between query and load.

Perhaps déjà vu is triggered by some 'situation' at activates a lot of flash-drive<->a-box cell assemblies? Then some 'result' wires are on, even though the system was not in query mode.

Perhaps this would flip some interpretations into thinking that a memory is being retrieved. But confusion, because no query was set?

The temporal extent of déjà vu might then perhaps be a hint about some temporal aspects of the information processing of this arrangement. Perhaps after a few seconds the 'time-situation' moves forward, and the familiarity with the situation moves forward, too.

                           +----  at t1, the situation moved forward
                  t0      t1
      |         |Sit|--> |Sit'|                                     |  conceptron (cortex)
                |   |    |    |
                |   |    |    |
      |         |FFF|    |F  F|                                     |  flash drive
                  +-- unusual large cell assembly

Sit - Situation cell assemblies
FFF - Flash drive cell assemblies

And the strange overlap of situation and memory would be over.

2-mode a-box

A mode for retrieval and a second mode for load.

  • From the neuroscience of the hippocampus



Figure 10: Schematic showing regions of the hippocampus proper in relation to other structures.

The neuroanatomy of the hippocampus says 'I need to be isolated'. Curled up with the extra ventricle and so forth.

It is interesting that empirically, cell assemblies can be re-ignited with single neuron activations. [See Rafael Yuste presenting this work].

It would mean that single target k-lines [Minsky] might suffice as retrieval implementation.

That is, simply make a bunch of random wires that go from flash-drive to Conceptron. Even activating single neurons will already ignite cell assemblies.


  • If a flash drive has an evolutionary driver for slow activity, perhaps the reason for theta rhythm in the hippocampus is that there is some other limiting factor, saying that theta is after some criterion the slowest possible.
  • I think this means that when building an artificial conceptron with a flash drive, we can think in terms of 'very slow' or 'frozen'.

Creating Sync Activation

This assumes that the 'area' neurons (cerebral pyramidal neurons) have this property (at least roughly).

'Reactive Phase' Property (Hypothetical)


+---------+     +-----+    +-----+
|      O  |     |  X  |    |  X  |
+---------+  ,  +-----+ ,  +-----+

      t0           t1         t2

 [synaptic input]


          ---->  shifted by 0.5 time

+------------+    +-----+    +-----+
|         O  |    |  X  |    |  X  |
+------------+ ,  +-----+ ,  +-----+

    t0  t0.5     t1.5       t2.5

     [synaptic input]

O - Inactive
X - Active

That is the time of the input determines the time of the firing. This allows a mechanism that abuses the fact that if the input to multiple neurons is in sync, the resulting neuron firing is in sync, too.

This might or might or might not be how cerebral pyramidal cells work.

As we shall see from the mechanisms shortly, you might at first wonder about the absence of any 'synchronizing' elements. It might be a feature, not a bug.

I have since realized this is studied under the topic of Phase resetting.

Re-ignite From A Shared Paced Nucleus

This would depend on the reactive phase property.

Step 1:

(This also applies to neurons in a single area).

Let's say you have 2 areas with neurons in 2 phases:

      +---------+           +-------------+
      |         |           |             |
      |  Xa     |           |   Xb        |
      |     Xa  |           |          Xb |
      +---------+           +-------------+
                area A                area B

Xa - neurons firing in phase a
Xb - neurons firing in phase b

Step 2:

They both span neuronal ensembles into a shared nucleus C.

      +---------+           +-------------+
      |         |           |             |
      |  Xa     |           |   Xb        |
      |  ^  Xa  |           |     ^    Xb |
      +--+------+           +-----+-------+
         |      area A            |   area B
         |                        |
         |                        |
         |      +-------+         |
         +------+-> X <-+---------+
                |    X  |
                       nucleus C

Xa - neurons firing in phase a
Xb - neurons firing in phase b
X - neurons in nucleus C (perhaps with 1/2 tick rate)

Let's say that C is a paced nucleus, which has neurophysiology that keeps it in a certain phase. All we need to require is that the neurons in C can be excited from A and B, but only fire from an 'intrinsic' phase-giver. (This is biologically plausible as far as I can see. A theta rhythm pace nucleus exists if I remember correctly). We can easily imagine implementing this with cycling gradients of chemicals or so forth.

Observe that X is allowed to be an associated neuronal ensemble, spanning A, B and C.

Then, you can extinguish the activity in A and B, and subsequently reignite from C, where now everything is in phase because it comes from C.

Step 3:

Extinguish activity in A and B:


      +---------+           +-------------+
      |         |           |             |
      |         |           |             |
      |  ^      |           |     ^       |  (empty, but not for longer than 2 neuron time steps)
      +--+------+           +-----+-------+
         |      area A            |   area B
         |                        |
         |                        |
         |      +-------+         |
         +------+- X   -+---------+
                |    X  |
                       nucleus C

X - neurons in nucleus C survive


      +---------+           +-------------+
      |  X!     |           |     X!      |
      |         |           |             |
      |  ^      |           |     ^       |     re-ignite, same phase
      +--+------+           +-----+-------+
         |      area A            |   area B
         |                        |
         |           !            |
         |  !   +-------+   !     |
         +------+- X   -+---------+
                |    X  |
                  ^    nucleus C

X! - spanned neuronal ensemble, same phase (from C)

Ignite or Re-Ignite From Inputs


               | ^    | empty
   +------+    +-+----+
   |    ^ |      |
   +----+-+      |
        | !      | !
        |        |
        |     +--++
        +-----+   |
                 input nucleus


               | ^ X! |
   +------+    +-+----+
   |  X!^ |      |
   +----+-+      |
        | !      | !
        |        |
        |     +--++
        +-----+   |
                 input nucleus

               ! t0

X! - in phase, from the input nucleus

If we have the operation available to suspend input nucleus, we can have the following arrangement:


       +-------> X    |
   +---v--+    +-+----+     X ensembles still alive, still in their phase
   |  X   |

              |   |
                 input nucleus

    [ t0, t1, ____ ,  ]

                    put the nucleus on hold for a little...


       +-------> X    |         X in phase 1
   +---v--+    +-+----+
   |  X   |

   +-----+            +------+
   | A   |            |  ^ A |  A in phase 2, depends randomly on the hold period
   +---^-+            +--+---+
       |                 |
       |         +-------+
       |         |
       |         |
       |         |
       |        !|
       |      +--++
       +------+ A |
                 input nucleus

 [ t0, t1, ____ ,  ]

 [ O, X, O, X, O, X, O ]   timeline X
            +----|    phase difference
 [ O, O, ______, X, O, ]   timeline A

                 ta - the load time for neurons A

X - still in phase from input load time t0
A - in phase from load time ta

Observe these 2 cases: If you suspend in exactly 1 neuron time step time or a multiple of it, you will go exactly in the rhythm of X. In this case, igniting A after the suspense time will make X and A go in sync. But if you do not care about the rhythm of X, and you suspend for any stretch time, then the A inputs will arrive out of sync with X.

Note that X and A could be allowed to exist in the same area, in this case, neurons would fire out of phase in the same area (empirical question).

Suspend And 'Auto Compose'

We can imagine building the following into a Braitenberg vehicle:

move    +-> [ eyes ]
        |     |
        |   +-v-+
susp.   +--||   | input device
        |   +--++
        |      |
     +----+    |
     |    |    |
     +----+    |
 eye motor d.  |
          |    v  |
          |       | neurons

  • Input device receives sensor data from the eyes.
  • Input device a paced nucleus again, so all its wires are on at the same time.
  • Neurons are a network of excitatory units that support neuronal ensembles.
  • Eye motor device moves the eyes of the vehicle.
  • At the same time (with a branching wire to be sure), send a suspend signal to the input device.
  • The suspend signal should suspend the input device roughly as long as the eye movement takes. (would be ok if it takes slightly longer, but shorter might be a problem).
  • Sensor data changes rapidly during the movement and is relatively static in between movements.

        |                [B]    |
        |                       |  visual field, actual
        |                       |
        |  [A]                  |

        (A)              (B)
         |                |
         |                |
         |                |
inputD +-+-+            +-+-+
       +-+-+ Pa  [___]  +-+-+ Pb
         |   |            |   |
     +---+---+        +---+---+
     |   v   |        |   v   |
     |   A  <+        | A B  <+
     +-------+        +-------+

        t0               t1

          eye movement,
          suspend input device

P - eye position encoding
A - neuronal ensemble representing the visual object A and Position a.
B - neuronal ensemble encoding B and position Pb, in a different phase from A

Let's also add the position P together with the input device. When the vehicle moves it's eyes, the input device sends position and eye sensor data to the neurons, this is synchronous because the input device has a controlled phase.

When the vehicle moves its eyes, we suspend the input device at the same time. The ensembles are allowed to stay active in the neurons. We see ensemble A, representing sensor data about the A object, composed with the position data, is allowed to stay stable in the network.

After the eye movement is done, at t1, new sensor data is coming into the neurons. If the eye move time interval is allowed to be any kind of interval, i.e. non-discrete, the fresh neurons B will be virtually guaranteed to have a different phase from A.

With this mechanism, the eye movement and input apparatus of the vehicle is like a moving conveyor belt that represents object data, it doesn't move around topologically, but it moves around in time-phase-space, and it ignites representations of the sensor data + and position data at different places in this time-phase space.

This mechanism would create ensembles of synchronously firing neurons, which encode position and sensor data together. This is one idea of how to make a visual field. Since everybody else in the system could see what position goes together with what sensor data.

"Auto composes" because this mechanism is automatic, but it depends on a 'reactive phase' property of the neurons.

Observe that this theory also fits this:

If nobody cares about the synchronicity, this would come as an artifact of the mechanisms of the input nucleus. It might then be that you observe sync. activity in the brain but it doesn't serve much of a functional role in the system.

  • Maybe you don't need to 'suspend'?
  • If the machine has a gate on the input nucleus, then perhaps attention can be a compose mechanism.
  • If we can require 'phase reset' wires, then the thalamus could innervate with phase reset wires and doesn't even need to be in sync or phase itself.
  • Then, whatever is being paid attention to, i.e. what activity we allow into the cortex, would be in sync, because we are resetting the phase together with the 'new' inputs.
  • The phase reset wires don't need to be a thalamus concept.
  • Perhaps something like the logic of Modeling a threshold-device would oscillate whenever there is a lot of activity coming in.
  • This would be like implementing a bouncy substrate when you put an activity in, by paying attention, it bounces. And perhaps the bouncing is a phase reset implementation.

How To Make Use of Synchronous Activation

I have seen it said mysteriously 'it is not clear how synchronous activation does XYZ'. Where XYZ is then binding, awareness, consciousness etc. If you don't have a theory of the functioning of the machine, saying 'sync activation' is not even wrong. There is not even anything that can be wrong if you don't have an explanation (a theory) that says what kind of piece of mechanism you are looking at.

Synchronous activation only does something if something else in the system behaves differently, depending on whether there is synchronous activation, or not.

  • Neurons that are only above the firing threshold when there are synchronous inputs.
  • In a high-dimensional network, since you can listen to everything a little bit, your neurons can tell you what is active together.
  • This depends on the time integration properties of the neurons. Presumably, there are different kinds of neurons in the brain.
  • I.e. what is the time window for 2 things to count as synchronous?
  • At the same time this window should be small so you can differentiate the synchronous populations apart.
  • The power of a sync. population scheme comes from the fact that now you have composable elements.
  • I.e. if you have a bunch of musical notes or building blocks, you want to be able to put them together in all kinds of ways, this would be the main point for a compositionality.

Earl Miller 'Cognition is Rhythm'


He mentions the work of Wolf Singer (below), too.

Cortical neurons are multifunctional, they have 'mixed selectivity'. In other words, neurons are part of different ensembles.

Interesting that in Hydra these things are separated into modules 54.

The data support a model where pattern completion may arise at this near decomposability of circuits into their subordinate functions and information representations and also suggest a flexibility for this feature, depending on circuit stability, given that, as Hydra’s circuits mature, this feature is lost. The flexibility of processing in the mammalian cortex potentially leaves this feature open, creating an inherent hierarchy at the overlap of this functional decomposition.

The base mechanism [Singer, Abeles]:

  1. Neuronal net where neurons can have different phases.
  2. Compose ensembles (temporarily) via sync. activation.

E. Miller calls this 'Neurons that hum together'. (I don't know why he then says wire together there).

–> Some kind of nice on-the-fly composability was required for brain software that composes explanation structures from elements

(I.e. the means of combination of the software paradigm.)

An ensemble B is a temporarily allocated, coherent subpopulation in the network, firing at rate b-rate. Let's call B ensembles that fire in the beta wave range, and G ensembles that fire in the gamma wave range.

I didn't gather what Miller thinks how this sync. activation is created.

They say that ensembles go in beta oscillations top-down, encoding 'prediction'. So [B1, B2, B3, ...], call it beta. Prediction error ensembles are presumably a set of ensembles [G1, G2, G3, ...] which only ignite if the beta prediction is violated at sensory/earlier cortical areas. (Call it theta).

The model:

top-down            +---------------- shallow cortical layers
prediction          |
       | +-----+----+-+   ^
       | |     |    | |   |    <----- high cortical area
       | |     |      |   |
       | |     |      |   |
       | |     |      |   |
       | |     |      |   |
       | |     |      |   |    <----- early cortical area
       | |     |      |   |
       v +-----+------+   |
           |           'gamma'
           |           bottom up
       ----+           prediction error
 deep cortical

When Miller talks of beta, gamma and theta in this talk, I have 3 superheroes in my mind that battle it out.

"Beta is predicting. Beta is suppressing Theta…"

The level of abstraction is very high. Since Miller is lumping the ensembles into their frequency bands.

I see the danger of this landing a bit on wonkyness to other neuroscientists. I suppose it is obvious to Miller that the mechanisms are open questions.

I like that the whole point of frequencies is to separate the ensembles, so I split this again:

                  ??? override, extinguish?
                B <----+
        | +--+ +--+  +-- |
        | |B1| |B2|  |G1||
        | |  | |  |      |
        | |  | +--+  |  ||
        | |  |           |
        | |  |------>|  ||   sensor input
        | +--+  ???  + -+| <-----------------

B1, B2, - 'top-down' ensembles, each in their beta frequency
G1, Gn  - 'bottom up' ensembles

  • Presumably, different beta ensembles spread to different cortical areas. But it would be quite interesting if that spans globally.
  • What is the mechanism that makes a top-down ensemble 'suppress' a gamma 'error signal'? Whatever it is, it is a Socratic wire implementation. Confusion and Socratic Wires.

The setup:

      top-down ensemble
   |   |
   |   |
  ||   |          |
  || B |          |
  |+-+-+          |
  |  |            |
  |  +--[b] <-----+--------- [input]
  |  |            |
  +--+------------+       sensory area (for the sake of argument)
case 1: input roughly supports B so B stays active

      top-down ensemble
   +---+ :(
   |   | It is sad now. No support from the sensory area
   |   |
  ||   |          |
  || B |          |
  |+-+-+     [g]  |
  |  |        |   |
  |  +--[b]   +--+--------- [input]
  |  |   :(       |
case 2: input doesn't support B, but g

Idea 1:

  • Since B is not supported by the sensors,
  • there is a sudden drop in overall activity in sensory-area
  • This drop makes some threshold-device oscillate,
  • This could mean that the thalamic input gate is opened wide
  • Or, that B was inhibiting its alternatives at layer 6 (-> cortico-thalamic), and now this inhibition is suddenly gone.
  • A rush of activity comes in, activating even more [g]
  • And this is somehow in gamma? Thalamic bursting mode?
  • The system flips into a second mode, a gamma signal mode,
  • which is an ensemble that spreads through the shallow cortical layers?
  • and somehow this is now undermining the support for B on all cortical layers/areas?
  • This must be a very parallel mechanism, following thalamocortical circuits which are parallel (?)
  • Then G would gain sudden support in the network, while B is loosing support.

    top-down ensemble
 +---+                      B not inhibiting gamma nucleus anymore
 |   |
 |   +------------------------------|    ???
++---+----------+            --       [ gamma nucleus ]
||   |          |                         |
|| B |       !  |                         |
|+---+      [g] |                         |
|            ^  |  !!              ++     |
|            +--+--------- [input] <------+
|               |

          [ G ] :D

Gamma nucleus?

  • Maybe claustrum somehow orchestrates excitation (threshold-device-like) or makes phase resets (somehow gives a signal to go in gamma? - that sounds strange to me)
  • Maybe Layer 6 cortical->thalamic inhibition? (and/or Layer 6 -> TRN)
  • Perhaps 'fresh' neurons in the thalamus go into burst and that makes gamma activity?
  • Then thalamic relay neurons would suddenly receive sensor input and their inhibitory modulation would drop at the same time,
  • They would be fresh relay neurons, we could biochemically implement them bursting now
  • Or some other mechanism

It is aesthetically pleasing that in this mechanism the strong gamma frequency can override the weaker beta. But beta has the benefit of already being established.

There must be mechanisms or wiring that prevent the 'top-down' ensembles from falling into gamma frequency by themselves. It would then be a memetic landscape trick, that 'established' ensembles go in the mediocre 'power' of beta, which has the chance to be 'overruled' by a Socratic input override mechanism, using gamma frequency to establish a sudden swap from B to G in attractor space.

Perhaps this can be achieved by biochemically making thalamic relay neurons 'tire' of making gamma/bursts. You would then be an ensemble stretching the organization, but your relay neurons at the thalamus give a low hum of beta only.

Back to the superheroes for a sec:

                     +-------------- broadly speaking inhibitory
     Beta            |         Gamma
   +---------+       |      +---------+
   |         |-------+-----||         |
   |         |              |         |
   |         |              |         |
   |         |              |         |
   |         ||-------------|         |
   +---------+              +---------+
     --+-                     ---+--
       |                         |

          the balance of power        ^
     ^                                |
     |                                |
     |                                |
     |                                |
the benefit of being             the benefit of being 'fresh'(?)
established                      the power of gamma? (~5-10 faster than beta)

If beta is too strong, your top-down expectations rule how you perceive the world - hallucinations, schizophrenia, … ?


If gamma is too strong, your expectations are overruled by reality too much, presumably, your world is less infused with meaning. - autism?


From memetics, we know that if any of the mechanisms fail, the ensembles will happily exploit the circuits to be active more.

During hallucinations, perhaps the mechanism that prevents 'established' ensembles from going in gamma is broken. Perhaps the mechanism that makes unexpected inputs go in gamma is broken. etc.

  • Perhaps the absence of a (newly established) beta ensemble is a confusion signal
  • It is interesting that theta and gamma are phase-aligned

Miller lit:

  • I didn't mention his traveling wave data
  • Also, working memory has a position. Frontal activity is on during working memory, is either the left or right hemisphere, depending on the location of the stimulus. This follows the left-right split of the visual field. This is quite astounding. The default assumption might have been that visual inputs somehow distribute on hemispheres, but this says that this stays split. That working memory has a location, too somehow perhaps.

Composition / Neural Syntax

Beyond sync. ensembles.

The time step composition idea of neuronal ensembles:

     theta freq.

     alpha freq.

| A         |  B        |      2 beta freq. fit into 1 alpha freq. and so forth

     beta freq.



Vaguely, the idea is that ensembles would be composed in time frequencies of multiples (of two?). That is ensembles A and B fire in beta, then, if B activates A again, the whole thing makes an alpha time step. Youc an also say the waves are cross-frequency coupled.

But what are the mechanisms that produce this composition? Surely, there must be answers from simple cybernetic reasoning of the neurons and the devices around them. (G. Buzsáki's work is probably relevant).

Something cool about this idea is how the ensembles are composed very similar to music; The aesthetics of this are neat.

Moche Abeles

György Buzsáki calls this a neural syntax.

Origin of syntaxis

First recorded in 1535–45; from Late Latin, from Greek: literally, “arrangement in order,” equivalent to syntag- (stem of syntássein “to arrange together”; syntactic ) + -sis -sis

From dictionary.com.

The core notion of syntax is the ordering or arrangment.

Buzsáki's recognition of the importance hierarchical organization of brain rhythms of different frequencies and their cross-frequency coupling



On the topic of music in rhythm, perhaps music is to some extent exploring the syntactic rules of ensembles and so forth, then. Because why are these multiples of 2? The cybernetic psychology of music would eventually perhaps describe how it is orchestrating ensembles in our brains. Perhaps whatever explains Why Flowers Are Beautiful, Elegance, Explanation, Aesthetics also explains what music is doing to the mind.

Perhaps audition is especially well suited for this, it is a fast sense compared to vision (2), and perhaps it is already tuned for representing temporal sequences, more than other senses. Motor movement, too, presumably deals with temporal sequences. There seems to be some kind of rhyme then between these aspects: Audition, movement, music, rhythm, dancing and playing instruments.

Syntax and composition would be a common thread then, linking this to one of the core functionalities of brain software. (It is the means of combination from 5).


Consider a neuronal ensemble with a well-connected center, and some less-connected outskirts, or 'halos':

        [ halo ] 1    [ X, O, X, O]

[ center ]            [ X, X, X, X ]

        [ halo ] 2    [ O, X, O, X ]

                       t0, 1, 2, 3, ...


Figure 11: This was created from https://vehicles.faster-than-light-memes.xyz/art/p/assembly-friends/3 (move a sensor ball into the visual field). With a threshold device, neurons go in and out of being suprathreshold. You will see neurons in 1/2 neuron tick rate and so forth.

Note that when you look at neurons, if the neuron fires faster or as fast as your measurement device's time resolution, it will look statically 'on', this is simply a measurement issue. Just as is the case here, too. The time resolution is how often I re-draw the neurons, and I re-draw them as often as the tick rate. So the neurons that look 'on' all the time are firing at the fastest rate.

If we have discrete timesteps, you see the center neurons firing at each time step, (i.e. they are all suprathreshold every time).

The halo of the ensemble is either active or not, depending on the activity flow of the rest of the network, as well as the threshold device (inhibition) rules.

The halo neurons might go in a frequency that is multiples of the neuron tick rate.

A composition theory of ensembles will have to live up to the challenge of explaining neuron frequencies better than this default 'way the mechanism is set up' way.

Sync by Overlap Kernel

This is a variation of the re-ignite scheme:


        | A    |
        |   +--+----+
        |   | C|    |
        +---+--+    |
            |     B |

A and B - ensembles firing out of phase
C - overlapping A and B


           | C  |   |------------- thought pump

    reduce excitability, until only C remains.


Re-ignite from C:

         | +----+
         +-+ C  +--+
           +----+  |

        | A'   |
        |   +--+----+
        |   | C|    |
        +---+--+    |
            |     B'|

A' and B' - same ensembles but in phase (from C)

Observe that step II in principle would be allowed to go down to a single neuron. Or, with intrinsic firing rate, and on-the-fly excitability (Alternative Plasticity Models), you might even simply extinguish the whole ensemble, then it could be re-ignited from either a single random neuron or from intrinsic firing rate that goes in-phase together for some unspecified reason.

I realize that C doesn't need to be in the overlap either, if A and B are associated in the network, then a kernel of activity without overlap would perhaps activate them both, this time in phase with this kernel. Again, this depends on the hypothetical 'reactive phase' property.

Synchronous Activation Might Not Matter That Much

Note that it is not clear that synchronous activation or even a composition means much. It might be more or less artifacts of the mechanisms of this thing. It might be we are looking at the car engine vibrating and wondering what the vibration does for the acceleration of the car.

I see that it makes a lot of sense to associate and compose hyperdimensional data structures. I label this 'expectation structure', it is a bit like asking for a banana and getting the gorilla together with it. In a network where things like bananas and gorillas are dynamically associated, depending on the situation. This is useful for instance, when the position encoding of an object goes together with the what encoding. (Getting A Visual Field From Selfish Memes). And this might already be the whole mechanism that is needed (without the need for sync. or rhythmically composed activation). If X is on, then as a consequence, A is on. That is, if the system pays attention to position X, then A becomes more relevant in the system, too. And this is as fast as the neurons go. This might be enough to represent the notion that what is located at X is A. And since A has the chance to inhibit its alternatives, it is only A and not some other object.

The point of course is to say it might just be a 'bind'. But perhaps the system mines synchronicity to make a bind. Perhaps both of these things happen in a layered way.

But It Does

Didn't have to be this way, but seems like empirical work shows the cortex is well-suited for sustaining and using synchronous activation.

I found:

On the role of neural synchrony in the primate visual cortex Andreas K. Kreiter and Wolf Singer (1996).

In: Brain Theory: Biological Basis and Computational Principles. (1996). Netherlands: Elsevier Science.


  • Do neurons fire out of phase in the same area?
  • If they do but it is multiples of each other, that might just come from a thought-pump, or simply activity shifting around.
  • (to see why this is true, imagine a neuronal ensemble with a center, which fires at every tick, and some surrounding activity that flips back and forth between a bunch of neurons - they will all fire apparently in 1/2 the frequency).
  • Do neurons have or do not have the 'reactive phase' property?
  • What phase-maker nuclei exist and what is their relationship to the brain?
  • What sync-listener neurons exist?
  • (otherwise, we can say sync activation probably just doesn't matter).
  • Perhaps other syncing-up schemes exist, perhaps neurons firing together naturally sync up as clocks do or something. (Irrc. The electronic field of the neuron activations is a candidate mechanism for this. Wild but what can you do?). If this is true, this would enable a completely separate or alternative set of mechanisms than what I said here.

Coherency Challange Model of Working Memory

Just an idea.

  • Make an area that is very hard to spread into. Let's call it the challanger-node for reasons that will become clear shortly.
  • (Why that is PFC in the brain: 1)
  • (The link between prefrontal and working memory is empirically tight at this point).
  • Assuming that neuronal ensembles compose, that is they hum together (How To Make Use of Synchronous Activation).
  • Because of the No Gain, No Chain property of ensembles (The Biology of Cell Assemblies / A New Kind of Biology), if I am a meme and my competitors manage to spread into the challenger-node area, when they can receive support from it, then I also care.
  • This also meant that ensembles needed to be able to make chains with whatever other area. This is given by high dimensional computing, if we allow everything to listen a little bit to everything else.
  • It is not useless to have a very hard-to-get-into area (challanger-node). It will shape the memetic landscape, biasing to the existence of memes that fulfill the challenge of the hardness to get into.
  • One idea that resonates (hehe) with things like Dennetts fame in the brain, Baars 'global workspace' and current neuroscientific theories of subliminal/conscious information state that there is high coherence for a particular "idea" in the network.
  • I say that an ensemble has a lot of support from the network.
  • If challanger-node has low excitability, or high attenuation, for instance, it becomes a coherency measurement device.
  • I.e. the challenge of the challenger node is how much support you have from the network.
  • Presumably, that means global support in the brain, since the cortex is connected with itself everywhere to some degree.
  • I.e. challenger-node is listening from all over the place to my ensembles. Only if my ensembles are especially coherent, i.e. fire together; Do I have the chance to excite neurons which can be part of my support in turn. (Also see Moshes Abeles synfire chains).
  • You would think this is then localized hemispherically in split-brain patients.
     /        -\
    /           \
  -/             -\
 /                 -\
                 coherent ensemble,
                 global support from the network

The presence of challenger nodes allows for the existence/selection of global coherent memes. A challanger-node can also represent the state of loaded working memory.

  • Via affordances, commitment, orchestration and the interpretation game (at this point you need much of the rest of the system to explain something), you can create and select ensembles that fulfill the expectation of having spread into challanger-node.
  • All these are topics of the rest of this page. But in short, a cognitive user has buttons with previews (Getting A Visual Field From Selfish Memes). In order to manifest a banana in the cognitive machine, some affordances represent the counterfactual notion of a loaded working memory with banana content. (This notion of a loaded working memory is what challanger-node provides additionally to the system).

    If these affordances are not available in the current state, the user must use perspective jumps ('navigate') Implementing Perspective, p-lines, which is affordances again.

    The user can commit to the notion of a banana in working memory. She of course does not know how it works (Magical Interfaces, The Principle of Usability, Memetic Engines Create Competence Hierarchies Up To User Illusions). She is selected mimetically, too. To be a user that doesn't know, but just demands.

    This creates a situation (or dynamic memetic environment, equivalent to a procedure in the software paradigm Context Is All You Need?).

    This situation might be created by perspective wires and is allowed just as anything else to be random in a baby's brain. (I.e. your buttons do random things, in a parallel memetic search mechanism you will find memes that eventually represent approximate knowledge of how to use the computer this software runs on).

    Finally, the system is not settled, until the interpretation reaches some amount of stableness. The failure to find a stable interpretation might be some kind of confusing situation like tip-of-the-tounge for the cognitive user.

  • So the working memory hypothesis here says that the activity of the challenger node represents whether or not an idea is coherently, and globally represented in the network.
  • This gives the rest of the system a handle on how to bring about such a situation.
  • You are probably not coherent unless you cause the system to ignore your alternatives, and to fill in the blanks of your details (confabulate). The hunch is that global coherent activity is more meaning-laden than 'local activity'. This potentially sheds light on autism, working memory issues in autism and so forth. Perhaps their network is too locally connected for some reason, perhaps because their PFC is too hard or too easy to get into? Or perhaps because of developmental differences that change the connectivity?

This is just an idea. Doesn't say why there are directed traveling waves in PFC during working memory tasks.


  • Why directed traveling waves in PFC?
  • Presumably 'working memory loaded' must be represented at layer 5 (affordances), somewhere. Or everywhere more or less?
  • Presumably the affordance pay attention to x is either identical to the 'working memory loaded' affordance, or tightly linked.
  • Would it be fair to assume that the presence of working memory changes the memetic game completely? One would almost think that once this is there, the memes that don't play to be coherent globally are not significant or something.
  • Here is a further idea:
  • If you lower the excitability globally now (let's say this is another button to press via an orchestration mechanism).
  • Then you increase the challenge of challenger-node dynamically, and because this is a measure of global coherence, I feel like this would say the ensembles need to fit the network exceptionally well. I.e. this is a mechanism to find precise ideas that fit well (G. Palm).
  • This sounds a little bit like an abstract thought to me, where you try to have a minimal and sufficient explanation structure.
  • See also . I have speculated that a threshold device might be responsible for Hofstadters 'abstraction ceiling' brain-feeling.
  • Don't migraines feel like some kind of muscle of the mind is tensed up? Perhaps whatever nucleus is enabling this move has some kind of epilepsy during migraines.

Notes that produced the idea:

Traveling waves in the prefrontal cortex during working memory

During baseline conditions, waves flowed bidirectionally along a specific axis of orientation. Waves in different frequency bands could travel in different directions. During task performance, there was an increase in waves in one direction over the other, especially in the beta band.

One fragment of mechanism that comes to mind would be Attenuation. This forces ensembles to 'move' around. Together with a topology (or 'geometry') in the network where closer neurons are connected, you get traveling amebas of activity.

This doesn't explain why the activity is regular in its direction during working memory vs. nonworking memory conditions.

It would fit with the requirements of working memory that we have high attenuation and perhaps low or no plasticity. Because you don't want your network to be permanently shaped by the working memory items.

But what is the implementation of 'holding in mind' and 'erasing'?

Perhaps you can re-ignite the same ensembles temporarily, presumably from Thalamic inputs. (This would fit experimental outcomes where whatever thalamus goes to PFC is essential for working memory). (this is an easy prediction to make anyway because of M. Sherman's driving inputs).

Erasing is then terminating the 'hold in mind' process.

  • You could do that randomly every say 500 ms
  • Then somebody else in the network needs to re-establish a 'holding in mind' if they still care
  • You can implement a 'tiring' mode to the 'holding in mind' neurons, so if the same neurons fire for x amount of seconds, it becomes harder and harder for the holding in mind ensembles to stay active. (i.e. the ensembles would need more and more support from the network).
  • This has certain biological reasoning appeals, too. Because you don't want the animal to be stuck with a single idea. You only want single ideas if they are worth it.

Here is a small twist:

What if working memory is implemented by making a region that is simply ridiculously hard to spread into, i.e. ensembles need a lot of support from the network to spread into it? (perhaps simply with low excitability, but why traveling waves?).

You automatically force the memes that want (and can) spread into there to be very competent in gathering the forces, moving the brain into a state where they are supported - sort of globally.

Not because the PFC is such a great region. But because it a region. And you always will get memes that use a region. Because the competitors that would would outdo them. Conversely, you only ever care about a region because you have competitors that would get support from that region.

So it is the tip of the iceberg creme de la creme of memes that can spread into working memory.

They must be amazingly useful buttons to the cognitive user, setting large cascades of effects in the brain in motion, that will give support from much of the network.

Perhaps at the level of working memory, we start having a little bit of leaky abstraction, we must 'hold in mind' a piece of info rather clumsily. This is probably a feature, not a bug, it says that those memes need to be implicated with the user intent and so forth. Which makes this even harder.

[ The subtext here includes Dennetts 'fame in the brain', subliminal vs. distributedly supported and "global workspace" by Baars ]

  • I am assuming that Thalamus (MD?) -> PFC is necessary for working memory (i.e. activity in PFC). This comes from M. Sherman's driving inputs.
  • People with schizophrenia have smaller Thalami and are less good at working memory tasks55
  • Let's roll with that tip of the iceberg gathering the forces memes idea:
  • Then perhaps, PFCs function as being a 'hard to spread into' region is diminished, and all kinds of memes spread into it.
  • Perhaps this would make such patients more distractable and so forth, where random ideas manage to go into working memory, but then such working memory disruptor memes are something more like local winners. Not like the usual strong iceberg memes with their 'global' support.

Absence Detectors, Counterfactuals, Imagination States

In some cases there may be a stellate cell, a so-called interneuron, interposed between the sensory fiber and the pyramidal cell, perhaps for the purpose of switching an excitatory sig- nal to an inhibitory one. This is in accordance with the observation that small stellate cells populate particularly the sensory areas of the cortex and are especially frequent at the layer of the cortex, the so- called fourth layer, where the thalamic afferent fibers terminate.

Braitenberg 1977.


Aaron Sloman mentionend:


Necessary Connection

Possibility Spaces

joy lines

Basic mechanisms for rewarding memes/cell assemblies:

  1. Make them active longer (via plasticity they will entrench in your network and will stay around).
  2. Make them active at a higher frequency (same thing, plasticity rules).

| X            X           | predictor / thinking goo
| |     X      |           |
  |     |      |
--+-----+------+-----------------X-------------   j-lines
---------------------------------X-------------   [joy maker nucleus]
                               Darwinian approval [ yes / now]

Here is a simple reward scheme you could make for cell assemblies.

  1. Make some random wires through the thinking goo, call them j-lines for "joy lines".
  2. Joy maker nucleus selects a random subset of j-lines, call it j-mix, and activates. This will morph the memetic landscape of the thinking goo. Intuitively - if there are 2 roughly equally likely cell assemblies F and K, then if there is a random j-line going though K, but not F, it will be more stable and win.
  3. Now you can run your usual cognition machine (for instance in the simplest case a sensor-predictor loop).
  4. From a Darwinian wire criterion you get 'worked well' or 'not good'.
  5. If good, take the current j-mix and shoot activity up the wires. Presumably, you will reward all the cell assemblies still in short-term memory. (I.e. the cell assemblies your j-mix is activating right now will stay around more, because of our plasticity rules). Perhaps you could also feed the situation / sensory inputs that lead up to the approval, to get a richer memetic network. -> This is of course allowed to become arbitrarily complicated. -> Eventually, the memes in the thinking goo will become capable enough so they try to hack the Darwinian appproval for joy. -> That's a fundamental problem a meme engine has to deal with.

More generically, I call such a mechanism perspective-lines. Because of the way they change the memetic landscape (dynamically). This is almost the same as the so-called 'attention' of current ANNs. Abstractly, it is changing a meaning-landscape into a different shape. 'Perspective mechanism' would have been the less confusing label. Whatever.

A toy idea for a meme-pruning algorithm / A Candidate Dream Mechanism

This is probably wrong, maybe too complicated.

Consider something very similar to the above:

| X            X           | predictor / thinking goo
| |     X      |           |
  |     |      |
--+-----+------+-------------------------------     perspective-lines
------------------------------------------------    [ dream-nucleus ]

Let's say the system goes into dream mode, where it doesn't need to perform prediction/perception on the fly all the time.

There are many things you might be doing during dreaming. It is a basic move of explaining biological intelligence to consider that 1/3 of everything this thing does is sleep - so you can put as many mechanisms into a sleep mode as you put into a performing mode.

Here is a meme-pruning algorithm (just an idea):

The main idea is that you can try a subset of your thinking goo56. If it manages to predict the situation, then reward the smallest subset that manages to predict - automatically punishing selfish memes that don't contribute to the prediction/ success.

  1. In dream mode (no or low sensory inputs, no pressure to perform cognition).
  2. Select a random subset of perspective-lines p-mix, activate that subset.
  3. Select a known working sensor->predition pair (allowed to be more complicated).
  4. Feed the sensor states into thinking goo. (For the sake of argument, your p-mix and the sensor state you feed are the only sources of activation in the system now).
  5. Run your usual cognition,

    If the predictor comes with acceptable predictions, then reward the current p-mix. (Shoot activation up the wires). This way you punish all selfish memes that didn't contribute to the prediction capability of the thinking-goo. And you will get the kinds of memes that predict real things about the world.

  6. If the predictor is not capable of having a satisfactory prediction for your input states, either select a new p-mix, or augment the the current p-mix to p-mix-2 by adding more p-lines. Then continue with 5 until you get results.

Open questions:

  • Would you feed high-level prediction states and see what the continuations are?
  • Or would you be able to feed sensor data? It doesn't look like thalamic input nuclei like LGN are on during dreaming, speaking against this idea57


  • This fits well with the anecdotal and experimental observation that dreams from early and later in the night have a different character.
  • Incidentally, it seems like the earlier dreams have less cognition 'on'.
  • The dreams later in the night then, have a self and language and situations and so forth. This fits with the idea that whatever is happening, maybe it is trying out subsets of cognition.

Maybe this can be simplified into 'try out subsets of your cognition' and simply have the rest of the machine reward memes like usual. You get the same benefit - rewarding 'sufficient' memes, pruning away the free-rider useless memes.

Notes on The Agendas of Memes

Political memes

Large memes, very smart memes, that know how to use much of the computer to be successful - to be thought again. They spread into mid-term memory, they hijack where you look, they make you feel this or that way and so forth.

They represent 1:1 the neuroscience of 'the belief immune system' [Didn't find the research links right now, at least one researcher is using such terminology, quite sure].

Considering the logic of our inhibition model, where 1 cell assembly 'wins out':

         \             /
          \     A     /                   meaning-level
           \         /
            \       / <----->   B
             \     /   inibit
              \   /
                X                        sensor-level

A - Stable cell assembly

A Stands in competition with meme B:

   meme A                 meme B (competitor)

+-----------+     alternatives                       high meaning level
|           |   |-----------|
|       A   |
|           |
|     +-----+--+            +- -- - -+               medium meaning level
|  |  |     |  |                     |
+--+--+-----+  | |-------|  |
   |  |        |               b     |
   |  +--------+            +-- -- --+               sensor level
   |                +-----+
   +------------->  | c   |------|
    activate        +-----+

A good meme

  1. Activates its activators, that is how it is constructed in the first place.
  2. Activate the alternatives of competitor memes, on every level of meaning.

Meme A here will activate c, even though c might not have to do anything with A at first glance. This will happen if c is an alternative to whatever sub-cell assemblies would support B, an alternative to A.

If meme A doesn't make this move, then it is just not a good meme and is discarded.

The dress:


Figure 12: The dress blueblackwhitegold

If you don't know it, it either appears blue gold or blue black. Funny, it used to be white-gold for me back when I first saw it. Now it is blue black. And I cannot flip into seeing it white-gold, either.

I think what happens here is there is a further layer at play:

  1. sensor level (looks blue)
  2. meaning level (I see a blue dress)
  3. psychology level (I understand the scene)

3 Says that you don't randomly flip something fundamental as the color of a dress. It is a larger situation, that a higher order meme is creating for the rest of the meme-machine.

You successfully predict yourself to competently parse the scene without going crazy. And only the memes that are well-supported by the situation will survive.

Baldwian Spaces and Darwinian Wires

The general view on what the cortex is doing for an animal:

It represents an ever wider analysis of the situation and affordances; This is only useful when it is about survival.

In broad terms, there are 2 elements in the system:

|                                  |
|  ^                               | thinking goo (cortex)
+--+-------------------------------+ (makes memes possible)
   |                                 (large network, many possible meanings)
   |                                 (made from stupid stuff)
 --------------------  [Darwinian Wires]
 organize, drive, extinguish activity - shape the memetic environment
 reward the thinking-goo when it is useful for survival

                     (not many possible meanings, pre-allocated meanings)
                     (precise and competent, implements math when it needs to)
                     (has the intelligence of evolution inside it, is highly engineered)

The thinking goo is the ultimate Baldwin space, that must be shaped in a late binding.

This doesn't have sharp delineation, like much in biology. So when you look at the cortex you still see this or that wire and this or that architecture shaped by Darwin's hand. We can expect that wherever evolution could help out, for instance with development plans, it did so.

Still, I submit the fundamental dynamic nature of such a navigating device lies at hand.

The basic problem the 'rest of the brain' has to solve is to make the thinking goo be about survival and success.

My view is that there are 'smart Darwinian wires' that come from evolution, that are designed to make a thinking-goo into a good thinking-goo. That probably means things like rewarding and selecting memes, making memetic drivers in the system for success somehow and so forth.

One job of the computational cybernetician then is to describe and fabricate such memetic landscape machine tricks on one hand. (Similar to the ones that are implemented by the Darwinian Wires in the brain).

And on the other hand to make a computational model of a thinking-goo. It is not required, possible or desirable to get all the details exactly right the way the human brain does. But it is very useful to have a working implementation, if just for the benefit of knowing that it can work.

Note also that we don't care about the contents of the thinking-goo per se. We care about the kinds of content that it should support, so we can build a resourceful implementation of thinking-goo. (This is roughly what it means to have a dynamic computer program. 'Data is the ultimate late binding'. And so with the brain, the network and its content are the ultimate late binding).

In a further step, we can provide machine intelligence with the power of the computer. This is an edge that is not available to biological intelligences, that don't have the benefit of running on a computer. Such things are on the topic of intelligence mechanisms, which I would say are one level above the main concerns of a model of cognition. Roughly speaking, intelligence is the realm of stuff that a cognitive user knows herself; While cognition is the realm of stuff that the cognitive user has no idea how they work.

The Puzzle of The Hemispheres

I don't have answers yet.

It fell out of favor for a while to talk about lateralization, because popular science was taking it as a beautiful story of 'your other side knows different things' Bla Bla.

I think it is now that more nuance comes back into our conceptualizations, And there is a rich puzzle of of lateralization after all.

Givens / Reasoning:

Split brain patients [Grazzaniga].

  • It seems like the hemispheres only have some trickles of information flow left.
  • Perhaps with assembly calculus we can make a computational definition of split/merged cognition machines:

that is the extent of the (theoretical) maximal cell assembly of the network. If it spans the 2 hemispheres, we can roughly say there is 1 cognition software running. If it doesn't, then it is 2.

  • It is funny that 'alien hand syndrome' is conceptualized as a motor-system problem so to speak.

When in reality the reason for this is crazily profound.

  • I think it is almost obvious that there are 2 'selfs' and so forth in a split-brain patient.

(everything speaks for it, nothing speaks against it).

  • Split-brain is interesting, but it tells us how the system looks in an unnatural configuration.

What we learn from considering split-brain and lateral neglect and so forth is like shadows, which can tell us about the nature of the intact arrangement.

Thalamus is ipsilateral

[Lecture 8. The Thalamus Structure, Function and Dysfunction]

  • The thalamus is an ipsilateral concept. I.e. it looks like the input organization is separate.
  • If the Thalamus Matrix system provides context (perspective) to the Cortex, this organization would roughly correspond to some kind of '2 perspectives mechanism'.
  • Cortex is connected via Corpus Callosum (CC), at least roughly to the same region on the other side.

    CC connects functionally equivalent cortical areas in each hemisphere with the exception of primary sensory and motor cortices, which are sparsely connected transcallosally (Innocenti et al. 1995). The common understanding is that the sensorimotor cortices (S1, M1 and premotor cortex) are connected monosynaptically via the CC only for areas representing the body midline area (Innocenti et al. 1995).

    Functional anatomy of interhemispheric cortical connections in the human brain

    The prefrontal and temporoparietal visual areas are connected interhemispherically by poorly myelinated, small-diameter, slow-conducting fibres (Lamantia & Rakic, 1990; Aboitiz et al. 1992).

    (from here again)

    This is a challenge to my 'stupid wires' model. If we assume the cortex implements an ideal assembly calculus, there would not be a reason to have slow-conducting fibers.

    I would postulate that the presence of slow-conducting fibers only makes sense if the information processing has something to do with time delays. The computational reasoning for this is clear: There must be an evolutionary driver for making timeless calculations as fast as possible. (per definition, a timeless calculation doesn't concern itself with the time passing. See functional programming, What is a calculation?). The only reason to put delays into your information processing is that your computation has something to do with time.

    The other alternative is that somehow those fibers are on their evolutionary way out. Or that a slow conduction was sufficient, so there was no driver for myelination. (This would be rather surprising, why myelinate all the fibers in Cortex but then leave some out?).

  • With a Hebbian substrate and a thalamus, you could do the following:
  • Connect roughly to the homotopic side, at the area that receives the analogous thalamic relay inputs, with random wires:

          |   |                         |   |
      +---+---+-+                  +----+---+-+
      |   |   | |                  |    |   | |
      |   | ^   |                  |    | ^ | |
      |     |   |                  |      | | |
      +-----+---+                  +------+---+
            |  C                          |   C'
            |                             |
            |                             |
            |                             |
      +-----+--+                    +-----+--+
      |        |TH1                 |        | TH1'
      +--------+                    +--------+
      |        |                    |        |
      +--------+                    +--------+

TH1 - Thalamic relay nucleus, side 1
TH1' - Analogous relay nucleus, other side
C - Cortical area, receiving inputs from TH1
C' - Contralateral cortical area with inputs from TH1'
C<->C' - A few random wires 'interhemispheric and homotopical'.

If this is the arrangement, you will observe that via Hebbian Plasticity, we will select wires that find some kind of symmetry in the information representation between the 2 thalamic relay inputs. Braitenberg 1977 mentions such symmetry detectors.

Simply imagine that from many random wires, the ones that connect to 2 pieces of network that is active on both sides at the same time, will be on. From then on, those symmetry wires will pattern completely to the symmetrical activation on both sides [Cell Assembly literature].

  • What is symmetry in derived meaning spaces?
  • The hunch is somehow that this would be a useful arrangement to 'share information' between hemispheres. (which must be the function of CC either way).
  • The 2 sides are functionally connected (see literature on functional connectivity networks).
  • Consider that organizations would have been possible:
  • Random wires between cortex hemispheres
  • Wire Thalamus -> contralateral hemisphere
  • Wire contralateral hemisphere -> Thalamus
  • Or wire contralateral hemisphere -> other subcortical structures.
  • The corpus callosum is curiously absent at v1. Exactly the area where we would have expected to gain use from 'symmetry detectors' [Braitenberg 1977]. (all primary sensory and motor cortices are sparsely connected transcallosally).
  • In general, this arrangement seems to mix aspects of converging information flow with separate information flow, but why?
  • Why decusssions?
  • Decussation seems to be a neocortical concept, but not enforced (audition).
  • What is the nature of hemispatial neglect?
  • What is the other 'Wernicke' doing? There is some literature on this. Iain McGilchrist mentions that the right side has a different vocabulary and so forth; So perhaps it is not true that only the left is doing language.
  • If the hemispheres implement some kind of '2 mutually complementary perspectives' mechanism, then what is the computational-level explanation of how and why?
  • Perhaps the best way to make neo-neo cortex is to make yet another artificial sphere, trihemispheric, or quadrihemispheric.
  • It would mean we have strong notions of what the basic circuits are, then we can simulate another symmetrical cortico-cortical connected element.

Smart Wires Vs. Random Wires

Turing mentioned this concept already in his 1948 'Intelligent Machinery'.

This picture of the cortex as an unorganized machine is very satisfactory from the point of view of evolution and genetics. It clearly would not require any very complex system of genes to produce something like [… a kind of] unorganized machine. This should be much easier than the production of such things as the respiratory center. This might suggest that intelligent races could be produced comparatively easily. I think this is wrong because the possession of a human cortex (say) would be virtually useless if no attempt was made to organize it. Thus if a wolf by a mutation acquired a human cortex there is little reason to believe that he would have any selective advantage. If however the mutation occurred in a milieu where speech had developed (parrot-like wolves), and if the mutation by chance had well permeated a small community, then some selective advantage might be felt.

Darwin did not need to be smart in order to make a useful thinking substrate.

We know from insect neuronal plans and so forth very detailed math and models what some circuits are doing, for instance fly motion.

The 'smart wires world', Vehicle 3. Those are wires that are shaped by evolution, therefore the meaning is allowed to be very precise. It means 'you move when x is true' and so forth. It also implements some complicated math: subtractive inhibition, divisive inhibition and so forth. If one of these neurons or wires is gone, that is a big deal.

The smart wires world stands in contrast to the ultimate Baldwian substrate (cortex). Welcome to a world where the wires are stupid and don't matter, but from the many stupid ones you get something useful.

I submit that the cortex looks like it implements some kind of high-dimensional computing [Kanerva].58 Therefore in some ways, everything is still the same, evolution makes wires that make computation. However since the computation is of a certain flair, there are completely different requirements for the implementation.

The wires in thinking goo (high dimensional computing device) are allowed to be random they are only useful because they are random.

As a biologist, I had the feature detectors in mind and thought about those neurons being connected. (wires, units, wires, wires). And of course, then you say it is so dauntingly big. This has its culmination in neuroscientists proclaiming it will take hundreds of years to understand it. This is when you look at the trees but you should have been looking at the forest.

Because of the nature of high-dimensional computing, you can see the thinking-goo as a kind of information representation substance. A piece of cheese has temperature, you don't need to worry about the atoms. So too, a piece of the cortex has information states, you don't need to worry about the neurons.

How this works is a computer science topic. But it works. - Santhosh Vempala explaining assembly calculus.

This is the source of confusion when you hear that this or that aspect of the cortex is innate. How does that fit with the data of the ferret re-wiring experiment? You get orientation columns in the auditory cortex.

The confusion comes from the fact that both things are true. The substrate is random and it starts representing its inputs more or less immediately.59

It is the ultimate dynamic late-binding substrate (to use software engineering terms).

Perhaps the reason the cortex evolved from a tertiary olfaction ganglion is not so surprising in light of the high dimensionality. Perhaps it is the nature of information processing of olfaction, where you have 50 sensors, not 3 like in vision. Perhaps this is the fundamental nudge towards the concept of 'mixing information' that is perhaps the most primordial aspect of the cortex then.

Similarly, you look at neurons firing and you say here is a Jennifer Aniston neuron. The network seems so smart, but it cannot work with single neurons either because:

  1. If you put concepts into single neurons, you don't have enough neurons for the amount of concepts.
  2. Neurons die, and that does not seem to disturb the system much.

The high-dimensionality computation perspective flip is satisfying:

Concepts and symbols can be represented by randomly allocated subsets of neurons, that temporarily ignite each other as cell assemblies. It doesn't need to be the same neurons tomorrow and it probably isn't the exact neurons tomorrow.

The neurons are important and unimportant at the same time. - That sounds great to say :P

Machine Intelligence Ideas

Additionally, we can have a sideways layer that is the engineering and analysis of how you give such biological intelligence, when runs on an electronic computer, some of the power of the computer it runs on. I label this the field of machine intelligence.

Machine intelligence ideas:

  • A machine intelligence should be able to have a seamless interface to the computer it runs on
  • Observe that a Lisp REPL is an arbitrarily powerful interface to a computer (). it runs a universal programming language, also it is the simplest possible kind of such an interface, basically definitional. (Since if you find a simpler Lisp, you change what Lisp is, you don't invent a new Lisp).
  • There are 2 kinds of interfaces: To an inferior Lisp (perhaps remote, distributed, etc.) and to the own Lisp, which is your interface to the program that is making your cognition. (This sounds tantalizingly meta and I suspect Minsky would have approved).
  • In both cases, you need to solve resource constraints and so forth (evaluating a non-halting program should not make the intelligence stuck and so forth). Of course, this could be solved by using stack-based language instead and so forth.
  • Idea1: On the level of the neuron, add effectors and sensors for an eval and apply
  • Idea 2: Introduce the intelligence after a developmental phase to its computer interface. In this scheme, we would introduce a kind of toddler to the computer it runs on at some point.
  • Idea 3: Same as 1, but think cybernetically about what kinds of modules or arrangements could be made. For instance, we could be inspired by the Flas drive implementation and make an eval-box implementation or something. (note that even without this notion we probably already made a flash drive and network that can go to disk, simply for software development reasons).
  • It sounds useful that a machine intelligence would be able to substitute parts of its cognition machine with scripts it creates,
  • It might create higher order abstractions like the ones that make up the affordances of our magical interface, only now the producer level of the interface also produces code to solve situations, thus, machine intelligence does not need to calculate x + y like humans, but it would have the cognitive affordance to calculate with computer speed. I.e. the way we can bring a banana to mind, the machine intelligence would be able to calculate.

Remembering In Roughly Two Steps


  1. Long-term memory, mid-term memory, and short-term memory are different systems [cognitive neuroscience]
  2. Consider patient HM. Without hippocampus (+ close by structures) mid-term memory is gone, with anterograde amnesia, but long-term memories (up to a few years before surgery) are intact. Short-term memory is intact, too.

Step 0: open one mind to receiving mid-term memories. I.e. activate the situation of I remember. I.e. activate cell assemblies that support the idea of remembering vaguely.

Step 1: Reinstantiate cortex activity more or less globally*, basically giving some context. You need a hippocampus, or at least a related temporal lobe to pull this off [see patients HM and EP].

Let's speculate: what comes from the hippocampus is quite course-grained, then:

Step 2: Fill the low-detail information 'scaffold' that came back from the hippocampus with life. Do this by using the cortex, its thinking, its imagination and so forth to mentally enact the memory. It's my hunch that this process even makes you voice act the people in your memories.

I think all steps 0-2 are neuroscientifically plausible, and fit with a high-dimensional computing framework like the cell assemblies.

Remembering in this notion is separated from the memory. Note that this applies only to mid-term memory. Other kinds of memory are virtually guaranteed to have different kinds of mechanisms.

Remembering is the process that creates cognition states/ imagination states. And presumably, that happens in Cortex. The Memory is allowed to be stored in the hippocampus, frozen for all we care right now.

Déjà vu

Doesn't it feel like you are 'inside' a memory during a strong déjà vu?

My idea is that whatever is saying 'Here comes a memory, please fill in the blanks, Mr. Cortex (step 0)' is accidentally on. And the system falls into an interpretation that says I am enacting right now a memory.

I.e. Déjà vu is remembering being on, without a memory coming in, but the interpretation of cortex is for a moment that the sensor input is the memory. And the rememberer resources try to fill in the blanks.

Déjà vu is the interpretation of being inside a movie of one's mind a little bit.

Perhaps this is a hint about the circuitry. The relative rarity of déjà vu might give a hint that it is usually not the same input stream circuitry where memory 'context' comes from and where sensory input comes from. But it is also not impossible to interpret one place as the other.

This of course all fits with the idea that Cortex is so vast. That Cortex has a tiny amount of sensor inputs compared to its interpretation. So it is always the interpretation that makes cognition, perception, imagination, memory, and dreaming … They are created by the vast interpretations of the Cortex and its nuclei.

Of course, step 2 depends on how fine-grained the memory coming from the hippocampus is. Presumably ruminating on an emotionally salient event, perhaps every few minutes, such memory would become highly detailed.

A strong déjà vu is almost existential, it is a very intense 'mind-moment'. Perhaps only psychedelics and things like out-of-body experiences are stronger. For this reason, I am very skeptical about inducing artificial déjà vu in the lab.

I think what you measure is something like vague familiarity or something. At best, the notion is 'very low-level déjà vu' and at worst, it is utterly confused and doesn't have much to do with an actual déjà vu.


*) perhaps via the thalamic matrix system


This preceeds The Slow Place: Evolutionary Drivers For Mid-Term Memory I changed my mind and think that it would perhaps make more sense when déjà vu is triggered by out of context query result information states.

More Rants On Neuroscience And Computation

The brain's function is to do computation - well what computation? It is like saying The car is doing acceleration, or the tree leaves are doing biochemistry.

In my view, it is in terms of software that you understand a computation.

When somebody looks at some neurons and decodes information out of the activity this way I can tell whether the animal is thinking left or right. Without a software-level computational theory of the brain, you don't know whether you are looking at a conveyor belt of information, some intermediate outputs of a calculation, or a representation of the outcome of some calculation. If you can decode information with great quality, it is even more likely that you are looking at some kind of message sender, where the message represents the outcome of a calculation.

I simply ask myself would you be able to distinguish a conveyor belt from a machine?. If you can't, then your model and theory are not computational, but some kind of substrate analysis (that is not useless, but it is not what will explain brain-software).

These kinds of nuances are lost, if computation is synonymous with the function of the brain. The only way the structure and function of the brain will be understood is in terms of the mechanisms of this computation, not the label computation.

My slogan way of expressing this problem is you can replace computation with wizardry in a neuroscience textbook and you get the same information content from that book.

Clearly, the function of the cerebellum is to make the wizardry necessary for error-correcting motor movements. It has these and such sensorimotor afferents in order to achieve its wizardry.

It's like saying the water flows through the xylem to the leaves, in order for the leaves to do it's biochemistry. This is ok and useful, but it is not a biochemical plane of explanation.

They should just say magic or wizardry instead. That removes the confusion from the picture.

I mean it is ok if you do brain-science and substrate analysis and talk about what kinds of things are connected to where and so forth.

The Brian as an Activity Management Device

From this, I have now grown a view of brain functioning that centers around the idea that the brain is an 'activity management device'.

You gate the activity coming in (Thalamus), you store some activity at a slow pace (Hippocampus), and you ignite activity internally (striatum?). And things like this.

Baldwian spaces make memetic substrates.

I want to move up from the neurons, like seeing the forest instead of the trees. Just like a stone can have a temperature, neuronal substances can represent information. So you can sort of imagine a magic stone substance that can be loaded up with information.

In general, it looks a bit like the cortex is a 'substrate that supports memes'. It can represent `vast` spaces of meaning, sort of everything possible.

|                                  |
|  ^                               | cortex 1 ('everything possible machine')
   | [brain nuclei]
 organize, drive, extinguish activity - shape the memetic environment

From AI we know that 'everything possible' is not useful unless I have something that selects and explores this space.

My main view then of what the rest of the brain is doing is that it is creating a memetic environment, in which only some memes (presumably useful for biological success), survive.

So you get this view of the brain shaping the memetic landscapes, to say what kinds of memes are possible. And the cortex fills the activation spaces60

Further speculation on the circuits of a holographic encoding

Statistical considerations on the cerebrum / Holographic encoding?

That is from the 1977 Braitenberg Cortes paper:

  1. Distinguish an A-system of cortico-cortical connections from a B-system.
  2. A-system is the apical dendrites of pyramidal cells in upper layers [1 and 2?], that listen presumably to everything else in the brain.
  3. B-systems are local connections, the basal dendrites of pyramidal cells,
  4. With some careful reasoning you conclude that the connectivity in the B-system is ~0.1. (you are connected to 10% of the local network).
  5. From statistical reasoning, it looks like you can make the square root of N (number of neurons in cerebrum), 'compartments'. Each compartment listens to everybody else. It is intriguing that size and count overlap with that of a 'cortical column'. Further, if every compartment listens to all other compartments, then probably the activity in a compartment should be correlated; Intriguingly, the orientation columns we know look like they have this property.

thoughts ->

(The software engineering approach dictates that we let our imagination go wild sometimes. Let's assume for a moment you would get this or that kind of computational properties of the system, then …).

From this then you would get the hypothesis that each compartment (map to cortical columns) receives a holographic encoding from the rest of the system. So each part gets a representation of the whole via the A-System, the network would not mean we throw away the messiness of the connections of the network. It would still have the cell asssembly properties, Hebbian plasticity and so forth. So each compartment would learn to represent different things, depending on the overall context. But - you could store and manipulate the overall context easily and this would be useful. I.e. that is the state of which compartments are active.

This is very intriguing because it would be a bridge to hyperdimensional computing. [HD/VSA]. In this hypothesis, you could store an encoding of the overall state of the cortex in roughly 105 neurons. Btw the numbers are very large. In the literature, the 'blessing of dimensionality' comes at 104.

We would be able to do HD computing, the fundamental computations are trivial to implement. Simply align 2 input vectors and do either addition or multiplication. There are many ways we could imagine a hypothetical circuit or nucleus, or the cortex itself to do such math.

From HD computing you get wonderful properties, in meaning-spaces. (See HD/VSA).

The fundamental operations are bind and bundle.

With bind you find the contrast of 2 points in meaning space, it corresponds to the surface of a rectangle, made from width and height.

With the bundle, you find the 'between' between 2 points in meaning space. You might say the essence.

The 'perspective', 'zoom', and 'inside perspective' move:

  1. Find the essence of a cloud of meanings (with bundle)
  2. Use this essence as a perspective point. I.e. you bind all the points, highlighting the differences between the points.

This would be a beautiful on-the-fly perspective mechanism, blowing up the meaning spaces around a point that matters.61

You would be able to temporarily and totally dynamically make the brain into a brain that is good at representing 'this kind of meaning space'. And that is allowed to be completely arbitrary and so forth.

This fits me with the observation that a different perspective completely changes the kinds of thoughts we think.

Via a zoom like this, we could move around in meaning-space. From outside, there is a cloud of meaning points that all look similarly far apart, after a zoom, it would be like sitting at the center of this meaning cloud. And seeing the differences between the meanings.

The notion of contrast and perspective is one of the challenges to purely connectionist models. At first glance, I would think everything is mud if everything is sort of associated with everything else.

If I would know that thee TRN was segmented, then I would go like holy shit that fits.

This would fit with a circuit/mechanism where the thalamus would have slaps of roughly 1/2 * 105 neurons, each making wires to the cortex, going through a segment of TRN, with 1/2 * 105 wires going through it. Half, because the thalamus is a unilateral concept (1 hemisphere).

This would fit, if you would want to gate 1 holographic encoding at a time. Because at first glance it would be a bit strange to ever separate one encoding out. (This consideration depends on the idea that TRN is a kind of activation gating 'attention' mask around the thalamus).

This is just 1 of many things that could be true though. Perhaps the Thalamus isn't encoding a context at all. And if it does, then maybe it is useful to have 'partial attention', after all. (The brain would be more complicated).

If the thalamus would represent the cortex context in this way, you might expect that there is a hippocampus -> [thalamus-HD-encoder-nucleus] -> cortex circuit; And the basic midterm memory retrieval mechanism would be to reinstantiate the cortex context. (See below musings on hippocampus, future).

'thalamus-HD-encoder-nucleus' should be roughly 1/2 * 105 neurons large. It would be one of the Anterior nuclei, then. If the simple 1:1 reasoning of 'what is connected to what' can be applied here. (Which is not clear).

Or, there are slaps of 1/2 * 105 neurons in such a nucleus. Where 1 slap would correspond to 1 context context.

Intuitively, this might be incredibly sparse somehow. Let's say that a tiny subset of compartments active is already meaningful. Speaking against the idea that you would need to encode all 1/2 * 105 neurons.

Cerebellum doesn't matter for HD computing

I have seen this being mentioned the cerebellum might be doing HD computing. I am not sure entirely what the reasoning here is, here are some thoughts why this doesn't make much sense:

  1. The cerebellum from all angles doesn't look like it is part of the most interesting aspects of cognition. In particular, it looks like it is doing fine-grained timing calculations, required for movement (Braitenberg On The Texture of Brains). At the very least you have to concede that the current literature conceptualizes it as some kind of movement computation-making computer.
  2. Perhaps the regular structure of the cerebellar cortex invites a hypothesis about it doing hypervector math. The issue here is that the parallel fibers are thin. They look like they have an evolutionary pressure to be slow (see Braitenberg). This doesn't bode well for a model of the cerebellum where you want to do timeless calculations involving the parallel fibers. (Every fiber in the cortex that does computation seems to have a driver to be fast, see Pyramidal cell activity above).
  3. The whole reason to consider HD computing in the first place is to explain challenging modes of thinking: succession, causality, directional relationships, contrast, perspective, essence representations (symbol representation) and so forth. It does not make sense then to point to the cerebellum to implement this, since we know neuroanatomically, that the cerebellum is not essential for the most interesting cognitive processes (language, thought and so forth).

Olfaction is a stepping stone into high dimensionality

The alternative:

What does make sense is to consider olfaction and the computations it implements.

Vempala is mentioning this here as a stepping stone to think about 'assembly calculus'.

Perhaps it is simply the nature of making information processing on olfaction sensors that simply gives the system a nudge to high dimensionality. This then would give us the evolutionary driver for the cortex, and its HD computing implementation.

  1. Take some tertiary olfactory ganglion
  2. It represents olfaction information states, high dimensions
  3. Make a few neurons sluggish, or make some neurons go back and forth, or duplicate the whole array
  4. Now the input to the next time step of the computation is allowed to be not only the sensors but the current 'situation' that the network represents.
  5. From biological/cybernetic reasoning, we see the evolutionary driver for analyzing bigger and more fine-grained situations.

The evolutionary story of the cortex would then be summarized as sniffing with more and more context.

Cortex, evolution

It evolved as a tertiary olfaction ganglion [Braitenberg, 1977].

One might consider the perspective that all of cognition is ever more fine-grained and ever more abstracted sniffing.

There is more time between me and my problem in the past.

This requires

  1. Some memory capacity
  2. Sniffing out the essentials of a situation
   [ ate toxin ]  <--   me
|                 time            |
|                                 |
     [ relavant-sitution ]  <--+  situation sniffer

There is an obvious evolutionary driver to increase my sniffing capacity here.

 me -->  [ goal ]

 me -->  [ problem ]

Evolutionary driver to make the time arrow longer.

If I can sniff out the relevant situations in advance, that is useful. If I can put more time between myself and my goals and problems, that is useful.

There is a use for a situation sniffer, with an evolutionary driver. Towards being able to think further ahead, make plans, have a feel for a situation, have a feel for a situation in longer timescales etc. etc.

Consider a tertiary olfaction ganglion. It might start evolving some more neuronal tissue, to be able to keep more memory states around, and to then be able to have more derived states. For instance in higher abstraction hierarchies or higher temporal domains.

One of the things that a theory of the cortex is doing is talking about how it is useful to have more neuronal tissue.

If I can sniff out a problem before I have it, I am putting time between me and my problem.

If I encounter a problem and I can keep track of what happened in the past, I can put more time between me and the problem.

When we go into the cognitive realm (like tree shrews, primates, and dolphins?…) there is enough the machinery of the system so that we can talk not only about the time between and my problem but inferential steps between me and my problem.


Thank for the hypervectors, (Carin Meier, Kanerva and friends). The hypervisors are such a beautiful building material. Able to mix and morph and be vague and about themselves and have directions. All kinds of things that I feel are important for building material of cognition.

The bundle is like the `mnemotrix`.

Rotate + bundle is like `ergotrix`.

Bind is maybe like a synchronous activity. It is I feel freshly derived signals from 2 signals.

Of course, since the brain is a messy information mixer, firstly it does all of them at the same time. (Except preserving the bind for important information?). And secondly doing them all at the same time is good design.

Cell assemblies and hypervectors

Hunch: if you have static, timeless cell assemblies, their function can be abstracted with a hypervector.

So I would like an assembly->hypvervector projection.

Hunch: maybe some frequencies of activation are there to hold ideas in the mind, without modifying them so much. There is certainly use in the system for timeless ideas. (But the brain needs to make activity back and forth, so it can only slow down time at best).

But we, thinking about the abstract machine that we can run on a computer, have the freedom to freeze ideas.

… ideas:

To find a hypervector representation of a cell assembly A (the symbol you might say):

  1. Find a kernel of neurons that can re-ignite a cell assembly A (meaning condenser…?)
  2. Look at an arbitrary abstract space of neuronal area (with geometry, you could imagine a straight line) (of size h-dim). You need to remember this space though, call it a hyper-line.
  3. Project into a hypervector (trivial), say which neurons are active 1, inactive -1: [ -1, 1, -1, … ]

hyper-line is in effect a projection map between hyper dimensions and cell assembly neurons.

(hyper-line neuron-idx) -> hyper-dimension

In order to go back from hypervector space to cell assemblies:

  1. Given a hyper vector and a hyper-line
  2. activate the neurons with a 1 in the hypervec, (optionally inhibit the ones with a -1).
  3. This re-ignites the cell assembly, or at least biases the system to do so.

So in order for this to work nicely, you can now do the math, depending on the dimensionality of the vectors you want to use. -> this is 1:1 to the space of neurons that map.

hm not sure yet, tried to have some ideas below. Doesn't flesh out yet:

I don't need to calculate the neuronal net at each time step if the function of the neuronal net was timeless.

Idea: from a neuronal area, you can make neuronal units a hyper-line mechanism.

It would be kind of cool if you would get this with pure assembly calculus.

This here is kinda of the first idea:

    nueronal area                  |
                         [ A ]     o ..
 +--------------------+            |
 |                    |            |
-+---X--------X----X--+------------o 1 hyper-line (part of the cell assembly A)
 |                    |            |
 |                    |            |
 |                    |            |
 +--------------------+            o 2
                                 hyper-hyper-line etc.

You can implement hyperline with cell assemblies/neurons, I am sure.

It seems tempting to attribute some of such function to the straight axonal collaterals of pyramidal cells through the b-system.

Say we have an assembly A active, now we can find a hyper-line or set of hyper-lines that are well suited to re-ignite the assembly A.

Simple Predictor Loop

See also

  ^     |
  |     |
  |     v
  |    p-state
  |      |
  |      +------[C] Compare the last prediction with the new input
  |      |
  +----sensor-state [inputs-brain]

Different design decisions can be made here.

In general, the idea would be that you keep around a representation of what your predictor [P] says, then you compare [C] the incoming sensory data with the prediction.

If the prediction is good, fine everything moves forward. (or decide here to reward [P]).

If the prediction is off, you can immediately make a surprise signal, and make the system be about the inputs.

[inputs-brain] is something that merely represents what the sensors say. Everything looks like the thalamus is the place where this happens in the brain.

Further, you can have evolutionarily drivers to make this the ultimate input nucleus, after all, you can copy all 'Input' machinery. But this time the input is some other pieces of the brain, internal inputs then.

You can hook that up to a Darwinian brain, too. Which makes value judgments on what kinds of inputs are interesting and so forth.

The basic move is to treat the predictor as a black box, you reward it if it does what you want. You, the smart Darwinian wires, never know how this thing works. But it helps with survival and procreation for some reason.

Cognition Level Mechanisms

As programmers, we have a strong urge to get all the abstraction levels straight.

The program runs on our brain, the thing that makes things like perception, memory, working memory, language, higher reasoning, social reasoning, feeling and so forth. Is a program that is implemented by the brain, but not in terms of neuronal activation or anything.

But in terms of its fundamental data structures and so forth.

One way to get the abstraction barrier between the 2 levels right is to have 2 implementations in mind.

I have so far 2 or 3 ideas of implementations, against which somebody might try to program a cognition machine.

One is cell assemblies, one is vector symbolic computing, and one is imaginary magic books in a vast library.

Consider motion detectors: They must represent their meaning in terms of the data structures of the cognition machine. They cannot be little stacks of motion movies, they must speak the language of the cognition states.

It's a bit like the transistors of your computer fire with electricity, yes, and you can imagine the memes flowing and so forth. But in order for you to be able to delete a file on the disk, it must be represented as a structure in the operating system.

Gregorian Cognition:62

There is maybe roughly 1 other abstraction level in a cognitive machine, that is when you start having a sophisticated user of the the rest of cognition. Making things like working memory, higher goals, structured thought and so forth. It is quite striking that the prefrontal cortex seems to be doing these higher executive functions, and that the pre-frontal is biggest in humans compared to other mammals.

This is then firmly the layer of abstraction that we know ourselves. I know how I use my working memory for instance by 'keeping the information in mind', for a while or by repeating a piece of language or visuals in the 'mental scratch pad'.63

These are internal muscles of a kind that allow me to use the rest of the cognition machine to bring to mind a piece of information etc.

This mental scratch pad is available to me as a user. What is not available is one level below - what I call the Cognition Level. That is how perception works, how memory works etc. I have no idea how it works. For instance, I just experience my visual field and that's it. There is approximately zero insight, by the user, into where this visual field comes from.

As an aside on AI history:

It is kinda of a cliche story of AI, that the people tried to put these highly derived symbol thought things into computers. The narrative then usually goes something like 'and then we figured out that you need something messier than symbols' or something.

I think the real story is quite a bit more nuanced, with the dynamic programming Lisp people talking about programs that modify themself, and build societies of small, contributing agents (who have competence without comprehension, see Minsky 2006).

Not to forget there is the whole branch of 'biological cybernetics' that could have been, but wasn't, the basis of AI and computer science. See Heinz Von Foerster etc. They built the first 'parallel computer' in the 70's or something; Completely different paradigm of computing based on McChulloch-Pitts neurons. If we had more of these ideas influencing computer science, who knows what kind of alternative thinking substances we would already have?



Braitenberg On the Texture of Brains -An Introduction to Neuroanatomy for the Cybernetically Minded, 1977


Rafael Yuste Lectures In Neuroscience, 2023


G. Palm Neural Assemblies: An Alternative Approach to Artificial Intelligence, (first edition: 1982, 2nd ed.: 2022)


Buzsáki, G. (2019). The Brain From Inside Out. New York: Oxford University Press.


Harold Abelson, Gerald Jay Sussman, Julie Sussman, Structure and Interpretation of Computer Programs, 1983


Turing 1948 'Intelligent Machinery'


Marvin Minsky, Seymour Papert The Society of Mind

Minsky, Marvin (1986). The Society of Mind. New York: Simon & Schuster. ISBN 0-671-60740-5.


The Beginning of Infinity David Deutsch (2011)


Surfaces and Essences: Analogy as the Fuel and Fire of Thinking, 2013 Douglas R. Hofstadter, Emmanuel Sander



'Hippocampophony'. This almost brings me to tears. Doesn't it make sense that thought would be made from something like music?




However, by 2021, Russell and Norvig had changed their minds.[19] Deep learning networks and machine learning in general require extensive fine tuning – they must be iteratively tested until they begin to show the desired behavior. This is a scruffy methodology.


This is, unless I am deeply mistaken, Charia Marlettos working definition of knowledge.


I know that was in his talk here: https://youtu.be/VzO8e_f2Hk8?si=Zo88DA-hvjKlKgJF


Guenther Palm has called a similar idea the 'survival algorithm'.


For instance, in Dennett's Consciousness Explained 1991; one of the central ideas is that the mind is a 'virtual machine' running on the computer of the brain.


Else, you would not be able to program it. - The wonderful focusing nature programming. The philosophy which needs to be right, otherwise it doesn't work.

Of course, I just call this cybernetics, blissfully content with the idea that our mechanistic understanding and the power of abstraction will conquer all problems.


That probably has then to do with doing useful information processing, which means doing highly abstracted information processing. Because in order to have success, you want to be fast. And being fast and higher abstractions are somehow pointing to the same underlying logic in nature or something.

Has to do with resource constraints, the power of computer programming languages and building blocks and these things somehow.


Also Marvin Minksy, 'The Emotion Machine' 2006 and 'The Society of Mind'


Speaking of evolved programs strongly brings to mind the wonderful field of ethology - See Timbergen, Dawkins, E.O Wilson etc.


See Braitenberg 1977 On the Texture of Brains -An Introduction to Neuroanatomy for the Cybernetically Minded for wonderful musings on such topics.


Hal Abelson On The Philosophy Of Lisp Programming, how the AI lab shaped a whole philosophy of programming.


https://youtu.be/HFGh5MZhRnc?si=KU1HlQaHtlYOTqhf this is a theme touched on in SICP, too.

  • imperative programming: A machine with instructions
  • declarative programming: Timeless data transformations
  • logic programming: Relationships
  • actor-based systems: Little agents

And so forth

In some programming paradigms, you don't say up front what it does:

Chemical computing, genetic algorithms, and neuronal nets.


Software engineering is philosophy, on multiple levels so.

We need to come up with ways of making explanations.

For instance Stuart Halloway.

Gerald Sussman talks about this aspect of software engineering, too.

Is "Chicago" the 7-character string, or is the city in the territory? The quotation and the referent are different things. But the programmer will decide on this meaning.


This doesn't mean that we program a program in terms of such a language afterward. The program we build is allowed to be a development plan, not a detailed wiring diagram.


[[https://youtu.be/oKg1hTOQXoY?si=hO8FFik62dObTaGp] [Alan Kay at OOPSLA 1997 - The computer revolution hasnt happened yet]]


There is a program with this property, it is called 'emacs' and runs on unix.


Yes, that is /Snowcrash/s metaverse.

And yes those landscapes/cities are Hofstadter meaning landscapes.


See Braitenberg 1986 for a bit of solid reasoning. The bottom line is you have way more connections between the cortex than connections that go in.


Vehicle 12, Braitenberg Vehicles: Experiments in Synthetic Psychology


If you are a biological system, you the genes will be discarded by evolution if you don't make a machine that has to to with survival



Seems like the Tübinger Version of cell assemblies is the brain child of Braitenberg together with G. Palm.


There is another operation, analogous hypervector bind, that highlights the contrast between 2 ideas. I am not sure yet how to make that with cell assemblies. Maybe project into another area?


A higher time granularity can be achieved with slow fibers. That looks like what the cerebellum is about. See Braitenberg 1977.


That idea is from G. Palm Neuronal Assemblies


Unless I cheat and imagine saying it in different contexts, then it jumps between contexts


There is a wonderful rendition of something similar in Scrubs. I searched 'I am cooliam'. Only found the german version.

'Ich bin coolich'.

This is a related phenomenon, where the syllables in a repeated phrase will morph together in a new way, you don't separate out the meaning anymore in the same precise way.


The parallel nature of the computational paradigm being discussed here is I think some core notion.


I'm embarrased for the field of neuroscience that you can go through textbooks and lectures and nobody talks about this.

Maybe they all rush to 'do science' on cortex because cortex represents mental content, so you can always find neuroscience about some mental content and that always sounds cool? It's simply a mistake from my perspective. From my perspective, this looks as if you think you can explain a car by explaining the arrangements of the seats or something.


It is interesting to consider that eye movement is known to somehow come from whole cortex. This is challanging to a 'cortical modules' view. I enjoyed this Lecture by Pedro Pasik quite a bit: Lecture 6. The Oculomotor System I-II Structure, Function and Dysfunction


Hebb, D.O.: The Organization of Behaviour. New York: Wiley, 1949


I think that by default it's super vague. It is that you use your imagination to fill in the states, but you usually don't realize it works like that.

So vague state from hippocampus -> cortex -> cortex uses imagination to fill in the blanks.

I think you even use your inner voice, like a voice actor, to make the people in the memory come alive, lol.

It is different if you have something sharp and emotionally salient happening. I think then you repeat the memory over and over, producing a sharp and detailed version. Perhaps you need to recall a midterm memory every few minutes, or every few hours or something in order to get something crisp.

(and everything in between of course).

This assumes sort of average midterm memory capability. Some super-rememberers are far from the end of some bell curve.


thinking goo is a synonym for 'probably something like cortex'. Probably something that does high-dimensional, parallel computing. Although I am biased here towards considering it's implementing a cell assembly calculus. (What Braitenberg called a Conceptron).


The time scales of this 'immediately' would be very interesting to know.

It would show us aspects of the assembly calculus implementation the brain uses. And it might constrain what we can think in terms of architecture (like this predictor loop and so forth).


Here there are 2 ideas I have:

World A: The eager activation world: If you make a neuro-darwnistic rule that only the neurons with the highest activation survive, you have shaped your cortex substance into an eager activator substance.

World B: The harmonious memes world: You figure out some memetic drivers that give you memes that are somehow in harmony and supplement each other.

Perhaps these are equivalent, but why exactly I cannot say right now. Maybe something with abstraction and hierarchies.

Perhaps it has to do with the actuator branch of the system, it needs to do things in the world to be useful. So the memetic landscapes also need to be shaped so the system does something useful in the world.


Modifying meaning spaces to what matters is what 'attention' mechanisms in ANN are, too. I call this perspective mechanism because I think that is a better name.


Gregorian comes from Dennett, he points out that there these mental technologies, tricks and so forth. Pieces of knowledge - memes, that allow us to think.

Some of these he calls 'intuition pumps', or 'necktop applets'.

Not sure which book is from, 'Bacteria from Bach and Back' probably talks about it.


See cognitive neuroscience: Wendy Suzuki's Lecture is cool.

Date: 2024-02-08 Thu 12:45

Email: Benjamin.Schwerdtner@gmail.com