Neuronal Ensemble Memetics

There is quite a bit of a lack of what I call neurophilosophy in neuroscience. There is no model or theory that in some overarching way talks about what the relationship of the brain and the world is and so forth.

Perhaps the conventional view:

  wrong:


       +-----------------------+
       |                       |
       |                       |  ...
       |   ..<-[]<--[ ]        |  3
       |             ^         |  2
       |             |         |
       |          +--+---+     |
       |          |      |     |  1
       |          +------+     |
       |             ^         |
       |             |         |
       +-------------+---------+   brain
                     |
                     |
                     +--------------------  sensor data


Information processing flow

• Sensor data goes into the brain(or more precisely, the Cortex, which is seen as the 'real' brain).
• A presumable feedforward cortical -> cortical 'information processing' is 'feature extracting' information from the sensor data.
• Some theories exist of what this information processing could be (hierarchical 'grandmother neurons', 'ensemble encoding'), but I leave that for a future topic.
• It is interesting to note that the literature on object recognition centers on single neuron analysis (even 'ensemble encoding' has implicitly the idea that sup-pieces of information are encoded in neurons).
• At the same time, the rest of cognitive neuroscience focuses on how cortical areas are active and inactive.

David Deutsch would note that this is based on a flawed theory of knowledge. It is trying to induce knowledge of what happened. The same misconception is embedded in current machine learning [Why has AGI not been created yet?].

The world is big and our minds are small, we must speculate. And after the speculation has produced some candidate toy ideas, they have the chance to survive.

—

I'll just jump to my current interpretation:

updated:


     +-----------------------+
     |  +----+        +----+ |
     |  |    |        |    | |
     |  |    | +----+ |    | |  meaning level
     |  |    | |    | |    | |
     |  | A  | | B  | |  C | |
     |  |    | |    | |    | |
     +--+----+-+----+-+----+-+
     |         +----+ +----+ |  sensor level
     +------------^----------+
                  |
                  |
                  +------------------------
                                sensor data



A, B, C - Expectation data structures

Ironically, perception, just like science, is about the stuff that we don't see. I think we see this in the overall anatomy of the cortex. There is much space for meaning, what lives in the meaning spaces?

The brain is not a receptacle of information. But an active constructor of the inner world. Surely, philosophers of old must have had the idea already. I am aware of Prediction processing, [Andy Clark], similarly the iterative improvements on guesses that make a controlled hallucination by Anil Seth (2021). I agree with the notion of making the top-down processes an active part of the system.

Predictive processing still has the notion of information processing flows, where some kind of top-down computers are now iteratively shaping the bottom layers or such. It does not come with a computational theory, as far as I can see, that says how the brain is making predictions.

The neuronal ensembles offer an updated view.

If neuronal ensembles are high-dimensional data structures, it is like there are puzzle pieces or perhaps space-ship amebas, made from activity. The activity represents information in the network. Like an active data structure. These are patterns completed so that a piece of information is useful. This opens the door to information mixing. And whatever the current activity, it is in turn the context for what can ignite. To explain what a data structure is, imagine that you load a file into your operating system, this is roughly the relationship of a program and its content. Observe that in order to have any useful power over the bits that make up the file, you want the operating system, to serve as a system of affordances, including being able to observe the system - for instance, navigating the file tree of the filesystem. This is the relationship of brain software to its data, too. Data must be loaded, and when it is, it is a representation of the data. And the ensembles are suited to fulfill exactly this role.

They are data structures that want to be stable (Cell assemblies and Memetic Landscapes), this might mean that they have dynamic parts or fizzle-ling parts of temporal structure.

If there is a spaceship of correlated activity that says here is a banana, and if it wants to be stable, for instance, stay alive even if you don't look at the banana. If it also represents an expectation structure when the eyes are at this position, then the sensor level represents such and such data. It is implementing object permanence for us already.

From this fragment of explanation I sort of see how this can represent the causal structures of the world, eventually. It would perhaps be useless, if not for the fact that Cell Assemblies Have Drivers For Generality and Abstraction. Whatever data structures represent the abstract notion of a room, a box, or container, or hole, a volume, a surface, or the stickiness of a surface. Such pieces of activity are more stable than others because they are useful across many situations. IIIa): The Living Software. The composability of those must work on the fly and be reliable, too. I must be able to say x is at position y, binding these 2 pieces of information. (Raw ideas: Getting A Visual Field From Selfish Memes, Creating Sync Activation).

There is something deep about the physical world I think, that makes the notion of stable information useful for describing and navigating the world. Why this is useful is perhaps a deeper problem of a field that unifies theoretical physics and epistemology.

There must be something about making elegant, "minimal and sufficient?" expectation structures, using systems of building blocks (languages). Whatever the substances, the wires, and the devices around them - they must create mechanisms that cannot help but grow into software that arranges these expectation structures on the fly, which I go ahead and label beliefs and explanations. (The Structure And Function of The Cell Assemblies Is Their ad-hoc Epistemology).

This is a very parallel mechanism. More like a kaleidoscope where all the elements update all at once - but many times per second. The dimensionality of the arrangement is reduced, just like in a Rubik's Cube the elements are forced to move with each other.

Since these stretch into "sensor levels" (Input Circuits And Latent Spaces / The Games of the Circuits), they implement an expectation about what the world is like.

Expectation instead of prediction because it is perfectly allowed to represent a counterfactual. If you look here, you will see x is a valid and useful expectation to have;

The user is offered the affordance. And this is useful whether the user looks, or not. My theory is that this collection of object ideas can be interpreted as a visual field. (They all say where and what to expect when you look at that where).

I don't think prediction is the right word. Becuase those expectations are counterfactual. It is what you could look at, not what you will look at. And I suspect it is allowed to encode a range of such expectations, too (Because something want's to be as stable as possible).

Another update, taking Murray Sherman and the thalamus people seriously:

                                 meaning level
     +-----------------------+
     |  +----+        +----+ |
     |  |    |        |    | | ....
     |  |    | +----+ |    | |<------ affordances
     |  |    | |    | |    | |<------ interpretations
     |  | A  | | B  | |  C | |<------ affordances
     |  |    | |    | |    | |<------ interpretations
     +--+----+-+----+-+----+-+
     |         +----+ +----+ |  sensor level
     +------------^----------+
                  |
                  |
                  +------------------------
                                sensor data



A, B, C - Expectation data structures

The layout of the circuits puts motor output inside the loops. That is because the higher thalamic nuclei get inputs from layer 5 pyramidal neurons, which all branch. They make one output to motor centers, and one output to the higher thalamic nuclei.

This arrangement is strange at first, but evolutionarily satisfying, along those lines:

• A generic neo-cortex module evolved with thalamic inputs and motor outputs, thereby being immediately useful.
• The logic of the efference copy, which ethologists of old have predicted, is a strong evolutionary driver for branching the axon, back to the input nucleus, this efference copy was then input to the system again.
• Since there was a new kind of input to the system now, there was an evolutionary driver to duplicate the whole arrangement, into a higher-order input nucleus and a higher-order cortical region.
• And the input to the higher stuff is motor command data, hence the latent space of the cortex is a motor output encoding.

This is a drastic update from the current conventional view, where we have only 1 motor output module, the motor cortex. But in the updated view, all cortex is contributing to behavior. Also, the sensory cortex might be relatively generic, instead of areas being specialized for certain functions, the new view suggests a generic computational module that is repeated (and perhaps secondarily has some specialization). And all these modules are capable of making motor outputs.

The (neuro-) philosophy of Constructivism [Heinz Von Foerster, Humberto Maturana] had this idea that the mind creates the world. That perception is actively created by the mind. I think they also said that perception and action are intertwined, that one cannot see without moving the eyes and so forth. And I think it turns out that they were right. Some Memetics On Eye Movement And Maybe How That Says Where. The sensors are not a source of truth, the sensors are a resource to an explanation making system. In my view, the ideas are growing. They are biological entities (The Biology of Cell Assemblies / A New Kind of Biology)

Some strands of thinking in this direction then go on to say that we all have our own world and reality. Fallabilism is the notion that we are never right, that we can always find a deeper explanation. This also means that the truths are out there, for us to converge upon.

Since the trans-thalamic messages are all forced to make motor output at the same time, (because the layer 5 output neurons make branching axons), one layer's motor output is the input to the next layer. This is a 'poly computing' [Bongard, Levin 2022] implementation. Where the computation of one element is re-interpreted from the 'perspective?' of a new element. This arrangement mirrors the logic of an Exapatation in evolutionary theory.

(1) A character, previously shaped by natural selection for a particular function (an adaptation), is coopted for a new use—cooptation. (2) A character whose origin cannot be ascribed to the direct action of natural selection (a nonaptation), is coopted for a current use—cooptation. (Gould and Vrba 1982, Table 1)

What is a range of action from one perspective, is input data from the other perspective.

Why the range of action? Because the ensembles stretching across those circuits have memetic drivers to be stable, if they have the chance to spread to higher areas, they will want to do so. If the circuits force them to output motor data, they will have to play a game that balances their conflicting effects. If the animal moves, the situation will change, and if the ensembles represent the situation but also want to be stable, they will have to play the game of finding the smallest set of behaviors that keeps them stable.

Imagine looking at a line, the ensemble on the first layer says move your eyes like this (straight line). Because it wants the sensors to stay the same. This eye movement data is input to the second layer, which says I interpret this data as 1 line, color filled in and so forth. (something like that). Where does the color come from? From whatever color ensembles are active right now. Because the system is looking at a position, and has a certain shape/color interpretation, all these ensembles are active together and either compose via some phase-reset and synchronous activation mechanism or associate via on-the-fly plasticity. Both of these we can biologically plausbily require.

What would this mean for thought-level mental content? Perhaps it means that for instance, the affordances represent a range of thought narratives, which will incorporate their ideas or something.

Circuits and devices (like the Braitenberg thought pump) can make the ensembles play different kinds of games.

This way, if a neuronal ensemble wants to stretch across meaning levels, it will have to represent affordances at layer 5. This interpretation game is also a usability game.

Easy to imagine with eye movements again. If the banana from up top still says move here and you will see me, it knows itself how to be used. In this way, visual object memes are used by being looked at; If object memes fail to output the right eye movements, then they are worse memes. (It is a topic of development why our object memes are good).

If they are within reach, they can say grab me with your hand you will feel this weight and so forth. This way the object properties become lists of affordances. How do you understand the very large and very small, then? With analogies, I think. The sun is a ball going across the sky, atoms are billiard balls, and electrons are clouds or inkblots.

Observe that inner affordances, acts of attention exist, too. This way a thought-meme can say think me. This is the same as having a handle, a set of affordances, on the entities in your operating system. But in this paradigm, all the data structures try to figure out on their own how to be used. They are buttons with previews. And they are symbiotic with ideas that would lead up to them. This way a train of thought can have a handle on the first wagon, and the whole train would bias the system into thinking of the first wagon, if it could.

A little example of this is singing A, B, C, D, E, F, G, ….

P and O are memes, too. They have a relationship to A, and even more so to A with the tune attached.

• M. Sherman: An Input Nucleus To Rule Them All, Parallel Processing Flows.
• Those stacks of interpretation and affordance are all arranged in parallel.
• I think this will turn out to mean that the concepts of say 'a line here', 'a color stripe here', 'a painting here', 'I could scribble on the painting',… Are all immediately, in parallel, constraining themselves, and they all immediately go from lower detail to higher detail perhaps.
• "top-down" and "bottom-up" are then ensembles that span meaning levels - technically sweet I would say.
• Then it is not surprising that most meaning comes from something else than sensor data if one simply considers the amount of neocortex that isn't primary sensory area.
• And because of M. Sherman's A Curious Arrangement, all the meaning is knit together via action.

The edge of this expectation representation game is perhaps just when waking up, doesn't it feel like the system is booting for a moment? My interpretation is that all the neuronal ensembles need to ignite, using 'flash-drive' lookups and cues from the sensors.

"Surprise" and Confusion need to be handled, too. (also: Confusion and Socratic Wires).

György Buzsáki's neurophilosophy is rich. The notion of the pre-allocated hippocampal 'sequences' (4) is an implementation of the idea comes first.

Even an inexperienced brain has a huge reservoir of unique neuronal trajectories with the potential to acquire real-life significance but only exploratory, active experience can attach meaning to the largely pre-configured firing patterns of neuronal sequences.

The reservoir of neuronal sequences contains a wide spectrum of high-firing rigid and low-firing plastic members, that are interconnected via pre-formed rules. The strongly interconnected, pre-configured backbone, with its highly active member neurons, enables the brain to regard no situation as completely unknown.

As a result, the brain always takes its best guess in any situation and tests its most plausible hypothesis. Each situation, novel or familiar, can be matched with the highest probability neuronal state. A reflection of the brains best guess.

The brain just cannot help it. It always instantly compares relationships [the function of the hippocampus according to Buzsáki], rather than identifying explicit features.

There is no such thing as 'unknown' for the brain. Every new mountain, river or situation has elements of familiarity, reflecting previous experiences in similar situations, which can activate one of the preexisting neuronal trajectories. Thus familiarity and novelty are not complete strangers. They are related to each other and need each other to exist.

Buzsáki, G. (2019). The Brain From Inside Out. My emphasis.

—

Joshua Bach has a similar example. One remembers going into a dark cellar to fetch something as a child and having ideas of various monsters in the dark. It seems like the creatures are there in our minds. Not mere imagination, but the stuff that makes perception.

In my words, they are the meanings, beliefs, expectations, schemata, frames or situations. They exist in the mind, decoupled from the sensors. Perhaps sensors and action are "merely" resources for the ideas, to 'criticize' their alternatives. And this way, "good" best guesses are left over.

Neither the world nor the mind was a clear-cut 'ground', but grounding is an evolutionary process. And the function of brain software must be to facilitate 1) The existence of ideas 2) The criticism of ideas, in the broadest sense.

The infinite use of finite means by Wilhelm von Humboldt is what we call the means of combination in programming (5).

• Make some building blocks.
• Make rules for composing the building blocks.
• Buzsáki calls that the neuronal syntax of neuroscience.
• Chomsky calls that the universal grammar I guess.
• ???
• You have a system of expression with general, and vast reach, perhaps with certain universalities.
• I think eventually we might be able to quantify such reach, and we would be able to say that with s amount of ways of putting things together, and n amount of building blocks, you get to an exponential explosion of expressivity.
• This expressivity is at the core of natural language, and computer programming languages, and I think of the internal programming languages of brain software, too.

The means of abstraction would say that you need to be able to create new building blocks (given for a particular natural language via word coining). This includes giving names to "frameworks" in a broad sense. A frame in Minsky (7) is "A sort of skeleton, somewhat like an application form with many blanks or slots to be filled".

In programming, a framework might be a map or record, where the "keys" or "variables" (also 'roles') are filled with "values" (also 'fillers'). It can also be a sub-program, where some variables are substituted with particular values.

This allows the user of the language to speak of the relationships of things. And that is detail-independent one might say. (The same way information is substrate-independent, the substrate is a detail, then).

In what ways brain software represents such relationships and frames is one of the important questions.

[ ? action is how we come up with relationships in the first place. ]

[ observe that action can mean messing with the internal state of the system itself, too - internal actions ]

[ Two things are identical when changing either changes the other. ]

[ In order for anything to be stable, it has to move ]

[ ? … Then perhaps it is the ideas that change in the right way so they are stable, even when the details change, that are the abstract ideas ] [ This logic is mirrored in the so-called perisaccadic visual remapping (etc.) of visual neuroscience ]

Brian software abstractions allow us* to eventually express I see, or I understand. Without caring about the details, which are filled in somewhere else.

*) "Us" in this case means us, the hypothetical programmers inside a brain software system. Not the cognitive user of brain software, who is removed from such details.

At Some Memetics On Eye Movement And Maybe How That Says Where, I have the idea that eye movement motor data is an abstract encoding of the extent and relationships of objects in the visual field. This is also a frame because the 'line data' is allowed to be composed with any color and texture data. Making blue or red lines and so forth. A line is a thing where you can move your eyes like "this" and you still see the same thing along the way.

A competent meme can say all the ways you can make transformations (motor movements), and it is still true - it can do that by being abstract and removed from the details. Something like a high-dimensional, procedural puzzle piece, a bag of counterfactuals, a puzzle piece that fits many things.

For instance, the notion of a container fits the counterfactual notions of poor water into me and it would stay inside me, put your hand inside me and you will experience my interior, turn me around with water inside and water will flow out of my opening. (You don't need to do these things, it can say what would happen). There are many transformations a toddler can do with their hand muscles and so forth, perhaps animated by some alternative memes of container. Still, container would stay true. Or be superseded by a better representation of container. In a high-dimensional computing framework, such a puzzle piece with many tentacle arms in meaning-space should be easy to say, once some details are figured out.

Then I understand the scene perhaps simply represents the notion of a scene we understand, with the details filled in later. This rhymes with various change blindnes effects.

This one from Nancy Kanwisher and collaborators is amazing.

This hiding the details is a property of well-designed software this is essential for making the cognitive user interface magical: Magical Interfaces, The Principle of Usability.

[we can recover composition and abstraction beautifully via high dimensional computing, more to come].

This happened for genes, that somehow were able to express the realm of possible biochemistry or something. It also happened for the evo-devo genetic toolkit (Wikipedia, Sean B. Carroll 2008), I think, that can express animal body plans (universally?).

Also: Digital infinity.

What we need is an analysis of programming and software that is deep enough to include brain software. I am not done with a computational layer. I want the logic of organization of the computational layer. The elements (neuronal ensembles or high-dimensional data structures), the means of combination and the means of abstraction are the starting point, not the thing to be explained. The way architecture is relevant for understanding a building, not the bricks.

I also don't want to think of details of human cognition and say how this or that could be modeled. I want a fundamental theory, the way evolution does it for animals. The details of human cognition will come out of the elaborations of the rules of a deeper process (That's from David Deutsch).

Note that I think this probably means building in a developmental phase, that prepps the system for representing physical reality. Growing in Kantian representations style. On Development. I believe making machine intelligence and explaining brain software means standing on the shoulders of computer science (formerly just cybernetics) and neuroscience. It would be foolish to ignore the existing knowledge of computing and programming when the goal is to figure out what the biological version of it is.

+----------------------+  mechanistic layer
|                 |    |
|                 |    |  Neurons, wires, devices, transistors
+-----------------+----+
               +--v---+ <---------------------------------------------------- ensembles
               +------+
+----------------------+ computational layer
|                      |
|    ^                 | ? neuronal nets, conceptrons
+----+-----------------+   high dimensional computing frameworks
     |        +--------+ <--------------------------------------------------- data structures
     |        +--------+                                                      building blocks
     |
+----+-----------------+ software layer
|    |                 |
|             ^        | Memetic engine matrix, Societies of Mind (Papert, Minsky),
+-------------+--------+ Meaning Cities (Hofstadter)
|    ^        |     ^  |
+----+--------------+--+ systems of expression
|    |    ^         |  | layer, layer, layer, (?) ..
+---------+------------+ The content of the software is software again
          |
+    -   -+    ---   --+ cognition
|                      |
|                      |
+----------------------+ The user and her interface
                         magical - but not magic.

                 +-- This is biology again
                 |
                 v
  +---------------------------------+
  | M  |  C  |  S  ... Cog          |
  +---------------------------------+
       ^     ^
       |     |
       |     |
   abstraction barriers.


M - Mechanistic layer
C - Computational layer
S - Software layer
Cog - Cognition layer

A heart valve can be implemented in terms of the biological leafy portal structure, or a so-called mechanical valve, which has a completely different mechanism, but its function is the same. Namely, to prevent blood from flowing back into a ventricle.

In general, an abstraction barrier allows a function to be expressed without mentioning the details of the implementation or mechanism. (What an abstraction barrier is, is I think expressable by constructor theory).

That there is an abstraction barrier between the mechanistic layer and the computational layer is uncontroversial. Otherwise, you would claim that the brain is not a computer.

It is the computational paradigm the brain implements. That is currently widely assumed to be neuronal nets [Hinton, Sejnowski]. But many people feel that there must be some richer aspects of the brain's version of them. Geoffrey Hinton recently mentioned he thinks there should be fast weights, pointing to what others call the neuronal ensembles and so forth. [In conversation | Geoffrey Hinton and Joel Hellermark].

Note that Yuste hypothesizes that ensembles are created on the fly by increasing their intrinsic excitability, not the weights (Iceberg Cell Assemblies).

Sejnowski mentioned how neuromodulators modify the function of the net on the fly, and how traveling waves pose open question marks. [Steven Wolfram and Tery Sejnowski].

My idea about the computational layer is Braitenbergs "conceptron", with wires:

• A recurrent neuronal net with sparse activity, activity can survive and pattern complete (similar to Hopfield attractors)
• Ensembles are high-dimensional data structures, Ensembles associate via on-the-fly plasticity (Iceberg Cell Assemblies)
• Rich wiring is important because it says what kinds of activity you get: Getting A Visual Field From Selfish Memes, Input Circuits And Latent Spaces / The Games of the Circuits
• Ensembles are compositional: the means of combination, How To Make Use of Synchronous Activation.
• Roughly something like situation -> interpretation are the means of abstraction (Context Is All You Need?).
• Devices poke around in the the network: Tübinger Cell Assemblies, The Concept Of Good Ideas And Thought Pumps (Devices are random and merely heuristical by themselves, easy to implement once the function is understood).
• Devices can be used from inside the network, e.g. Implementing Perspective, p-lines
• This all allows for stable information with meaning (because wires). This meaning is allowed to represent abstract procedures about how to use the computer this software runs on.
• Note that this doesn't say what the mind is, it says what the building blocks are, which must somehow create the mind during development.

Imagine an alternative history, where people still speak Victorian style. And where Darwin's and Wallace's theories were ridiculed and forgotten. Like tiny fireflies of insight, burning away in the dark.

The people of this world moved on without a theory of life;

'The mechanistic and architectural power immanent in protein biochemistry' from Warren McLoccuch and Walter Tipps in 1943. Pathed the way of seeing the cells and organisms as biochemistry devices.

Many biologists think that it is the job of biology to explain how the biochemistry of organisms is 'fitting the niche'. Their implicit bio philosophy is that 'fitting the niche' is a theory of life, and questioning how biochemistry relates to animism and complex adaptations is seen as a problem for philosophers.

Some philosophers speak of Élan vital, the substance of life. Some say that Élan vital can never be understood, that some mysteries are beyond the human intellect. Some say there is a substance dualism between life stuff and so-called ordinary matter. An animal is something else than a rock, and yet one is alive and one is not. How does the biochemistry give rise to aliveness? Some argue that aliveness is perhaps an epiphenomenon, along with the ride of the real thing that is happening, biochemistry.

Some biologists think that the mystery will be solved in 100 years hence perhaps not in my lifetime. They have searched and searched, and yes the ship is made from wooden planks on the left and the right. Now they are going to look if there are wooden planks at the front and back, too.

The problem of aliveness was always there and then labeled the hard problem of aliveness by Chavid Dalmers in the 2000s.

The field of machine proteomics is inspired by the 1943 McLuccuch paper, and implements statistical fitting of 'proteomic nets'. They started tinkering with nanotech, which mimics the function of proteins and other elements of cells. They created statistical algorithms and found algebraic tricks. The valley-slope mechanism allows for simulating the biochemistry of organisms, tweaking each protein parameter at a time, and 'fitting it to a niche'.

They have some impressive results and the field of artificial biology is a synonym for machine proteomics in the mind of the public. Artificial animals that do things like brewing beer and so-and-so.

In some ways, the impressive results of machine proteomics are more confusing than useful.

The prevailing understanding is that the biochemistry of organisms is implementing something like a proteomics net with learning. Those are networks of proteins, which interact with each other and produce the properties of cells. They are fitted to a niche via the process that tweaks the connections in the protein net called learning.

Many biologists are trying to use machine proteomics when they analyze organisms and find striking analogies between artificial and biological proteomics.

Biochemistry was once a deep topic for cyberneticians, but this field virtually doesn't exist anymore. Biochemistry became biochemistry science in university courses, where students are trained to contribute to industry, using artificial proteomics, and making money for businesses.

It looks like this organ is doing biochemistry for x. Is a respectable thing to say, even though replaced the word biochemistry with magic and get the same information content, yet, such rhetoric is required to be a respectable biologist.

For instance: It looks like leaves are doing the biochemistry of the tree. This is taken as a respectable and sensible thing to say. Students who ask but what is the biochemistry? are placated: I don't have a proteomics diagram for the biochemistry of a tree. The field simply is not this far.

—

Curious person: But why should this proteomic net be this proteomic net?

Scientist: Because this way the organism is doing exactly the kind of biochemistry that allows it to survive the challenges of the environment. Organisms are environment overcomers, and they make biochemistry that helps them survive.

Curious person: Alan Ruting's universality of biochemistry shows that all proteomics do all kinds of biochemistry. Doesn't this mean that there is something else to explain? Something that says how the proteomics is organized to do its biochemistry?

Scientist: People like Larvin Linsky in the 50s-70s have tried to come up with an organization of biochemistry, but they have failed. Because in the real world, biological niches are more complicated than the toy niches of Linsky. That is also the bitter lesson of machine proteomics. You can only get better organisms by scaling up the compute and niche data.

Curious person: But organisms in reality do not have millions upon millions of niche data points, from which they find statistical correlations of proteomic sheets that fit. The moment an organism is born, something already talks about what kind of proteomic net will be useful for it in its niche.

Scientist: Yes, that is the mystery of life. People like me are making progress in understanding what kind of biochemistry actual organisms use in order to fit their niche. But it is a slow process. Look at how we know that cell membranes implement such and such a function… And we also recently found that skin cells also do biochemistry!… Also, we found recently the biochemistry mucus membranes. I have a colleague that takes samples from real humans. They are now investigating whether those cells are also doing biochemistry… By decoding from the proteomics of organisms, we can say whether it comes from an eye cell or an arm cell. Isn't modern biochemical biology fascinating?

Scientist 2: At my lab, we take the idea of organization seriously. We use graph-based biochemistry, to model how real organisms might do proteomics in hierarchies and heterarchies.

Curious person: Nah, I mean something more fundamental than such details. And I think Linsky was, too. Why isn't there a deeper theory of biochemistry organization that is independent of the niche?

Curious person (to herself): I have to think something else than what these people are thinking. Imagine an alternative history where Alan Ruting died early and he didn't come up with the [ommitted] theory of brain-knowledge. What a strange world this would be. They would say the brain is doing information processing and this would be the end of their neurophilosophy. shudder.

The image of the world in the brain is a dynamic representation; not only the neighborhood relations and other forms of association of things are represented, but also the laws that govern the evolution of the environment. The patterns of activity within the brain that represent the present state continually go over into new patterns even if the sensory channels are temporarily closed, by virtue of the internal synaptic connections. If the brain is well wired, this evolution of the internal state matches the evolution that would be otherwise imposed by the changes of the input. Thus the capacity to predict the future is an essential attribute of the brain. Normally, in the process that we call perception, the sensory input and the internal prediction are inextricably mixed. They are the two irreducible components of the process of acquisition of knowledge in all cases where the available information is incomplete. And the amount of information is always insufficient as long as the brain is so small and the world so large. In science these two components are called hypothesis and experiment. As in science, so also in the brain the discrepancy between the prediction and the actual evolution of the input is used continually to amend the predicting mechanism. Not only a new hypothesis has to be developed, when the old one proves wrong, but the principles that generate the hypotheses have to be revised. This is how the information from the macrocosm is incorporated into the microcosm of the brain. The brain as a foretelling machine clashes with the more down to earth view of the brain as an input-output device that gives appropriate responses to sensory stimuli. However, the two views can be reconciled. The prediction that the brain makes at all times by letting its internal state evolve in accordance with the most probable evolution of the world, contains also a prediction of the future position of the animal itself within its environment. We may suppose that the motor output secondarily follows the prediction of the next state of the motor system. In the action of tracking a moving object we notice how our output is not governed by the immediate input but is, rather, a consequence of an ongoing prediction. When we sing in unison with other singers, we notice the curious ambiguity between action and perception. Subjectively one's own action of singing is perceived in a smooth blend with the singing of the others, or both seem to emerge from a continual process of prediction. We are reminded of the anatomy of the cortex where the motor area is not subordinated to the sensory areas, but arranged in parallel with them in the continuum of the cortical plane.

Braitenberg 1.

This flip away from the input-output device [Sherrington's reflex model] to the brain with an active role, it's own agency, it's own inner states (- mentality). For some reason, this is still cutting edge neuro philosophy [4, 2, as well as A. Clark recently, are still pointing this out].

But prediction, or action experimentation - prediction (emphasized by 4) is not sufficient to explain explanatory universality, also called creativity by David Deutsch in 8.

If I say I imagine a wizard flying on a dragon through space, backwards in time to the big bang. What was the role of prediction? Marginal. What is the role of an expressive software language for representing dreams of possible realities? That's somewhat closer to the mark.

If Alice comes home from a job interview 20 minutes early, and Bob perceives her sitting on the sofa without saying anything. Then Bob has an idea structure in his inner world that represents Alice and her interview, probably with a place. This situation somehow is central to Bob's interpretation of reality. But the situation is in the past and the situation is invisible to Bob.

And I think that Bob probably had multiple different alternative ideas, one in which Alice interview went well, and one where it went badly. These expectations are made from counterfactuals.

I would say perhaps Bob would be able describe the place with richer and richer details if being asked: A desk, some stern interviewer, plants in the corner.

Perhaps we can label this the microcosm, inner world of ideas, or virtual simulated world, or the mental world of Bob. If Bob is using a foretelling machine to generate this, then he has fine-grainend competencies of instantiating a certain start state of this foretelling machine. We are then interested in the function and rules for instantiating these inner start states of his, which one might label situations (Hofstadter).

We wonder about the same interesting questions, for instance, how does Bob manage to not get confused between his inner start states, and the ones coming from the sensors?

One might say now that Bobs prediction machine is using his mentality as a resource to predict the world.

                     /------+        inputs, error  <+
                    /------+| <---------             |
                    |      ||                        |
           +--- -- -+- P   || ---------> prediction  |
           |        |      ||
                    +------+              ^
           |                              |
                              -- ---- -- -+ informs
           |     +---+
           +-----+ Ψ +----+  rich inner world
           |     +-+-+    |
           v       |      |
                   v      v

         ideas, plans, imagination - the mind


P - the predictor

But then the mind was again the interesting thing to explain. The structure, function and organization principles of what makes ideas and explanations.

Sometimes we are rooting for a certain kind of explanation. There are still bananas in the storage room. And we look for all the little hints that support the idea (called motivated reasoning in cognitive neuroscience). We are not predicting the world, we are grooming a belief.

I'm sure when toddler acquires language, they hear you say apple in a situation where there is an apple and a banana as possible referents. They will not only realize that apple is the word for the thing that they don't know yet, but they will do so retroactively, too. I think the way it works is that you lie in bed a week later as a toddler, and by now you have learned what a banana is, you somehow remember the apple-banana situation and you go: "aha! this thing must then be an apple". It's a bit of a stretcher to explain this process in terms of a prediction machine, or prediction-action-experimentation machine. This prediction machine is creating a fine-grained tapestry of explanation, where situations and micro situations are all made to fit in some kind of society of knowledge. And things like this.

A gene pool is a shared set of genes that is separated from other such sets. It is a useful abstraction for evolutionary biology, when considering speciation mechanisms.

The important thing about a gene pool is how it is separated from other pools.

You might want to call the maximal neuronal ensemble from above a pool.

The halo of an ensemble contains multiple sub-ensembles. You might go that way and call that a pool, out of which a thought pump would now choose a subset.

I find these contrived and would simply not use the term pool in neuronal ensemble memetics.

The closest to cultural meme pools would be things like 'information bubbles'; And the cutoff culture of uncontacted Amazon tribes. These truly might be the analog of a 'species' of culture. But why bother, if 'information bubble' is a good name?

A gene complex / Supergene:

A supergene is a chromosomal region encompassing multiple neighboring genes that are inherited together because of close genetic linkage, i.e. much less recombination than would normally be expected.

A gene eye view explanation lies at hand, they are selected together because they contribute all together on some function, presumably, if one is missing the whole gene complex would not have any effect on genetic fitness.

The genes in a gene complex form a coalition then. And contribute all to some function.

For neuronal ensembles, we see that all ensembles are made from sub-ensembles, even if they contain a single center in Braitenberg's sense. Because we can always imagine splitting an ensemble in two, and then each of them has a slightly different meaning depending on its connections. (Presumably random in brain).

This certainly is a thing for the ensembles, too. A compose would presumably be a fundamental operation of neuronal ensembles. That is juxtapose two ensembles together, forming an ensemble that is made from both, perhaps with temporal structure (or not).

Neural ensembles are their own composite data type, so all ensemble mechanisms will have a closure property in the programming sense. That is, the same operation will work for the composed ensemble in turn and so forth.

Vempala calls this hierarchical composing a merge. So he probably would call those merged cell assemblies?

Names: Supersets, composed ensembles, aggregates, super ensembles, ensemble complexes, the complete ensemble (in the sense from above. The largest ensemble present at one time in the network).

The other way around: Centers (in Braitenberg sense, imagine lowering the excitability, which neurons would be left over?), sub-ensembles, elementary ensembles,

Theoretically, you might label an atomic ensemble one that cannot be divided further for ensemble properties to hold. This would depend on the neuronal area (its inhibition model and so forth). It would also need to take into account the time window.

Rafael Yuste mentioned that they could not produce bigger artificial ensembles than 100 neurons⁸, perhaps this puts the size of atomic ensembles in a rough ballpark. (Certainly smaller than 100).

Ensemble composition into super-ensembles is not to be confused with the different notion of association, which finds the overlap of ensembles. Conjecture: This is another basic operation in ensemble theory: Activating 2 ensembles in an area together will rapidly create an overlap ensemble. (This is proven for assembly calculus [Vempala 2022], which uses Hebbian Plasticity). We say they associate. I also have called this merge. But given that Vempala calls the composition from above a merge, it would probably be less confusing to call it associate, or overlap find, or create a new ensemble that is similar to both.

This is a bundle in hyperdimensional computing.

This overlap-making is remarkable: The logic of the substrate simply forces the ensembles to create new ensembles, similar to whatever is active together. This is an example of where an explanation of the level of the neurons would break down. Why would activity move forward, leaving its neurons behind?

This is not a high-highfalutin problem. This is a simple analysis of what are the wires between the world and the brain, and what mechanisms allow ideas to exist as a kind of software on the brain.

This analysis is needed in order to have a complete explanation of what the brain is. I would label this 'neurophilosophy'.

All neuroscience is parochial unless it has a neurophilosophy. Its explanations are small, and will not talk about how things could be, and then say how the brain is doing one of the possible solutions to each problem. Without a neurophilosophy, you would not be able to look at a brain or its software and say this is a kludge. The way we can look at the anatomy of the eye and say this is bad design, the ganglion cells should point the other way, like in mollusks. (Blind spot)

Programming (or software engineering) is a kind of practical philosophy, which can be applied to any kind of problem domain. This page is an attempt to apply it to the domain of explaining biological intelligence.

It is a blend of art, science, and engineering, it cannot be otherwise. It might be a going back and forth between building blocks and their reach (described here The Science Fiction Software Engineering Approach). How exactly to do it is poorly understood.

On a content level, the mind must be a kind of software (see Magical Interfaces, The Principle of Usability). My view is that the philosophy of programming is a relevant perspective on brain-software. This kind of interpretation has found very little attention so far. Note that the subject of computational neuroscience is models of the nets, their wiring and their attractors. The subject of programming philosophy is programming languages, what affordances they provide, how to design them and so forth.

Minsky was explicit about trying to find the land in the middle, between neurons and psychology. I see his approach more than anything else out there as a 'software engineering' approach. Compared to this here, he was thinking top-down, high-level on what kind of software entities are needed in order to make minds [Minsky 1986, 2006] (which he developed together with Seymour Papert). This is software engineering.

Minksy had the knowledge lines and so forth at the bottom, I have the neuronal ensembles and I think they yield a powerful and neuroscientifically grounded intermediate layer. Helping us create an explanation of brain-software from bottom to top.

Note that high dimensional computing was the only thing since the 90's. Kanerva's intro is from 2009 (Hyperdimensional Computing: An Introduction to Computing in Distributed Representation with High-Dimensional Random Vectors). In case Minsky knew about neuronal ensembles, I think he simply missed out on them.

High-dimensional data structures are versatile and expressive. They can represent symbols, situations, interpretations, pieces of knowledge and so forth.

[high dimensional computing addresses the challanges posed connectionism]

[neuronal ensemble memetics is not genetic programming Notes On How Neuronal Ensembles Are Different Replicators From Genes]

—

I consider Hoftstadter's ideas an attempt to talk about this land of the middle, but looking from the top, from psychology and the cognitive psychology of language and so forth down into the meaning levels. [Hofstadter and Sander 2013] Those meaning entities (cities of meaning, analogies/little situations) correspond to software entities in my view.

—

Once the view is taken, it is obvious that brain-software must create some kind of advanced, self-assembling, 'magical' user interface.

This interface is so immensely good, that we might even take it as a benchmark of what a good interface is. We do this when we say this tool is like a glove on my hand, this is an extension of my arm, this operating system is doing everything automatically (meaning your intent is picked up and has effects seamlessly).

It is this seamless and prescient property, this "knowing on its own", this competence of brain software that I would like to call 'magical'.

The way I can decide to move my muscles and they move, without having any clue of how they move. The way I can decide to move my eye to an object and appreciate its effect on my mind - without the slightest hint of a clue of the details (i.e. the leakiness of abstraction is low).

The unlimited malleability and composability of 'imagination' or 'working memory' states. (i.e. the computational primitives are well-suited for expressing ideas and creating explanations).

'Bringing to mind', 'remembering', 'moving attention' and so forth are user-level internal operations with the same property of usability.

The user or users must necessarily be brain-software entities, too. It is a system that self-assembles into containing competent user-level entities, using elegant interfaces to producer-level software entities in the system. The computational paradigm must be extremely dynamic, that is the content of the software is being created and transformed on the fly. The content must contain the equivalent of procedures, too.

We can assume that the power of abstraction holds for brain-software, too. This means it would be level violations or leaky-abstractions if user-level entities were concerned with the details of producer-level entities in the software. This "detail ignoring", which is the nature of good interfaces, is what Dennett labeled 'the user illusion'. I.e the user doesn't know how the producer works (and shouldn't know).

When McCulloch and Pitts said the brain is a universal computer, this was not an end conclusion, but the starting point. Because different computational systems have different amounts of power. That is not their absolute power but something else. Their elegance, their swag, their expressivity, the beauty of their abstractions. Their ability to get stuff done.

It is a physics concept; Bringing it into biology is I think a relatively modern notion. [who talked about this first? Maybe Hopfields physics perspective on neuroscience 1982?].

[Sean Carrol is a communicator about the concept of emergence]

[Daniel Dennet: Free floating rationals. These rationals already exist, they do not emerge from biology.] (But maybe a theoretical physicist would argue that the whole explanation stack needs to be able to emerge those rationals, too).

[Kevin Mitchel emphasizes higher-order organization, it exists on its own, not emergently]

[Chiara Marletto gives an example: To understand the bits of a computer, one answer is the rules of the circuits, but another answer is the program that is running.]

These questions are arguably at the heart of what the cyberneticians of old were thinking about.

Biologies explanations are high-level already, called structure and function. It is the poetry of biochemistry, physiology, organismic biology and so forth.

"The function of the heart is to pump blood" is immensely useful.

I am happy with using this solid biological reasoning on the functioning of the system wherever it suits me.

[Stephen Wolfram calls this computational reducibility. That is you can make explanations about a system without needing to compute all the parts].

Marletto asks us to consider the notion of an adder. There is no perfect adder circuit, the wires of electrical circuits might get damaged by heat and so forth.

Our circuits instantiate the abstract, counterfactual notion of adder. It is real and not real at the same time. It is only useful because the notion is perfect, but because of fallibilism [Socrates], we will never instantiate a perfect version of it. Similarly, a biological system would never instantiate a perfect blood-pump. It's almost nonsensical to wonder what that would mean. Because it would mean knowing all future science, too. For all evolution knows there is a better way to build heart muscles when one is exploiting quantum effects or something.

This is the status of the function biological systems. For this reason biologists don't have trouble accepting that counterfactuals are essential for explaining the world.

It makes sense to me to say that the function of the brain is to create brain software, and this has the same status as the abstract, counterfactual adder notion. Just that this time, the program has a slightly different flavor. But there is a kind of software that is being approximated. It is a program with a self-assembling development plan, yielding a range of late-bound content. The content, in turn, is software [Von Neuman, McCarthy, Minsky], which enables a dynamic user interface [Alan Kay] to a user, which also must be self-assembled.

This is why you are not done explaining the mind when you explain the computation that the brain is doing. Because it is the organization of the computation, how it uses itself, that is the actual function of the brain.

This is mirrored in the fact that we can draw an abstraction barrier between what I call a cognition machine, or magical user interface, or the operating system of the brain, and its implementation. The implementation can be neural nets, hypervector framework, or self-modifying Lisp procedures. It just needs to be a generally powerful computing framework.

                   orchestration
                  -------------
     neurons      -------------  thalamus, striatum, claustrum, ...?
     +------------+
     | ^          |
     +-+----------+
     | |        |||                 + [ flash drive ] (Hippocampus)
     +-+--------+++                 + ? ... gadgets
       |        ||
       |        ||
       |        |+---->
-------+        |
                |
                |
                |
                v

• The neurons, their wiring, how they are connected to the outside world, how they are organized in nuclei and so forth
• The orchestration, which I label the machinery that creates, amplifies, extinguishes, composes (perhaps) activity

• biological cybernetics (e.g. Braitenberg vehicles and how to build them)
• neuroscience

formalisms that describe the layer:

• Conceptrons (hypothetical, Braitenberg)
• Assembly Calculus (Vempala)
• Computational models of neuronal ensembles
• The formalisms that take the ensembles as their center of explanation should center on the notion of the substrate, because we think in terms of the emergent properties of the neurons, per definition.

It might be called the machine level, neuron level, wire level, circuitry level, or substrate level.

It might be called the ensemble layer.

 +--+
 |  |  +-----+
 +--+  +-----+
    +--+
    |  |
    +--+

ensembles

• Consider magnetism: The atoms (the elements) are not magnetic, but the substrate is.
• Ensembles? See Summary So Far, Cell assemblies and Memetic Landscapes
• The short version is that they are self-activating subnetworks, they stay activated by themselves. They are the entities on top of the neuronal network, like looking at the image instead of the pixels (Rafael Yuste's analogy).
• I.e. the neuron doctrine is replaced by the neuronal ensembles.
• In computer science terms, they are a data structure emerging from the previous layer of the neurons.
• A high-dimensional computing framework made from the network and its activity.

If you have self-activating pockets or subnetworks of neurons, you can activate a piece of the network independently of sensor input, that is the doorway to mentality. That is a piece of information that can refer now to something in the world.

• Lorente de Nó:

  The active neurons are arranged in convergent chains of relatively simple composition, which may be called multiple and closed chains through which impulses circulate. The multiple chain of neurons is the elementary unit of transmission; it supersedes the classical reflex arc with a fixed number of synapses (Lorente de Nó, 1938b).

• chains ³⁰
• D. Hebb 1949: Cell Assemblies. (Although Hebbian Learning is not actually what cortex does (Alternative Plasticity Models).
• John Hopfield 1982: attractors in Hopfield nets. (Hopfield network). Inspired by Enst Ising's model of magnetism. Fascinating concept: Explaining magnetism was a theoretical / philosophical breakthrough, it explaining the concept of emergent properties. The energy is a precise theoretical notion, representing how stable the state of such a network is. Emphasis on pattern completion: Imagine a ball falling down the energy slope into an attractor. If you put the ball anywhere on the slope of a valley, it will roll there. That is called associative memory. Observe that attractors are points in high dimensional spaces.

• Valentino Braitenberg and Günther Palm 1977:

  If cell-assemblies are the terms of the cortical logic, there must be a device for the discovery of the ignition of a cell assembly and there must be the possibility of extinguishing cell assemblies. There must be a timer that regulates the succession of activated cell assemblies. There must also be a mechanism of attention that inhibits entire regions of the cortex once something interesting has ignited somewhere. Finally, there must be mechanisms that connect the rather cool cortical memory by correlation with the more impassioned system of inborn patterns of behavior, which is also subject to learning of some sort.

  Emphasizing that the brain must make use of the network and it's activity. If the activity is representing the pieces of mentality, there must be devices around the cortex, that are creating and orchestrating the activity.

  Cell Assemblies are heterogenious: They have multiple centers (attractors). Increasing the excitability of the substrate (globally), will ignite the halo of the assembly. Neurons that are associated with the assembly, but not part of the center.

  Decreasing the excitability will find the best connected center of activity. This thought pump mechanism is a member of a class of activity management devices that we can consider.

  These can then produce trains of thought, thought sequences, or refinements. The network and its orchestration can be in a dynamic state of refinement until it settles into some Hopfield attractors. But perhaps more often than not, the situation will have moved on by then, creating yet again the next evolution of the situation. (On obvious failure mode is getting stuck: Speculations On Striatum, Behaviour Streams).

  This query and result, this vague idea to a detailed idea, might the the fundamental eval-apply of brain software. This would mean that the means of abstraction (how to make procedures) in brain software are the same high-dimensional data structures again. (Context Is All You Need?). You can see how this makes sense by considering the state of the network (the situation) determines which activity is supported by the network (the interpretation). A vague idea is then allowed to be a query, where the pattern complete of the framework is the result. Or what Braitenberg labeled the trains of thought. The possible continuations of a state of the network. (This is a way to get to thinking, acts of attention, mid-term memory query lookups and so forth , A High-Level Story for Midterm Memory Retrieval).

  It would be fair to say that the halo of an ensemble then is something like a cloud of possibility. This makes the ideas sticky playdough puzzle piece balls with solid cores and doughy outskirts. Since the thought-pump mechanism takes 2 neuron timesteps, far-reaching thought jumps are done fast (because parallel).

  Also note that we can consider assemblies of different shapes and sizes, some with temporal structures that can represent sequences or plans. Some timeless, representing images.

  Up to the complete ensemble, which is the activity in the whole brain, given a time window. This is a framework to consider the whole brain (and its connections to the world) at once with the perspective of the activity flowing inside it.

  (Tübinger Cell Assemblies, The Concept Of Good Ideas And Thought Pumps)

• Moshes Abeles 1982: Synfire chain.

  Emphasizing the notion that synchronous activity could travel together through the network.

  Like dominoes made from clouds of associated neurons.

  Also, the electrophysiology makes it hard for an activity to stay. This must be fundamental, to solve epilepsy.

  A compositionality machine realized by a hierarchic architecture of synfire chain.

• Rafael Yuste (also 2): neurophysiology, finding ensembles and some of its properties.

  Finding that 'artificial' ensembles can be created by activating groups of neurons via optogenetics.

  That they are not created by Hebbian Plasticity in Cortex.

  That single neuron stimulation can alter the behavior and presumably the perception of mice.

  Learning to play 'the piano of the brain'. [see Yuste talks below].

• György Buzsáki (4):

  -—

  👉 actually, it turned out that Buzsáki was so relevant and cleaning up the conceptual notion of cell assemblies so thouroughly that my 'explore ensembles' project changed to 'understand and model Buzsákis ensembles'. This is in some ways the end of the notes page here; And the beginning of a new story arc. Latest HDC stuff here (Since Buzsákis reader centric Cell Assemblies is the same as Pentti Kanerva (1988) address decoder neuron, we are liscenced to model atomic ensembles as hypervectors).

  -—

  What a cell assembly is depends on the reader and the time window. With this perspective, neurons count as cell assembly, even if they don't activate each other, but when they contribute to the firing of a reader neuron.

  Neural Syntax: Cell Assemblies, Synapsembles, and Readers

  Cell assemblies do lateral inhibition via inhibitory basket cells. Implementing an Alternative mechanism.

  Cool neurophilosophical and empirical stuff.

  Neurons have intrinsic firing, this allows the internal ideas to be matched via action/experiment to the external world. (The brain always already has ideas about the world, it is not an empty receptacle that is filled with sensor data).

  A gamma cycle can be seen as a letter in the neuronal code. This is what I call an atom, or elementary data structure, which is composed in composite data structures.

  This would mean the atomic data structure is encoded in a 10-25ms time window. This overlaps with many physiological constants. Like the integration time of pyramidal cells and things like this.

  Then, there must be some scheme for composing the letters (the means of combination from 5) Perhaps something like Composition / Neural Syntax.

  [hippocampus, mental navigations, thought and externalized thought (what I label technology and cultural memes), hippocampal sequences are already there, which fits the 'the ideas must be first' idea.]

  [I was wrong to label the hippocampus a flash drive, it is an active element, providing pre-allocated trajectories to the neocortex, useful for both past and imagination and so forth]

Forgive me for saying it so but the neuronal ensembles are theory of mentality.

If sensor input is reliably activating some of these internal notes, we can say that the notes stand for the sensor input. The system now has the ability to epxress the world in terms of it's sensor inputs. Note this includes the effectors of the system, too. In this way action and perception together are allowed to represent expections of the world. If you move the eyes here, you will see x, and x has a color, too.

neuronal ensembles, built by the coactivation of flexible groups of neurons, are emergent functional units of cortical activity and propose that visual stimuli recruit intrinsically generated ensembles to represent visual attributes.

Visual stimuli recruit intrinsically generated cortical ensembles

Then, you reactivate them during sleep and so forth: Visually evoked neuronal ensembles reactivate during sleep

By activating ensembles, you can change the animal's perception (the most parsimonious explanation of the stuff Yuste did here):

Playing the piano with the cortex: role of neuronal ensembles and pattern completion in perception and behavior

In these experiments, they were able to trigger a learned behavior in mice by activating single neurons

ensembles appear necessary and sufficient to trigger visual perception

Great Yuste talks, summarizing the field and its history:

• Role of neuronal ensembles and pattern completion in perception
• Can You See a Thought? Neuronal Ensembles as Emergent Units of Cortical Function

Cool overview of current neurophysiological topics:

• Recording of Neuronal Ensembles 2022

—

I get the impression that systems neuroscience is discovering, or already contributing new perspectives, on the neuronal ensembles in the form of thalamocortical loops (and similar modules). The common language there is then the 'attractor states'. Input Circuits And Latent Spaces / The Games of the Circuits.

formalisms that describe the layer:

• Observe that per definition, the ensembles layer emerges from the neuronal layer

Names: Neuronal Ensembles or Chains (Lorente de Nó), Loops (perhaps resonates with software developers), Reverberations, Synfire chain (Abeles), self-exicitatory subnetworks, surviving activity, activity replicator, perception state, mental element, Cell Assemblies (Hebb), Neuronal Assemblies (G. Palm), attractors (Hoppfield and nowadays dynamical systems people), thalamocortical loop (as an instance of one), The notes that create the symphony of the mind, The alphabet of the brain (Yuste), a datatype (Vempala), a high-dimensional datastructure (Kanerva would agree), a pattern completion or associative memory datastructure implementation

This theory of mentality says what is the mechanism that is instantiating the computation of the brain, but a computation is a mechanical instantiation of something abstract. It says what are the primitive data structures and so forth, but it doesn't say how the brain is using those data structures to create explanations and navigate the world.

We label, rightly so, the mind or cognition as the software of the brain. This software needs to self-assemble because there is no programmer at hand.

I would argue that the brain's function must be this software and its entities. Otherwise, we would not feel like cognitive users running in a virtually created world with body and mental affordances.

What are the principles of self-assembling software? What kind of self-assembling could create something like our cognition machine? What kind of mechanisms does the brain need to implement in order to facilitate the growth of such software?

+-X--------X-+  random orchestration
+-----X------+  random commitment
 +----------+   i.e. random flash drive lookups,
 |   X      |        random acts of attention, ...
 |          |
 |    X     |
 |X      X  |  intrinsic firing rate
 +----------+

memetic engine

• Hypothetically, you add an intrinsic firing rate and tada, out comes an 'everything possible' machine; The memetic engine.
• Assuming (plausibly so) that orchestration, commitment and so forth are made from neurons, they all can be a little bit possible.
• I.e. your neurons all fire randomly a little bit. If you have an attention nucleus made from neurons you will also pay attention in random ways a little bit. (not saying it works like that, but then again, TRN the gate is sort of doing such a thing.)
• Philosophically, there is something deep I suppose: There is an activity that comes from the sensors, and then there is an activity that comes from yourself. From this, the system has the chance to grow a relationship with the sensors (and effectors), inside its high-meaning spaces.
• We can see this by imagining a neuron firing randomly. Usually, this doesn't do anything because there is no support in this space of meaning from the sensors and the network. But if there is an ensemble in the vicinity of a sensor neuron, the neurons firing in between allow the ensemble to 'shift' and represent the sensor inputs. And so forth.
• In other words, we make the substrate more dynamic, so that the ensembles flow around and have the chance to represent different things.

Observe that we have crafted a substrate with wires and gadgets, that is eager to represent sensor data because the data structures that have support from the activity of the sensors are the ones that are on.

From the perspective of the memetic engine, we can see that there are different 'complete ensembles' at each time in the system. I don't know why but I like to label them activity flows in this context.

+--+--+    +--+--+   +----+          derived meaning level
|  |  |    |  |  |   |    |          i.e. cortical 'association areas'
|  X  |    |  O  |   |  O |
|  |  |    |     |   |  | |
|  X  |    |  X  |   |  X |
|  |  |    |  |  |   |  | |
+--+--+  , +--+--+ , +--+-+ , ...
                                     sensors
 activity flows

             ^
             |
             |
             +--------------

              Darwinian wires
             ++ using heuristics

We can observe that whatever activity flow is stable, will be more around. And whatever activity flow is more around affects how the network grows (via the plasticity of the network).

Perhaps somehow a complete 'harmonic'(?) interpretation of the sensors is most stable. Something that stretches meaning and sensor level and makes a good explanation structure. I.e. the meaning levels will want to be as stable as possible, and this is useful because they will represent actual causal structures of the world by doing so. (Getting A Visual Field From Selfish Memes).

If the ensembles have all the chance to inhibit their alternatives, (via thalamic layer 6 neurons), whoever is fastest and most competent, wins. Because they go and inhibit their alternatives. I.e. whoever is active will try to make the machine go into a stable attractor state with no way out.

It is precisely the job of the machine (of the genes crafting the machine) to even the playing field. For instance by building in a restlessness, that forces the animal to move now and then. Or by having attenuation on the substrate, so the ensembles are forced to play longer games, in sequences. (Attenuation).

There is one other bag of tricks the genes have available: They can put some hardcoded heuristics like 'warmth is good', 'Mum smiling is good', 'cold is bad', then if you have a hypothetical nucleus (or Darwinian Brain) that can give a value judgment, you can build reward mechanisms. A wire that is "on" from some reward situation is what I call a golden wire. (Also: joy lines).

Golden wires can become about the machine itself. For instance, if it is possible to detect a situation where the system was making an explanation structure parsimoniously, in a short amount of time or with less amount of excitability perhaps, then this situation can be rewarded, too. (Why Flowers Are Beautiful, Elegance, Explanation, Aesthetics)

Whatever the rules are, we can analyze the entities in terms of abstract replicator theory [Dawkins, Deutsch]. It is whatever activity flows are stable and successful, including pleasing the reward heuristics, that will survive the most.

These activity flows will have a structure and function of their own. (I say they are a new kind of biology).

Names: everything possible machine, conjecture maker, variation maker, idea mulitverse algorithm,

[vague ideas]

The activity flows of the meme engine have an inner logic. Why are they more stable than other flows? Because for every interpretation the network represents, they are supported more than other flows.

We can say that the complete ensembles are competing for stableness and connectedness. Ensembles are allowed to have temporal structure, so their activity is allowed to flow across neuron timesteps. That is when we bring to mind the sequence "A, B, C, D, E, F, G,… ", even better if together with the melodie (more association, more support from the network).

Observe that we can say that such an ensemble has a starting point, the A and whatever notions are preceding it. Igniting the ensemble somewhere in the middle is harder, presumably because there are not many associations leading to this situation.

The complete ensembles

1. Are made from harmonic and symbiotic sub-cell ensembles
2. Stretch meaning and sensor levels (per definition, they are the maximal subset of the maximal ensemble active). (Another way to imagine this is to say it's the complete activity flow in the brain at any moment).
3. They are temporarily allocated interpretation represening machines, that do whatever they can do in order to be stable.

(also note that they can be strategic, they might play a game that goes across them being extinguished for a while).

How are you stable in this system? If the activity comes from the sensors, you represent things that are true across sensor changes. If the activity also comes from reward wires, you incorporate the reward wires, presumably by navigating the animal or the mind into a hard-to-fake reward state.

Machine-level mechanisms can shape memetic landscapes:

For instance, if we reset all or some neuronal areas now and then, we force the ensembles to ignite within shorter periods. In other words, we cut away the slow ensembles in the ensemble exploration space.

                                         The interpretation game


                                                                    +-----+ +----+
                                                                    |     | |    |  'commitment structures'
                                                                    +--+--+ +----+   user level
                                                                       |       |
                                Golden elegance wires?                 |       |
                                                                       |       v
                                      --------------------->           |       ----------- expression interface(s)?
                                reward wires                           v       |
                                                                               | ... allowed to be arbitrarily multilayered
                                                                               v

+--------+                                                         +-----+     +-----    causality structures,
|        |                                                         |     +-----+         explanations
|        |         everything possible a litte bit                 +-----+     |                          |
|        +--------------------------->                                         |                          |
|        |                                                                                                | abstraction drivers
+--------+                                                 [.]         [.]         [.]                    |
   memetic engine                                           ^           ^           ^                     |
                                                            |           |           |                     |
                         +----------------------------------+-----------+-----------X----------           |
                         |  ?                               |           |           |                     v
                         |                                  |           |           |
              +----------++                             +---+----+ +----+---+ +-----+---+
              |           |                             |        | |        | |         |
              |           |                             |   A    | |   B    | |    C    |
              +-----------+                             +--------+ +--------+ +---------+ , ....
            dream mode                                     situations


                                         whoever is active across many situations is representing something abstract

Presumably this is Popperian: Some random ideas that fit the network a little bit start to exist (conjecture), then if they are good ideas, they will stay stable across situations. Otherwise some other ideas will survive instead. The same process should apply to ways of thinking (Minskies term), ideas that create situations for other ideas. (Like ecosystems).

Levin asks us to imagine a spectrum of fleeting ideas that come and go, then some ideas that stay a little and make some niche construction, and at some end ideas that spawn other ideas. There is no distinction between thinker and thought in such a paradigm (see Levin being interesting again). From one perspective, you see a thought with some internal structure that is spawning other thought, from another perspective you see pieces of personality and so forth. And these things are allowed to be on continua and so forth.

These ideas by Levin fit well into my framework here, where I say that the brain is implementing a 'stable information' milieu. That computational paradigm is made from datastructures that are stable. I.e. the neuronal ensembles are both the data and the situations for further data processing (Context Is All You Need?).

—

Perhaps put to much emphasis on the reward wires on this page. The reward wires can only ever do 2 kinds of things:

1. Give a value judgment for situations
2. Reward or bias the activity based on machine-level heuristics. (Perhaps Good ideas from Tübinger Cell Assemblies, The Concept Of Good Ideas And Thought Pumps)

This cannot take the content of the ideas into account. It must be something that counts the same for all kinds of ideas. Darwin's hand could only ever build a machine with a propensity for creativity.

My thinking now is that the machine cannot help but make explanations of the world and the reward wires are secondary to this process.

—

• dream-mode
• dream mode perhaps picks some 'salient' or 'unresolved' situations (with some Darwinian criterion).
• Perhaps it simply mixes a bit of random perspective, orchestration with some situations from mid-term memory.
• If we then run the machine like usual, we find stable ensembles
• And the ones that are stable across many situations are the abstract and general ones.

Everything should be as simple as possible, but not simpler.

Albert Einstein

What are the principles that make good languages?

                                             +--+     user level
                             +----------+    +--+
 competence drivers          |          |
 abstraction drivers         |          |
                             +-+--------+
 ------->  ------->            |           ------ ------- ------ the user interface
                               v
layers below,                  |            +-+
development                  +-+---------+  +-+
                             | v         |           'common sense'
                             | |         |  +--+      affordances
                             +-+---------+  |  |
                               |            +--+      language level
                               v

The point where the magic comes in I think. The science of abstraction and elegance doesn't exist yet, but we see it in software design and so forth: There is something about minimally explaining something, also called elegance.

This 'minimal but sufficient' description is the meaning of Einstein's everything should be as simple as possible, but not simpler. This is a way to do theoretical physics as far as I can see. It is to take a range of phenomena and find a fundamental plane of explanation, where a minimal but sufficient amount of causality structures explain, in terms of common sense perhaps, how the world is.

We see it again in the genetic toolkits of evo-devo and so forth: There is something about expressing a complicated thing in terms of a lower layer of building blocks, that is powerful. This power is to some extent subject to the philosophy of programming. See Magical Interfaces, The Principle of Usability.

But the real science that will talk about it will be something deeper. A science of design and explanation. I don't think it would be wrong to call it 'cybernetic epistemology' A core idea of cybernetics is the idea of 'the interplay' and the view from above. Saying how the world is from a counterfactual plane, what the purpose of the system is and so forth. This is an obvious thing for me when you consider biological systems, the organism has a purpose and a function, and its structure will approximate a solution to the problem. That is the function doesn't exist, it is a counterfactual, but it is the most relevant part of the explanation of the organism. I think this notion of 'how building blocks fit together', and this 'harmony' are cybernetic topics, too. Also, I feel that David Deutsch is an early contributor to this science, and he highlights the importance of knowledge. Perhaps Constructor Theory will help with coming up with some theories of this science.

What are the principles that make good languages? Brain software must explore the space of possible good languages and design. From this reasoning, we see clearly that the analysis brain software is a special case of an analysis of whatever this 'elegance' and 'succinctness' in explanation structures is.

Either way, we can use some solid common sense biological cybernetics reasoning, mixed with knowledge of programming and interface design to at least describe this layer of brain software. And we have in some ways succeeded, too. The problem of the mind turned out to be deeper, one that will unify all explanation-making and art (See David Deutsch: "Why Flowers Are Beautiful").

Names: 'Epistemological cybernetics', 'Constructor theory of elegance', 'The science of elegance', 'The science of harmonic building blocks', 'Design Theory',

I conjecture that brain software grows an explanation language layer, which I label 'common sense'. This common sense can talk about objects, places, people, causalities and so forth. Its implementation in the brain is of course the neuronal ensembles, the high dimensional data structures that want to be stable. The logic of stratified design creates roughly a polarity of user and producer entities. Where producer entities have memetic drivers to be more general, harmonic and useful, and user-level entities have a driver to be less concerned with the details, compose useful languages with elegance, and strategically commit to plans and actions which will help them stay "on". Note that they don't need to know how actions work in detail, that again is handled by producers of the magical interface.

Turns out the best way to navigate the world as an animal, or be a human and make explanations together with other humans, is to simulate a world and a cognitive user inside it. This has the advantages of stratified design, a user level can compose a producer level. At each point of this cognition machine, the living software can improve itself. Leading to less leaky abstractions, a machine that just works, magically so.

the cognition machine

+------------+
|  +--+   ++ |         user level entities:
|  |  |   ++ |         - confident, wizards, use magical interfaces, competent, confabulate, demand
|  +--+      |
|        +---+         producer level entities
+--------+---+         - elegant, general, easy to use, living software, can make explanation structures

                       situation analysis
                       -  the Darwinian brain still binds this system into to the world,
                          makes it care about food and mates and so forth

The science describing the layer

Note that it is explained in terms of the lower layers. Consequently, I don't think that a formalism on this layer would be meaningful, it would just be advanced computational cybernetic psychology, taking into account the user and their interface.

• Memetics
• grown-up neuroscience
• psychology
• cybernetic psychology, or 'robopsychology' when it is for robots (that is Heinlein and Assimov's names respectively)

We easily see that coalitions of memes are implemented by associated or composed neuronal ensembles.

My hunch is that neuronal ensembles that are active together are symbiotic by default unless they are alternatives to each other. Then see Contrast And Alternative.

Note that you can always increase the time window, and you will have stretched what counts as single ensemble.

[ insert how the complete neuronal ensemble overlaps with the survival coalition we might call a self ]

[ not because we were looking for a self, but because we think about explanations of the mind and its biology and find entities that correspond, after the fact ]

From ct-life (Chiara Marletto).

A physical system M instantiates information if it is an information medium in one of its information attributes (belonging to some information variable S) and if the task of giving it any other attribute in S (allowing for waste) is possible. This intrinsic, counterfactual, property of M is an exact physical requirement, that certain interactions be available in nature

Information in constructor theory is defined by the counterfactual that 'it can be otherwise'.

The bits in the computer are only useful because they can be different bits, too. All permutations of of the information variable are possible.

This works in the genetic code, where information medium is DNA, the information variable is the set of information attributes Σ (sigma), the bases A,T,C,G.

Perhaps the property that all permutations are possible could be called permutativity.

But elsewhere on this page it might be labled composability. And I think it ties together with composotionality in linguistics.

It would lie at hand to assume that neuronal codes should have this permutativity property.

That everything possible to say is also "possible" to say.

And it would be a sufficiently relevant property of the functioning of brain software, that one might expect considerable energy being diverted to keeping it so fluent.

This is why my suspicion falls onto sleeping and dreaming, which takes up a significant portion of time, yet it seems like everything with a brain also sleeps.

Whatever it is for, it is fundamental in the functioning of brains.

And the property of dreams, where it seems like all kinds of possible personhoods and possible situations, possible waking dreams (in a funny circular fabrik of language) are being instantiated.

Perhaps one reason is to make the machine well oiled? To make it so that it can still say (express, represent) different kinds of personhoods?.

Because fundamentally the power of information lies in what can be said, not what is said.

WIESEL TN, HUBEL DH. SINGLE-CELL RESPONSES IN STRIATE CORTEX OF KITTENS DEPRIVED OF VISION IN ONE EYE. J Neurophysiol. 1963 Nov;26:1003-17. doi: 10.1152/jn.1963.26.6.1003. PMID: 14084161.

Perhaps Cortex doesn't actually do Hebbian Plasticity.

• Intrinsic excitability mechanisms of neuronal ensemble formation
• https://youtu.be/VzO8e_f2Hk8?si=XeWb1kw0N4eD9Wx_ (with a talk by Rafael Yuste talking about the iceberg ensemble model).
• Non-Hebbian Synaptic Plasticity Induced by Repetitive Postsynaptic Action Potentials

• Neural Assemblies, Guenther Palm 1982
• Braitenberg On the Texture of Brains 1977
• Santosh S. Vempala, which is a computer science flavored version.
• Rafael Yuste: Can You See a Thought? Neuronal Ensembles as Emergent Units of Cortical Function
  • This is a modern neurophysiology lens. When I look at this, I feel like it looks like Cortex is implementing a 'hebbian substrate', with cell assemblies as a playdough, sticking together.
• I publish my large code comments on assembly explorations:
  • Wavemaker (Experiment)
• I have to check this out: From basic network principles to neural architecture: emergence of spatial-opponent cells R. Linsker models some Hebbian plasticity and got orientation columns out of that.

Zebrafish larvae are translucent for the first time in their life.

Scientists at the Howard Hughes Medical Institute studied live zebrafish larvae that had been genetically encoded with a calcium indicator called GCaMP5G. They suspended the larva in a gel and then beamed it with lasers. Just before a neuron fires, its action potential is expressed via a spike in calcium ions, so when one of the genetically modified larva's neurons reached its action potential, it glowed. This showed the researchers the firing of the neurons without them having to attach a bunch of electrodes to the fish. Over the course of an hour the researchers used laser beams to scan the larva every 1.3 seconds, exciting the retina of the zebrafish with each scan. This microscopy method allowed the researchers to record up to 80 percent of the fish's 100,000 neurons at single-cell resolution. This is the first time scientists have recorded such a high percentage of an organism's brain activity at such a high resolution.

Funny, there is a meta-phenomenon to this video: There is a huge spike in youtube 'most replayed'. At 00:18, hard to miss, a huge flash of activity through whatever that big middle part is.

Maybe it means nothing, but our imagination is pumped. It thinks! The small flashes are just as interesting as the big flashes. Neuronal flashes would make sense in light of thought-pump mechanisms.

It is perhaps counterintuitive if this is a thought pump we are looking at, a big flash might be more something like a re-orienting. It is that the current interpretations are not so interesting, useful, or make enough sense, then it makes sense to lower the threshold a lot of a moment. This corresponds to searching in a much wider meaning space radius for a new interpretation, presumably because the current one is insufficient.

Whether those are thought pump oscillations I cannot say. Since we see a frame every 1.3 seconds, this might flash like this all the time but we just did not have luck seeing it.

—

Perhaps thought-pump mechanisms are layered and come in local and global varieties?

—

Funny, those zebrafish guys analyze the activity in terms of cell assemblies: Neural assemblies uncovered by generative modeling explain whole-brain activity statistics.

They get these subnetworks, quite interesting. It has some philosophical overlap with the cell assemblies of Braitenberg and Palm that I describe here. But the methods are different. As far as I can see, they don't have an inhibition model; Which is one of the requirements of getting to these pockets of self-excitatory activation I talk about here.

See Wilson, Cowan 1972. It is intriguing to consider the 'population dynamics' of inhibitory and excitatory neurons. Maybe all you need is a 'bouncy' excitation control,

Maybe you get away with modeling such a bouncy threshold.

;; Imagine [I], the population of inhibitory neurons,
;; and [E], the population of excitatory neurons.

 ;;                                     P
 ;;                                     | (input from sensors etc.)
 ;;                                     |
 ;;                                     |
 ;; +--------+ <--------------  +-------+-+
 ;; |        |                  |       v |
 ;; |   I    |---------------|  |   E     |
 ;; +--------+                  +---------+
 ;;     ^
 ;;     |
 ;;     |
 ;;     |
 ;;     |
 ;;   inhibitory-drivers

If there is a lot of (sensor) inputs P coming in, your threshold control might automatically go into an oscillation mode.

Like pulling on a spring and then it bounces around. This might already be the mechanism to get thought pumps in the cortex.

—

I get an inhibition model with a few parameters:

• The bounciness, how much I oscillate.
• The set point of excitation, that I make the system go towards.
• Maybe a min and a max of excitation/threshold.

(Idea also from Braitenberg when he considered the thought-sequences).

Another thing you can put in the inhibition model is attenuation, this is an (inhibition) rule that says, 'if you were active recently, it's harder for you to be active now'. I.e. fresh neurons are eager.

This is one of the known properties of biological neurons, and it has a use in a thought pump mechanism.

In effect, we memetically select for 'freshness', at least temporarily. Attenuation is a memetic landscape concern.

Another way to put it is fresh memes are eager.

Consider: Verbal attenuation is very quick. I think I can say a word 4 times in succession and it becomes weird. ⁴² You can do this experiment yourself by speaking aloud and repeating any word at random. Unless you cheat in some way and imagine speaking the word in different contexts or so forth, after a short amount of repetition the word will sound weird. You might even have the feeling 'How is this even a word?'. 'This is strange'.⁴³

Almost as if we would experience now the syllables and the sound, but the meaning is gone. See Semantic satiation.

The neurophysiological interpretation at hand is that some neurons that represent the meaning of the concept, are now becoming attenuated. I.e. the meaning is hard to re-ignite, sort of gone.

There is a satisfying reason why the attenuation would be high in some language cortexes. And why we would model different neuronal areas with different attenuation parameters.

With high attenuation, you get 'guaranteed' thought-pump sequences. Since we make it harder for the same cell assembly to stay ignited. In a thought pump jump, we heavily bias the system towards settling down on a fresh idea.

If we consider that language is virtually useless, unless it has some kind of syntax. It seems like there would be an evolutionary driver then, to make language cortex with especially high attenuation.

[My vision is the activity flowing like an ameba, or the 'ball of highest connectivity' moving from place to place].

More conjectures:

I think perhaps the opposite driver would be found in the cortex that represents 'meanings that stay'. This might be the visual scene - objects stay around without their meaning changing from looking at them. I think I can look at an object for more than 10 minutes and still don't feel like it becomes meaningless. This is in a way the opposite challenge to models that simply say 'attenuation makes satiation effects'. You need to also explain why there are no satiation effects somewhere else, then.

—

The cell assembly configurations in a way are the attractor states of the system. (I am not a math guy, but it sort of sounds like something like that).

Either way, I want to give this a name conflagrant modes, which comes from the Braitenberg paper.

Names for the possible configurations of activity:

conflagrant mode, ignition modes, supported activity configurations, attractor states, activity flow states, interpretations, representations

"My conflagrant modes tell me this is a bad idea". (When you have a hunch, meaning that some of your meaning spaces support the idea this is a bad idea).

"My current interpretations are joyful".

Names for the cell assemblies that are around a lot:

stable meme cities, well connected meaning cities, stable ideas, ideas that fit many situations

"This just gave me a spark for an idea."

"You just ignited some interesting ideas in my networks."

"These ideas are so out there, I have to build an ignition infrastructure to support them first."

"Your perspective just made me flip into a completely different conflagrant mode."

—

Here is a 'rolling malus' implementation in the browser.

;; Here, I do 1 simpler that is just carry over an attenuation malus
;;
;;
;; 'Rolling malus' implementation
;;
;; 1. Every time you are active, your malus goes up.
;; 2. With every time step the malus decays.
;; 3. The malus is applied to the synaptic input divisively
;;
;; kinda simplest thing, 1-3 could also be more complicated functions.
;;

(defn attenuation
  [{:as state
    :keys [attenuation-malus attenuation-decay
           attenuation-malus-factor synaptic-input n-neurons
           activations]}]
  (let [attenuation-malus (or attenuation-malus
                              (mathjs/matrix
                                (mathjs/zeros
                                  #js [n-neurons])))
        ;; decay the malus from previous step
        attenuation-malus (mathjs/multiply
                            attenuation-malus
                            (- 1 attenuation-decay))
        attenuation-malus
          ;; accumulate the malus on everybody active
          (.subset attenuation-malus
                   (mathjs/index activations)
                   (mathjs/add (mathjs/subset
                                 attenuation-malus
                                 (mathjs/index activations))
                               attenuation-malus-factor))]
    (assoc state
      :synaptic-input (mathjs/dotDivide
                        synaptic-input
                        (mathjs/add 1 attenuation-malus))
      :attenuation-malus attenuation-malus)))

Code Here

Ah, the joy of seeing one's ideas inside the computer.

This playground is up here: Assembly Friends #2.

That is a random directed graph with some geometry. The geometry wraps so that at one end the connections go to the other side (like a torus).

There is a threshold device, too:

(defn rand-cap-k-threshold-device
  [numbers]
  (fn [{:as state :keys [synaptic-input]}]
    (assoc state
           :activations
           (ac/cap-k (rand-nth numbers) synaptic-input))))

;; ...

(rand-cap-k-threshold-device
                             [(* (:threshold-device-high
                                  controls)
                                 n-neurons)
                              (* (:threshold-device-low
                                   controls)
                                 n-neurons)])

That is a cap-k, going between 5 and 10% of the neurons or something; Randomly.

It is one of the simplest threshold devices you can come up with I think. There is a chance it already gets the job done.

It would not find G. Palms good ideas though. Those depend on a threshold, not a neuron count cutoff. Because a good idea would mean that you have a high threshold and a lot of activity at the same time.

The white box is a sensory field. Whenever one of the stimuli (colored balls) is in the white box, a subset of neurons in the neuronal area are activated (unconditionally) - the projection neurons of the stimulus.

Finally, the connections in the area are updated via Hebbian plasticity.

• All neocortex (vs. Allocortex, the evolutionarily preserved, old cortex) receives driving inputs from Thalamus nuclei in layer 4.
• All sensory inputs to Neocortex go through the Thalamus. Note that olfaction is the exception, it goes to Allocortex.
• Thalamus is in a strategic position to modify the inputs.
• LGN is the visual relay nucleus, MGN auditory and so forth.
• Relay nuclei make driving inputs to Cortex (v1,…)
• modulatory inputs are shaping activity, driving inputs carry information and are the key inputs for activation. (this distinction is a key contribution by Sherman and collaborators).

Cortical hierarchical arrangments:

conventional view, *wrong*:

                       cortico-cortical connections,

        cortex primary                  cortex secondary
    +-------------+                    +------------+
    |             |                    |            |
    |   ^     ----+----------------->  |         ---+-------------> ....
    |   |         |                    |            |
    +---+---------+                    +------------+
        |
        |
        |
    +---+--+
    |^  |  |   Thalamus, "Relay / Gate"
    ++-----+
     |
-----+  sensors

In this naive view, you say that the cortico-cortical connections would make information flow inputs up a hierarchy of (perhaps 'feed-forward') cortical areas. Say (v1->v2->Mt->…).

As far as I understand, Sherman and collaborates primarily established the concept of driving vs modulatory input, which leads us to this update:

updated: Thalamus makes driving inputs to *all* cortical areas,




                                         +--  cortico-cortical are not driving inputs
                                         |    (their role is something more subtle)
                                         |
          cortex primary                 |         cortex secondary
     +-----------------------+           |      +---------------------+
     |                       +-- ---  ---+  ----+>                   -+ --- - - > ...
     |          ^          | |                  |  ^             |    |
     +----------+----------+-+                  +--+-------------+----+
                |          |                       |             |
                |          |                       |             |
                |          +-----------------+     |             +------------> ...
                |                            |     |
           +----+------+                  +--+-----+-+
           |    |      |                  |  v     | |
           |^          |                  |          |        Thalamus
           ++----------+                  +----------+
            |
            |   first order nucleus         second-order nucleus
            |   LGN, MGN, ..                Pulvinar, ...
            |
------------+
  sensors

 --->
driving inputs

- - ->
modulatory inputs

Note that this importantly includes most of the Thalamus, which is left out in the above 'model'.

• Relay nuclei make driving inputs to the cortex.
• A relay nucleus is in turn characterized by its driving inputs.
• First order relay nucleus: Receives driving inputs from the periphery
• Higher order relay nucleus: Receives driving inputs from the cortex (layer 5)

So this goes in a zig-zag with thalamus.

It is crucial, I think, to realize that the information flow is in a zig-zag, but the processing flow is parallelized across the relay nuclei of the thalamus.

'Bottom-up' vs 'Top-down' is somewhat misleading from this view, they label a virtual information flow that goes across elements, but those compute in parallel.

Similarly, 'ventral stream' and 'dorsal stream' are virtual information flows, but the computation is parallelized.

'Conventional' view:

                 cortical areas

                                      ^
             +------+ +------+        |
             |   <--+-+--    |        |
             |      | |    ^ |        |
             +------+ +----+-+        |
                           |          |
  +-------+----------+-----+-+        |
  |       |          |     | |        |
  |  <----+--     <--+---  | |        |
  |       |          |       |        |
  +-------+----------+-------+        |

                                      bottom up
<------------------------------------


------------------------------------->
    top down

A series of (probably feed-forward), serial cortical areas. Where information processing goes in a bottom up fashion from area to area. Presumably, each area represents higher derived feature detectors. And this might be modeled by neuronal nets, although this is contested.

At the same time, there is a top-down processing with big open questions, presumably constraining the information processing.

Updated view:

             cortical areas            thalamic relay nuclei    ^ |
                                                                | |  Virtual top-down and bottom-up
              +--------+                                        | |  information flows
              |     c4 | <------------------- d                 | |
              +--------+                                        | |
              |     c3 | <------------------- c                 | |
              +--------+                                        | |
              |     c2 | <------------------- b                 | |
              +--------+                                        | |
              |     c1 | <------------------- a                 | |
              +--------+                                        | |
                                                                | |
                                                                | v


a  - input relay nucleus
b  - second order relay nucleus, ...
c1 - primary cortical area, ...

The information processing flow is parallelized over the thalamic relay nuclei, i.e. information goes from a->c1, at the same time as b->c2 and so forth.

Perhaps it makes more sense than to say there is a virtual information flow, and that goes top-down, bottom-up. Note that the information processing arrrows are allowed to be shorter, this is the point of parallel processing. It means that you can do more in shorter timesteps.⁴⁴

This model applies to ventral and dorsal processing streams, too.

My reasoning:

It's not clear whether it is the whole story, but the base problem of the (fMRI) so-called 'cortical areas' is explained by this. Cortex activation, presumably including the BOLD signal, would be driven by whatever thalamic nuclei they look at.

This fits with the observation from the infamous ferret routing study [Nancy Knawishers Lecture is nice]. If the primary auditory looks at LGN, you get vision and orientation columns and so forth in the primary auditory cortex.

The most parsimonious explanation I can think of:

1. Neocortex is relatively generic.
2. The neocortex will represent inputs, depending on what inputs come from the thalamus in layer 4.

It makes sense that evolution would hit on a more general-purpose computer and repeat its architecture over and over. Our computer chips are general purpose and there is a reason. It is easier to program. General purpose is more powerful than narrow purpose.

This doesn't rule out that some cortex is different from other cortex (see musings on attenuation above for a reason). What might be true in a weaker form is that all sensory cortex is relatively the same.

The strongest claim here would be something like: If you could experimentally swap say MT and Fusiform thalamic input circuits*, you would simply swap movement and face representing "cortical areas". Presumably without much loss of function.⁴⁵

*) pre-development

—

Questions:

(Got some of my questions answered by M. Sherman!)

• Do relay nuclei ever branch to multiple areas?

Yes, it’s common got thalamocortical axons to branch to innervate multiple areas.

• If the arrangement is relay nucleus -> cortex -> relay nucleus and so forth, isn't there one Cortical area at the end of this arrangement that doesn't have a relay nucleus as a target.

Good question. The arrangement is cortico-thalamo-cortical going up a hierarchy. This is feedforward. But we just published a study showing that there is a feedback version, so the highest cortical area would still project to thalamus. Evidence suggests that all cortical areas have a layer 5 projection to thalamus.

• Do relay core inputs ever converge on a cortical area?

Yes, but it’s not clear what this means. For instance, one example is LGN and pulvinar innervating V1. The LGN input is basically driver, and the pulvinar input, modulator. Whether this is typical of other convergent examples is unknown.

• Why have a V1 when the LGN is already representing the information?

LGN may crudely represent information, but it has very limited computational power. That’s why cortex evolved. A better question is: Why have an LGN? or Why not have retinal input go directly to cortex? We have tried to address that.

• You might wonder if that sheds light 'functional connectivity' [Dynamic functional connectivity]

  Does it mean that the data is replicated? Or does it mean that multiple different processing streams are interpreting the data in different ways? Perhaps it means that tightly synchronous cell assemblies form?

  Perhaps it is hard for architectural or memetic reasons to have multiple large cell assemblies in the same area. (Would on the inhibition model the brain implements, if it implements an assembly calculus). (Perhaps multiple cell assemblies in 1 area simply don't make much sense, since they merge). Perhaps that would be a reason to have multiple processing streams on the same information.

• Perhaps another element in the basic circuitry of the cortex is needed to explain it?

—

Note that the neocortex has 6 layers, allocortex has 3 layers. This implies that the neocortex evolved by duplicating the cortical layers, and by hooking up to the thalamus<->cortical organization. (Allocortex doesn't have layer 4-6).

This means that layers 1-3 are the evolutionarily preserved layers. Making it obvious again: You need to understand the thalamus; Neocortex is a thalamocortical concept.

• Thalamic nuclei are heterogeneous, we differentiate at least a matrix and a core system [Matrix-core]
• Matrix is less well understood and projects diffusely to the whole Cortex. It receives inputs from the striatum and subcortex [got this from another talk]
• Core is making the driving activation to the Cortex, projecting to layer 4 neurons.
• Core neurons receive glutaminergic inputs from the periphery

—

My musings:

Matrix would be a candidate for an Hyperdimensional vectors [hd/vsa] implementation. If this is true, you would be able to read out a hypervector from the matrix nuclei.

At the very least, the interpretation that Matrix is implementing some kind of 'perspective', 'context', or 'landscape shaper' mechanism lies at hand. Surely, it must globally modify the attractor landscape of the system.

One might muse that such an element would help create mid-term memory. In that case, some yet unspecified process would somehow make the hippocampus store hyperdimensional meaning points of a holographic cortex encoding. (see Statistical considerations on the cerebrum / Holographic encoding? for why). Another unspecified process would recruit neuronal activity from the hippocampus to the matrix, giving the cortex a context/perspective. From there, the cortex could then fill in the rest of what we call the memory of a memory. (more below).

• Thalamic Reticular Nucleus (TRN) is like a blanket around the thalamic nuclei

      +-----------------------+
      |     ^  C              |  Cortex
      +-----+-----------------+
            |
            |
      +-----+-----------------+                     3)
      |     +->B    |         |  TRN  <-------------+
      +-----+-------+---------+                     |
            |       | 2)                            | drives TRN and inhibits Thalamus
      +-----+-------+---------+                     |
      |  ^  | A     _         |                     | Serotonin++
      +-----------------------+                     |
         |                  Thalamus   |------------+
---------+                                          |
 driving inputs                                     +---[brainstem etc.]



---------->
 modulatory inputs, cortex, subcortical


A - Relay nucleus driving neuron, makes projections to layer 4 in Cortex.
B - All relay neurons leave collaterals in TRN
C - layer 4 input neuron

2) - TRN inhibits back to Thalamus
3) - Neuromodulatory inputs from the brainstem and so forth are making strong inputs to TRN,
     also, Serotonin modulates the TRN

The TRN is a regular sheet around the thalamic input nuclei. The evolutionary drivers for TRN are easy to understand. It is an input gate. With low inputs from 3 (brainstem neuromodulator makers), in awake states, the gate is open. With high 3, in drowsiness [this is established experimentally with cats and so forth], the gate is closed.

TRN is controlling 1:1 how much activity flows into the cortex. This was one of the basic requirements you have for an activity management device, it controls its inputs.

—

So I am imagining a blue (blue are the sensors on my whiteboard) input nucleus ball, the ball receives inputs from the sensors, and represents the faithfully (drivers), there is a dynamic lid around the blue nucleus, that allows the machine to say how much of the activity flows into the thinking goo around.

—

Serotonin: At first glance the role of serotonin is ambivalent, it has to do with attention (cognitive neuroscience term), but also with drowsiness and sleep? [Lecture 8. The Thalamus Structure, Function and Dysfunction].

It lies at hand that closing the gate (activating TRN) is a mechanism happening during sleep. Shut down the inputs (mostly). This way the thinking goo can do something else, presumably it is relaxed from analyzing sensor data and performing as a behaving animal (broadly speaking).

Speculation: If we activate TRN, we shut down the inputs; Making the internal activity in the cortex less about the sensors and more about its 'own' context.

You can make this tradeoff presumably every time the sensor data is not that useful for whatever we want to do. Where the sensor data would somehow mess with the kind of information processing you are doing.

Note that it would be more useful if the default setpoint of such a gate is somewhere in the middle. This way, you could open the gate, and flood the cortex with inputs (awake, surprise?), or close the gate, and starve the cortex of inputs (drowsiness, perhaps thinking and imagination?).

I think we can assume this is a rather general element and operation in the system, useful for many kinds of mechanisms.

Of course LSD is a serotonin receptor agonist, and what is being reported from such altered states broadly maps onto what you might expect from the gate being closed, making the cognition be about itself - making the cognition about something like imagination perhaps.

From this, you will see that a maximally large dose of LSD will make you blind and deaf since the inputs going into the Cortex are maximally low, similar to a sleep pattern presumably.

It is ironic when substance users talk about opening their minds, while the mechanism that produces this state comes from shutting down the input gate to the world.

I suspect much more interesting and ultimately trippy thoughts are thought by building once understanding; Using pencil, paper, whiteboards and code editors as thinking aids;

—

Questions:

• Does TRN have any segments?

  [1 and 2 deleted for brevity]

• TRN is activated sparsely and allows fine-grained control by cell assemblies spreading into it.

  (Perhaps the fact that layer 6 neurons innervate TRN speaks for 3.).

The relay nuclei of the Thalamus have inhibitory interneurons [I], which receive inputs from Cortex [layer 6]. Layer 6 cells also make branches into TRN and they make branches back to input neurons at layer 4 - quite the puzzle! Let's concentrate on the [layer-6] -> [I] part for a moment.

At first glance there is something strange in the light of memetics, this circuit violates the basic tenant activate your activators. It inhibits its activator!

This is not so if we assume that all activity is sparse [Valid assumption afaik].

Consider 2 memes now, being activated by some input neurons A. Both memes go and activate some random I inhibitory interneurons, which meme is more successful? It is the meme with the random wiring that makes it inhibit all its alternatives.

naive:
1)
     +---------------------+
     |   [Ac]  [Ac1]   ^   | Cortex
     +--^-------+------+---+
        |       |      |
        |       |      |
        |       |      |   Bc is an alternative to Ac
        |       |      |
        |       |      |
     +--+-------+------+----+
     |  |       |      [B]  |
     |  |       v           |
     | [A]|----[I1]     [B] |
     +----------------------+  , where I inhibits [A] and [B]
                                 [I1] - inhibits [A]


actual:
2)

     +---------------------+
     |   [Ac]  [Ac2]     ^ | Cortex
     +--^-------+--------+-+
        |       |
        |       |        |   Bc's support is being inhibited
        |       |
        |       |
        |       |        |
     +--+-------v--------+--+
     |  |       [I2]---|[B] |
     |  [A]                 |
     |                  [B] |
     +----------------------+  , where I inhibits [A] and [B]
                                 [I2] - inhibits [B]


Ac1 - hypothetical meme inhibiting its own activator
Ac2 - (actual) meme inhibiting its alternatives

If there is only so much excitability to go around (threshold device, see above), the memes need to compete for well-connectedness. You only need to be the best-connected guy in the network, in relative terms, not absolute ones. (Consider cap-k as an inhibition-model).

Therefore, Inhibit your alternatives is the second basic memetic move, and reified in this cortico-thalamic arrangement.

Murray Sherman mentioned this, too:

There is a problem here…

Please imagine that this circuit is based on this [cortex cells innervating relay cells, and neighboring inh. interneurons].

And we have lots of cells that are doing this at a single-cell level. Whenever one of these cells is active, some relay cells will be excited, others will be inhibited.

However, the techniques we use to study this pathway are limited because we don't activate single cells and record from single cells. Instead, we excite or inhibit whole groups. Now if you look at the way this is organized, if you were to excite all of these [many in layer 6], even if you use things like channel rhodopsin, the result is that all of these [thalamic neurons] will be excited or inhibited.

… If you had pattern activation, you'd get a very different response down here.

This is an important proviso for most of the experiments being done. When we talk about knocking down a pathway, lesions or chemical ways of reducing all these cells [whole populations of cells all at once] and then even with modern techniques. We are not seeing the actual way this circuit is activating/behaving. And it's for that reason I think that it's very difficult I think to get clear evidence for what the effect of this pathway is on relay cells. Because all the experiments I know of with very few exceptions either excite or inhibit large groups of these cells.

And you wipe out the beautiful, detailed single-cell circuitry that really is playing the role.

M. Sherman about layer 6 cortico-thalamic circuits

If we activate some patch of cortex neurons, there is a problem. It is like making all the pistons go up and down together in a car, you don't see the actual arrangement of what this thing does in vivo so to say.

In this case, you will see the inhibitory neurons being activated, inhibiting Thalamus->Cortex inputs. But the important thing in this arrangement is the fine-grained flow of what the activity does, you cannot make a mass action circuit analysis and expect to not see a distorted picture.

You cannot look at a circuit without considering what happens when the activity is flowing inside it. And the activity is sparse.

Let me bring back the inverted pizza slice:

         -----------------
         \             /
          \     A     /                   meaning-level
           \         /
            \       / <----->   B
             \     /   inibit
              \   /
             --\-/---------------------
                X                        sensor-level


A - Stable cell assembly

Let's map this onto thalamocortical circuitry:

                                                      ....     .... n-relay-layers
                                                   +--------+                       +-------+
                                                   | H    1 |  secondary cortical   | H   2 | , .... 30+
         -----------------                         +--------+                       +-------+
         \             /                           +--------+
          \     A     /           meaning-level    | k      |  second order relay
           \         /                             +--------+
            \       /                              +--------+
             \     /                               | H      |  primary cortical
              \   /                                +--------+
             --\-/------------                     +--------+
                X                 sensor-level     | k      |  first order
                                                   +--------+  relay nucleus




A - Stable cell assembly

k - low dimensional neuronal area
H - High dimensional neuronal area

(Speculative)

There is an entity one abstraction layer up from the neurons and the circuits: The cell assembly. A cell assembly can stretch meaning and sensor-level neuronal areas. We can say it is a data structure, created by the fundamental operation of pattern completion.

From biological reasoning [could have been otherwise, but consider the anatomy], it makes sense to locate this sensor level at the first-order relay nuclei of the thalamus.

Assuming that 1 thalamic relay nucleus can have multiple cortical area targets, the information flow would fan out. [Reasoning from the existence of ventral and dorsal streams, how could this be made except by splitting the paths somewhere?].

Considering that there is much more cortex that is not primary sensory areas, we can assume that there is roughly something like the inverted pizza size shape; If you map a cell assembly to neuronal areas, the 'meaning level' parts of it are necessarily larger than the sensor level parts.

—

Further, this circuit supports one of the fundamental notions that we get from memetics. Via the inhibitory interneurons of thalamic relay nuclei. (Described at Contrast And Alternative).

Inhibit your alternatives, on every level of meaning.

         ----------------- -- ---
         \       \     /        /
          \     A \   /  B         ....
           \         /        /    k
            \       /        /     H
             \     / \             k
              \   /   \    /       H
             --\-/-----\------
                XA ---| XB       thalamic relay nucleus
                    ^
                    |
                    |
                    |
            A inhibits B on the sensor level


A - Stable cell assembly
B - A possible cell assembly, without enough support from the dynamic network to be active

You see from the cell assemblies clearly, whatever interpretation the system is expressing at the moment at large, i.e. which assemblies are stable. These will modulate and create contrast at the level of the sensor inputs coming in - down to first-order thalamic relay nuclei.

In this view, the thalamus is far from a simple relay. It is an essential component of the memetic landscape of the cortex. The memes in the cortex modify the inputs to the cortex. To fall into stable interpretations, which shape the inputs.

Why do they shape the inputs? To reproduce themselves.

The purpose of activity flow is to reproduce itself.

An assembly calculus will integrate multisensor information immediately, just as well as 'uni' - modulatory inputs. All you need are areas that mix inputs from multiple input streams.

This is a very important requirement for brain software.

From Sherman:

• driving inputs are glutaminergic excitatory synapses.
• We can say there are first-order and higher-order Thalamic nuclei, they are defined by their driving inputs.
• Every cortical area receives driving inputs from thalamic nuclei in Layer 4 (Neocortex).
• An example of a higher-order nucleus is the sub-nuclei of the pulvinar, they go to the 30-something visual areas in the cortex []
• A higher-order thalamic nucleus receives driving inputs from the Cortex itself, from layer 5 neurons.
• Curiously, all input neurons to higher-order thalamic nuclei make branching axons. The other branch seems to go to motor* centers. Asterisk because this was preliminary. Perhaps it is 'lower targets', not 'motor targets'.
• Why have this branching axon? Sherman speculates this might be the biological implementation of efference copies. Consider when moving one head, almost everything in the retina/sensors changes, but the brain somehow should keep track of this 'self-movement' and factor it out in its perception.
• You can see this yourself by pushing gently on the side of your eyeball. The world moves a little. The fact that this is not the case when we move our eyes shows that some processing is canceling out the changes in the world coming from one own movement. In order to do such an information processing, efference copies were postulated very early.
• Observe that the efference copy in this model is the one going into the higher-order nucleus. Since the other branch goes to motor centers.

      cortex 1                           cortex 2
      e.g. v1                            e.g V2

+----------+---+ 1-3               +---------+-------+ 1-3
+--------------+                   +-----------------+
|              |                   |                 |
| ^            | 4                 |        ^        | 4
+-+------------+                   +--------+--------+
| |          | | 5                 |        |       || 5
+-+----------+-+                   +--------+-------++
  |          |                              |       |        pattern repeats, up the higher order nuclei
  |          +---------------+ 5a)          |       +------------------------+
  |                          |              |                                |
+-+---------+                |    +---------+--+                             |
| |         |                |    |         |  |                             |
| |         |                +-+--+-->      |  |                             +------->
|           |                  |  |            |                             |
+-----------+                  |  +------------+                             |
   first-order relay nucleus   |   higher order relay nucleus                |
   e.g. LGN                    |   e.g. Pulvinar (1 of ~30 subnuclei)        |
                               |                                             |
                               |                                             |
                               |  5b)                                        |
                               |                                             |
                               |  motor* centers                             |
                               v  colliculus, red nucleus, etc.              v



5a) A layer VI neuron making inputs to a second order 'relay' nucleus, branch a.
5b) The branch b of the same neuron, going motor- or similar kinds of targets.

This is challenging to the conventional view, that cortical areas are somehow functionally specific. Here we have a circuit that implies that all cortex does at least a little bit of movement.⁴⁶

This fits again with the view that the cortex is made from relatively generic, general-purpose elements. Additionally, such an element would affect animal behavior. So it would be immediately useful evolutionarily. The hypothetical proto-neocortex ('single area') would go to behavior targets, presumably already from layer 5. The input organization of this cortex would evolve together with the cortex. The logic of the efference copy would be a strong evolutionary driver for making the layer 5 output neurons branch, leaving collaterals in the input nucleus. Eventually, the targets of these collaterals would have an evolutionary driver for specialization, making a new nucleus. The whole organization would now have a reason to evolve a duplicated cortex element, which analyses the latent space, the output of the first area.

Something like that, perhaps the order is swapped somewhere.

What is the role of the cortico-cortical connections? Probably they are important, too.

—

From the lens of the cell assemblies and the memes:

To analyze a circuit from the view of memetics, we can imagine a few different 'possible memes', let them play the game of the circuit in our imagination, and see who has the most success.

Making the biologically plausible assumption:

All wires are kinds of wires.

The kinds of pre-and postsynaptic neurons, how you branch, to which nucleus you go and so forth are allowed to be determined darwinistially, but that the actual fine-grained connectivity is simply random (at least at the start).

Let's say we are some memes that are active in v1, looking directly at some sensor inputs. So the activation that makes us active is from a first-order nucleus, coming from the sensors.

The prime goal of the meme is to stay active. If it can bias the system to keep the sensors the same, it will do so.

From the circuit above there are 2 games these memes need to play:

1. I spread into some layer 5 neurons, which make the appropriate motor output that keeps the sensors the same because this is where my activity comes from.
2. I want to spread into the higher-order nucleus because this is my primary way of influencing the rest of the Cortex to keep me active. I.e. a meme that spreads to the higher guys and has friends up top will be selected compared to memes that don't do that. I.e. cell assemblies that stretch across cortical areas have more connectivity.

So 1 and 2 are two different memetic drivers, 2) makes you want to spread into as many layer 5 neurons as possible, but 1) gives this a twist: The behavior stream you are outputting shouldn't change the situation away from the situation that made you active in the first place.

The cell assemblies now have a memetic driver to output a stream of possible behavior (at layer 5 neurons). A stream of possible behavior that keeps them active.

You can look at a line in your visual field, and the cell assemblies in your primary visual areas make outputs at layer 5 that correspond to eye movements, staying on that line in the visual field.

Whoever is looking at the higher relay nucleus now (the efference copies of those movements), sees an encoding of things like lines, color blobs, edges and so forth. In the language of possible eye movements.

This would be to a low-dimensional latent space between the high-dimensional cortical areas.

It would mean that the latent space of the cortex is encoded in behavior streams, which keep the sensors/situation the same.

Let's imagine some red sensor memes in v1 when we are looking at an apple.

Now this circuit forces the memes to play with open cards in a way I think. You want to spread into the higher relay nuclei and influence the rest of the system, but the circuit forces you to make some motor output at the same time.

Imagine meme 1, which has a few random connections to the motor output, making the animal move its eyes for instance. This is not a better meme compared to meme 2, which makes the eyes stay in the same spot -> My sensors stay the same, so I stay active.

But meme 3 is even better, it spreads maximally into layer 5, and it outputs a range of movements; A range of movement that will keep the eyes inside the patch; Stopping exactly at the edge of the patch.

I am imagining the following experiment:

• Make a monkey look at a red circle.
• Somehow read out the activity coming from v1 to motor targets (the branching axon 5).
• If you would run a decoder (or whatever you do), you would be able to read out a range of eye movements.
• For a large circle there is either a correspondingly small or correspondingly large movement range signal, depending on whether this is encoding the freedom or restriction of movement.
• Or, if it is a mix of such signals, you would have to decode more cleverly. Or perhaps setpoints are communicated, not procedural movements. Or a mix of those.

Alternatively, you would knock out all other motor pathways except v1; I think you would see eye movement. Small eye movement inside small patches of color, large eye movement for large patches of color. Line tracing movements for lines and so forth.

The memes that hit on the right connectivity to make movements that keep the sensors the same, will simply be selected in favor of the ones that don't. But the game is multilayered, the overall cortex context comes into play, too.

The memes that find the widest possible range of movement, which keeps the sensors/inputs the same, have a greater selection advantage still. Because it is those memes that spread most efficiently into the higher thalamic nuclei, and thereby have a chance to form cell assemblies with the higher cortical areas.

—

Ever since I came up with this idea, when I look at objects I wonder. Isn't there sort of a wall you hit when you move the eyes inside the edges of an object?

—

There is one other thing that comes to mind; Which is tactile mitgehen.

Alien hand syndrome Alien hand syndrome has been shown to be prevalent in roughly 60% of those people diagnosed with CBD.[6] This disorder involves the failure of an individual to control the movements of their hand, which results from the sensation that the limb is "foreign".[2] The movements of the alien limb are a reaction to external stimuli and do not occur sporadically or without stimulation. The presence of an alien limb has a distinct appearance in CBD, in which the diagnosed individual may have a "tactile mitgehen". This mitgehen [1] (German, meaning "to go with") is relatively specific to CBD, and involves the active following of an experimenter's hand by the subject's hand when both hands are in direct contact. Another, rarer form of alien hand syndrome has been noted in CBD, in which an individual's hand displays an avoidance response to external stimuli. Additionally, sensory impairment, revealed through limb numbness or the sensation of prickling, may also concurrently arise with alien hand syndrome, as both symptoms are indicative of cortical dysfunction. Like most of the movement disorders, alien hand syndrome also presents asymmetrically in those diagnosed with CBD.[7]

Corticobasal degeneration

Let's assume for a moment that in CBD the usual (frontal) motor pathways are impaired. Perhaps this is freeing up the memes of S1 and so forth, to make movement that makes the sensors stay the same.

If this is true, you would say that mitgehen comes from s1.

—

This idea doesn't depend on the branch b of the layer 5 neurons to go into motor targets. Could be 'chemical balance' or whatever nuclei, too.

—

I feel like cell assembly memetics can elucidate all mysteries of the circuits of neuroscience.

Consider the circuit, consider the memetic games of the circuit. Imagine a few alternative memes.

—

Memes know no boundaries, they will spread into whatever circuitry is available. If the circuitry forces memes to play a certain game, they will do so.

—

Claustrum speculations:

My reasoning: Claustrum looks like an element that implements some machine-level operation. It is orderly connected to everything else in the neocortex, stimulating it leads to loss of consciousness [Koubeissi et. al (2015] ⁴⁷. It is at least interesting that this would overlap with what we expect from the hypothetical threshold device (further up). Stimulating it would either confuse this element to think that there is a lot of excitation, which needs a high threshold to keep in check. Or it would directly excite the inhibition neuron populations - In both situations, we would expect the activity of the cortex to drop. Perhaps leading to unconsciousness. But this is just speculation.

Further, there would be a class of epilepsy that would all be some kinds of claustrum pathologies. The opposite might also happen if you have too much activity in the claustrum - epilepsy in the claustrum. If the claustrum is the threshold device, this would lead to profound inhibition of Cortex.

It might be otherwise, but what if the mechanism of Cortical spreading depression is some kind of epilepsy in some inhibitory nucleus? Whatever that nucleus is, it is powerfully inhibiting the cortex. But what in neurobiology travels at 1.5-9.5mm/min? Some kind of offending activity, spreading somehow in a direction that is usually prohibited by the circuit perhaps. Perhaps something usually goes 1000 that fast, but in the other direction?

(later)

Audio diary (stream of consciousness evolving ideas, better at 2x speed):

1. Magic Gem Epistemology of Meaning-Level Wizards
2. Musings On The Epistemology of Cell Assemblies
3. Small Summary And Musings How Context Is Implementing Procedures In Assembly Calculus
4. Speculations On The Mechanisms Of 'Good Ideas'

We can assume that evolution hit on a knowledge representing substrate. This substrate is populated memes - pieces of knowledge that replicate. This is only useful evolutionarily if this knowledge is useful.

It is fair to say that humans inhabit the cognitive niche (Pinker). Or something along the lines of the purpose of humans is to create knowledge (Deutsch).

The brain must support what I call an ad-hoc epistemology.

Alternative names: biological rationality layer, animal epistemology, living epistemology

I think there was/is a tension in neuroscience and computational neuroscience - that as a biologist you see a thinking machine, but what is the structure and function of this? The answer was it does computation. So now you say this thing does this computation and so forth. But this is like looking at a car engine and saying this does acceleration. I would say that you can replace computation with wizardry in a neuroscience textbook and you get the same information content from it.

What is missing is having a shape of the function in mind, because it is this shape, the rationale for what the structure should be, that allows us to satisfyingly explain biological systems. If the shape of the function is not perfused with an explanation model of what this should be, then you will always only see single pistons turning and gasoline flowing and so forth, you will see the trees but not the forest.

Now the cell assemblies are sort of a little bit 1 perspective on the intermediate level, the level of the language and primitives of the cognition software.

And this computationally intermediate plane is understood in terms of something living. The cell assemblies have a biology to them.

My philosophical view and agenda is to make the ideas come alive in our brains so to speak. Now these ideas have quite different evolutionary environments, landscapes, and pressures, they run on a computer, and they create software, that competes with other software creating agenda-having entities. They live on subsecond timescales in vast meaning spaces. Building an intuition around the properties of this world and its inhabitants is an aspect of explaining cognition in my view.

The mechanics and evolution of these little entities, their structure and function, are allowed to dissolve in our minds, the way that the structure and function of biological systems do.

But they are computational constructs, with the malleability to represent computation, knowledge creation, psychologies, societies of a kind and so forth. One perspective on them is how they construct building blocks, very chemical. Another perspective on them is how they make harmonious societies, very system-biology, ecology thinking and so forth. Another perspective is how they create knowledge, very epistemological, practical philosophical and so forth.

So from biological cybernetics we get this substrate, that is allowed to be put together so beautifully complicated.

This then is one perspective and candidate intermediate land and language between us and the neurons. This would consist of a computational framework (assembly calculus, thought pumps, hyperdimensional computing and so forth, something that says what kind of neuronal areas and nuclei you have, something that says what kind of attention mechanisms are available). The behavior of this system is described in terms of memetics. That is sort of alive, sort epistemological, sort of ecological, sort of software, sort of building-block making. It is psychological only in the sense of making sense of large-scale processes inside the system. The entities themselves don't have psychologies, they only have tiny agendas.

• How big are cell assemblies in this view? Tiny, or is there 1 at each time step in a brain?

  I suppose the answer might be it depends on how you want to parse it, and both ideas are good sometimes.

• How do cell assemblies relate to interpersonal (cultural) memes? (The ones from Dawkins and Blackmore).

  Cell assemblies are a brain computational construct, interpersonal memes are abstract replicators representing some knowledge of how to spread between brains.

  Both of these things are pieces of information that replicate and are subject to natural selection. Both of these can therefore represent useful information. This is interesting in light of the epistemology of the memes and memetic landscapes. (See David Deutsch's rational memes.) I think an analog of rational memes is the ad-hoc epistemology of the animal brain.

  It is intuitively obvious, how a piece of knowledge can spread interpersonally. Dawkins's tunes and habits of walking.

  It is required for such a meme to spread and be 'expressed' by the computer it spreads into. Any creative cognition machine is susceptible to such memes. (It must support it in the first place to count as a cognition machine in my book). This works, regardless of whether that cognition machine is implemented in terms of cell assemblies or something else.

  (But the landscape of available brains is the landscape of the social memes, so incorporating truths about certain animal implementation details of the brain you spread into makes sense. For instance, a meme might work by making you angry, but this same meme might not work on a Spok brain. In a world of Spoks, such a meme would be a bad meme. In the world of angry apes, it is.).

  The cell assemblies out of which a 'brain-phase' substrate of an abstract meme is made are different between persons (just to be clear common sense and neurophysiology apply).

  A social meme must navigate the messy biology and cognition of many brains in order to work.

• Mac Shine on thalamocortical circuits
• https://pubmed.ncbi.nlm.nih.gov/32795658/
• Diffuse neural coupling mediates complex network dynamics through the formation of quasi-critical brain states
• Eli Muller X Summary (Basically a graphical abstract)

https://pbs.twimg.com/media/F2ef-gVaAAIieKN?format=jpg&name=small

(from Eli Muller's X)

Speculations On Striatum, Behaviour Streams (perhaps should be called affordances streams)

—

Note that machine learning attention is a perspective mechanism (they gave it a slightly worse name in my opinion).

—

It is remarkable that via neuronal ensemble memetics, we see that something (initially) random would be so powerful.

This is very speculative… perhaps this is just a starting point and reality is crazier and simpler than this.. :

Perhaps it goes something like this:

     higher-order visual cortex
     +---------+
     |         |
     |     A   | stable representation of A
     |     +   |
     +-----+---+
           |
           | layer 5
           |                               primary visual cortex
           |                             +----------+
 +---------+                             |          |
 |         |                             |   +--+---+
 |         |                             |   | X| A |   <--------- [ retina data ]
 |         |                             |   +--+---+             3.
 |  +------+---+  1.                     +---^------+ + --+ -+       The overlap of expectation and sensor data ignites
 |  | ----     |                             |        | A |  |       the interpretation "A in the center of visual field".
 |  | -------  +----- ----------- ---------- +        +- -+  +
 |  | ----     | Expectation: A will be in the center of the gaze
 |  | x->a     |                               ^
 |  +----------+                               | ^
 |    pulvinar                                 | |
 |    (all kinds of possible eye movements)    | |
 |                                             |
 |    _                                        |
 |    | says go (by inhibiting the inhibitor)  |
 |    |                                        |
 |  +-+--------+                               |
 |  |  ?       +-------------------------------+
 |  |          | 2.
 |  |          | Striatum says 'go' to the movement and to the expectation at the same time
 |  +---------++
 |   striatum |?
 |            |
 |            |
 +------------v-----------> [ eye motors ]
                             X----------->A
                               move



                            virtual visual field
                +-------------------------------------+
                |                                     |
                |               X------->A            |
                |                  x->a               |
                +-------------------------------------+


X - the center of gaze
A - Object
x->a - eye movement that puts A into the center of the visual field

This would mean that there should be projections from pulvinar to v1 because the eye movement data from higher order cortex needs to go through a higher-order relay nucleus (from Sherman's thalamocortical circuits).

Something is weird in this model. If layer 5 goes to motor areas and higher thalamic nuclei, why does the go need the striatum? As far as I know, the striatum says to thalamic relay nuclei how much they are inhibited, not to motor targets whether they should go. Does that mean there is another element that is missing from this? It would be between the pulvinar and motor targets, it would be on when striatum says go.

A principle of the efference copy is at play here: It is exactly the power of splitting the message in 2, that there are different interpretations of the message possible. Movement by the motors, and the expectation of what to see at the primary visual in this case.

You could make the following experiment:

1. Make a monkey look at a visual field with nothing but a red blob in it.
2. Read out all wires that go from pulvinar to v1.
3. When the monkey has a blob in the periphery, you see wires being hot.
4. When the monkey moves its eyes to the blob, the wires are really hot for a moment (bursting?)
5. If you could, you would read out from those wires red blob data. You would see that these wires change the attractor states in v1 to see a red blob in the center of vision.

The fascinating thing here is that eye movement data is red blob data for somebody else.

1. ? Perhaps now the same wires say make little eye movements that stay on the blob. And More red blob and its edges or something.

• The cell assemblies of A want A to be in the center of vision.
• The cell assemblies in v1 presumably are happy to represent sensor data. There must be something in the logic that makes the sensor data be relatively even memetic players I feel like. Otherwise, how can we have this smooth transition when moving the eyes? I.e. the center of vision neuronal area is happy to swap from white cell assemblies to red cell assemblies.
• Perhaps this sheds light on hemispatial neglect, if the higher-order area I talk of is gone, nobody represents the objects to that side anymore.

• If you would experimentally knock out the layer 5 of where cortex, you would

  1. Prevent some forms of eye movements
  2. Disrupt the animal from understanding where, distance, spatial relationships

  (I realize this doesn't say much when we say disrupt where cortex).

A fundamental tool of replicator theory is asking Cui bono?.

—

Questions:

• What is binding the object representation with the where information?

  The answer might stare us in the face here: It is perhaps partly encoded in the combination of eye movement and expectation. If I move my eyes like this, then I see a red apple This represents the red apple and where it is located, the information is bound via the nature of the cell assembly prediction representation. (The continuation of this idea is below, Getting A Visual Field From Selfish Memes)

• What is the nature of the inner eye?

  Can I not zoom and move around in imagination states?

• Why are eye movements spared from sleep paralysis?

  Assuming the primary role of sleep paralysis is to prevent moving because it is dangerous. I think the answer most at hand is that eye movements don't matter. I.e. when you sleep in the trees, then moving your arms is deadly, but moving your eyes is not.

  Now one might wonder if eye movements are perhaps fundamental to cognition, attention, mental moves, and guided imagination. Then one would think it is obvious you should be able to move your eyes during dreaming.

  I don't think it needs to be this way. Why not implement a gate nucleus right between the eye motor muscles and the output nuclei? (Because eye movements are not dangerous I think).

• Whatever the hearing attention moving effectors, do they have all the same logic?

  I.e. those hearing attention movers, do they also originate from virtually the whole cortex? Are they spared during dreaming? Are blind persons using a hearing attention movement encoding to locate where information?

• Why do cell assemblies not cheat by representing the same object multiple times in the visual field?
• Is there any visual illusion where
  1. You see more of the same kind of object than is there?
  2. Do You feel that the same the object is in multiple places?
  3. You feel that you move your eyes but you see a different object than expected.
  4. You move your eyes and suddenly the distances between objects are different than expected.

I don't know of any of such illusions. So I think whatever the circuitry is, it doesn't make it worth it for the memes to trick the system in this way. Perhaps except 1, then I feel like interpretation is many of x. Like when you look at ants and you feel I see many ants.

—

Zooming and the 'local' relationships of parts of an object.

This made me think of a new kind of user interface arrangement. Instead of submenus that you flip through with page buttons, you have a zoomed-out view. Then you zoom into one part of the interface, and you get a higher detailed view.

The submenus align with the notion of places we visit.

The Zoomer menu aligns with the notion of the global scene and the local scene or something.

Pretend for a moment we are looking at a scene with open eyes, without head movement (the same logic will apply to head direction and so forth). For present purposes, imagine there is a where cortical area that encodes an eye position, in absolute terms. (It says eye coordinates).

Let's say there is a primary area to represent retina data, and some what areas representing color and object identity and so forth.

Observe the simple case:

• The user looks at a certain position in the visual scene, which means there is a cell assembly A active in the where cortical area. (The motor encoding for the position is active)
• [?] Auto associate the cell assemblies in primary (encoding sensor data) with A. This would for instance work with cortico-cortical connections and plasticity.
• Further, what represents an interpretation of what is being looked at in primary.
• All these 3 things what, where and sensor data are active together, and have a chance to associate in cell assembly stretching all 3, you might call it A-wws, for A where, what, sensors. I just call it A, even though it is not the same A from step 1.

                           where
                         +---------------------+
                         |  A                  |
                         |                     |
                         +---------------------+
                         | ^ A                 | layer 5
                         +-+-------------------+ eye position
                           |
                           |
                           |
 what                      |
     +------+              |
     |      |              |
     |      |              |
     |  A   |              |
     | A <--+------------->v^ (? thalamocortical wires? cortico-cortical wires?)
     +------+               |
     |color | ...           |
     |      |               |
     +------+               |
      ..                  +-+---+
                          | v   |  primary
                          |     |
                          +-----+


primary - primary visual v1
A - stable cell assembly, stretching multiple cortical areas

Should the user decide to look away from the object, the cell assembly A will simply stay, in where and what. In a meme world, A (and its meme friends in the rest of the network) will want it to stay active. And A will spill into primary again, given the chance. Perhaps if the user closes their eyes and pays attention to A, presumably A would have the chance to spread into primary a little again.

A says 'You can expect to see this if you move your eyes here'. Note that the where part of A, namely the activity of the where cortex, is a sub-part of the data structure of A. And parts of information ignite the whole; So should the user pay attention to 'this position', then the rest of the ensemble will re-ignite in what, and presumably a little in primary. Should the user move their eyes to this position, A-wws is re-ignited in full all its glory.

This logic will create a competitive environment of 'object memes'; All saying 'Look at me!'.

                          visual scene (virtual)
        +---------------------------+
        |                           |
        | +----+                    |
        | |    |                    |
        | | B  |                    |
        | +----+                    |
        |                 +------+  |
        |                 |      |  |
        |                 |  A   |  |
        |                 +------+  |
        +---------------------------+


A, B - object memes

Note that an object meme that does not say 'how' to look at it, is bad.

Memes want to be good interface entities.

That is, since there are user-level entities in the system (they are a necessary part of explaining brain software), every meme wants to be easy to use by the user. That is a little bit like A and B want to be great buttons, advertising what they can do to your mind.

Click me and your mind will change in this way….

How to click an object in the visual scene? It is hard to describe, but it is very easy to do. It comes across like deciding to move one's eyes, or deciding to look at an object, or perhaps giving in to the temptation to look?

So the object memes have memetic drivers to be looked at easily, which firmly encodes where, via an eye movement motor encoding. The memes say what and what to expect via their associations. That is simply what is part of the cell assembly. Remember that the cell assemblies pattern is complete; if the cell assembly looking at position x is pattern completing the redness of the apple and a little bit the apple. Then this is an expectation structure, and I think this is all we need in order to represent the notion that there is an apple at position x.

Binding is an 'expectation coalition' of temporarily associated, compositional data structures.

Further, memes will go out of their way to mean many things, to be implicated in more aspects or situations. If a meme can advertise a hunch of some potential interesting sequence of mental state, if only you would pay attention to it, it will do so.

Note how the dynamic part of the system is pushed to edges as far as possible via memetics:

 Cortex:

+-------------+------------------+
|             |                  |
|   assoc.    |    where         |
|     ?       |                 -+------  stable
|             |                  |
+-------------+-----------+------+
|                         |      |
|  what         |         |  V1 -+-------- dynamic
+---------------+---------+------+
                |
                stable

Note that the memes in what and where can stay on even though the retina data changes. It is only V1, representing retina data, for which staying on across eye movements would constitute major confabulation. Presumably, such an immense top-down influence on sensor-level representations is the mechanism of hallucination.

So V1 should be very impressible somehow by the sensor data coming from LGN and the motor movement messages, presumably coming from pulvinar.

As a healthy person, the edges of such a mechanism happen for instance when walking amidst trees. Some 'person' representing memes may be activated in the higher visual areas, giving the vague impression of a person in the corner of the eye. This problem is remedied with the help of some 'startle' or 'surprise' situation, which makes one pay special attention to the spot in question. Presumably resetting the system, to pay attention to the sensor data coming in. (Counterintuitively, resetting the whole thing might slow down how fast you recognize the visual scene in such a situation). These trigger-happy 'person' memes are thereby kept in check because the new sensor-level representation can now facilitate the real what interpretation for the position: Just a tree.

Presumably, this process is disrupted in a hallucination situation. It might be that hallucinating persons are somehow less surprised by out of place interpretations, but this is speculation.

The higher visual areas want to stay on, and that is useful in the system, too. That allows us to represent a stable visual scene.

In principle, there is a memetic driver to push dynamic parts to the furthest edge possible.

Note that cell assembly memetics gives a fundamental, high-level neurophilosophical conceptualization of top-down and bottom-up information processing. The memes (cell assemblies) want to be stable and active, across as many meaning levels as possible. Top-down and bottom-up are thereby explained in a unified model of 'stableness across levels'.

Also called 'perisaccadic visual remapping', or 'forward receptive fields'.

• https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5133303/
• https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10187178/
• Neuronal mechanisms of visual stability
• The Updating of the Representation of Visual Space in Parietal Cortex by Intended Eye Movements

'Remapping' is upside down from my perspective. The retina data is a dynamic thing.

The eye position/eye movement is not a resource to the visual brain in order to say what bottom-up sensory things to what higher level object things. The eye position/eye movement in my view is part of the interpretation structure in the first place. It is simply 1 data structure (1 cell assembly) per position and object, that represents what to expect when looking at this spot.

I consider 'predictive brain' [Andy Clark, Anil Seth, …] a neurophilosophy. It makes sense how perception is constructed, via a series of improving guesses. It is a framework that helps thinking of top-down and bottom-up processes.

This is somewhat aligned with the view I have here. After all, the basic operation 'pattern complete' is a prediction mechanism.

I see the biggest difference is in how the memes represent knowledge. They are approximations to an explanation structure, which is a counterfactual.

This emphasis on 'invisible', counterfactual entities deserves its own section on its own. It is that the memes represent models of the world. The are not merely some information processing means for prediction. They are the whole point of this software.

Speculations On The Puzzle Of Tourette's, Musings On The Hypothesis Of Behaviour Encoded Situation Analysis

Givens:

• Input to the striatum is from 2 types of neurons in layer 5 of all cortex. [insert sources]

• Striatum is implicated with 'generating' 'movement patterns'

  Other expressions of emotional states, such as smiling, also originate in the striatum and pallidum, and we are very sensitive to the emotional states these movements represent. ‘Faking’ a smile with conscious movements uses cortical motor control instead, and produces a result that looks false to others in most cases.

  (here)

• Striatum makes modulatory, inhibitory inputs to thalamic relay nuclei [M. Sherman]

• Consequently, this doesn't look like the pathway carrying the information for how to move. It is much more parsimonious to consider the role of striatum as some kind of selector, filter, or orchestrator.

  (This is counter to the current mainstream thinking about striatum).

• Striatum by default inhibits thalamic relay nuclei (via globus pallidus int.)
• Parkinson's is a degeneration of Substantia Nigra Pars Compacta, which makes dopaminergic innervations to the dorsal striatum etc. Pars compacta

• Sleeping Sickness,

  The disease attacks the brain, leaving some victims in a statue-like condition, speechless and motionless

• Oliver Sacks in Awakenings (1973) talks about the sleeping sickness patients that were rejuvenated by L-DOPA. (Small documantary about this).
• But this had the effect that the patients had compulsive behaviors.

Speculations:

• Let's assume that layer 5 is the 'behavior output layer' for all Cortex.
• Further, layer 5 behavior streams are the latent space encoding of higher-order thalamic relay nuclei (see A Curious Arrangement on this page)
• There would be 'behavior streams' that simply participate in the information processing of analyzing the situation.
• This would only be useful if the layer 5 neurons that go to thalamic relay nuclei are correlated with the ones going to the striatum. [which I don't know whether that is the case or not. Considering that column activity is correlated, the question is whether those neurons are co-located in columns, and that sounds likely to me].
• Striatum is an element that listens to all "latent" behavior streams of Cortex.
• What exactly is relayed at those thalamic nuclei? The answer is part of the puzzle of the striatum.

But here is sort of one idea that makes some general sense:

Say that the striatum is a selector, it looks at all latent behavior streams being expressed at the cortex and takes a value judgment from some Darwinian wires [Braitenberg 1984, also called Darwinian Brain]. And removes the brake from the streams that get the most approval. Perhaps it is selecting only a tight subset or a single one. This would solve Minskies 'pulled by 2 spirits' challenge.

Consider an animal that is roughly equally thirsty and hungry, it stands just between a water hole and a pile of hay. By the time it walks over to the hay, the thirst dominates and it goes over to the water. But by the time it is at the water, hunger dominates. You probably want to select one behavior and stick with that for a while.

     what striatum sees:


                     /----------------------|
                  /--                     /-|
                /-                     /--  |
             /--                     /-     |
          /--                     /--       |
        /-                      /-          |
      +----------------------+--            |
      |                      |             /+
      |  +---+       +-----+ |           /-
      |  |   |       |     | |         /-
      |  +---+       +-----+ |        /
      |        +---+         |      /-   time flow
      |        |   |         |    /-
      |        +--++         |  /-
      |           |          |/-
      +-----------+----------/
                  |
                  |
                  |
                  |
                  behavior streams


----


                            cortex II
                       +--------------+
                       |              |
                       |              |
        cortex I       +----------+---+
     +---------------+ |          |XX |   layer 5
     |               | +----------+-+-+
     |               |            | |
     +----------+----+            v |      /----+-
     |          | XX |              +->  /-   /-|
     +----------+-+-++                 /-   /-  |  (pretend striatum takes all inputs into account
                  | |                +-----/    |   at once for simplicity right now)
                  | |                |     |    |
                  | +------------->  |     |    |
                  |                  |     |  /-
                  v                  |     | /     |
            +-----------+            +-----+/      |
            |           |             striatum     |
            |         1 |                          |
            +-----------+                          |
            Thalamus, relay nucleus    |-----------+ inhibition via GPi

            +-----------+                          |
            |           |  |-----------------------+
            |         2 |                      |
            +-----------+  |-------------------+   |
                                                   |
            +-----------+                          | n per nucleus?
            |           |  |-----------------------+
            |         3 |                          |
            +-----------+  |-----------------------+ n?
                                        |
                ...                     |
                                        |
             n-relay nuclei       behavior orchestration organization
                                  perhaps 'the piano of behavior'


XX - active layer 5 neurons, correlated, with targets in the thalamus and striatum

In this model:

• GPi is like an array of gas pedals that are 'off' by default, 'the piano of behavior'.
• The striatum is removing the brakes, orchestrating what behavior is expressed.
• The striatum is a brain, concerned with selecting, switching, and committing to behaviors.
• It would make sense if this selection takes a Darwinian value judgment into account.

    +--------------+
    |      +---+   |
    |      |   |   |   possible behavior streams
    | +-+  +-+-+   |
    | +++    |     |
    +--+-----+-----+
       |     |
       |     |
       |     +---------                               +---------| THA 1
              commitment assemblies?  --------------| +---------  THA 2
                                                      +---------| THA 3
            'commitment organisation' ?               +---------
                                             behavior orchestration
                 ^
                 | value judgment  ^
                 |                 |
     +--------+  |                 |
     |        |  |                 | dopamine ++
     |    D   +--+                 |
     |        |                    |
     +--------+                    |
                               [VTA?, SN ]

D - Darwinian Brain
THA - Thalamic targets
VTA - Ventral tegmental area
SN - Substantia Nigra

Note the failure modes of this arrangement:

• Switching commitment too much: Erratic behavior?
• Switching commitment too little: Failing to multi-task?
• Committing too little: Parkinson's? Sleeping sickness? Catatonia? Basal Dementia?
• Committing too much: Overeating, violence
• Selection too permissive: Ticks, Tourette's?

Why the tremor of Parkinson's? Perhaps something tries to offset the lack of commitment?

Why swearing and ticks with Tourette's? In the model, the striatum is too permissive. (But the striatum is not encoding behavior information, it only selects). The logical conclusion is that there are behavior streams that output 'swearing behavior', virtually constantly.

But why is this useful for the information processing the brain is doing? Who says 'Thank you this is the kind of information I needed to do my processing'?

Hypothesizing:

• Swearing is a more primitive form of language (see Pinker 2007).
• From biological cybernetics: When wondering how to build language into Braitenberg vehicles. It is a useful first step, to make vehicles make sounds for the situations they are in.
• (semi-wild, but parsimonous I think) An information stream representing the notion 'I am dissatisfied', could be encoded in terms of swearing behavior.
• With the bonus hunch that the insular/cingulate cortex would represent dissatisfaction (hedonistically, socially,…), and so forth.
• If the striatum is too permissive, it could accidentally commit to expressing this behavior.

Not because the person wants to swear, but because part of how we analyze situations includes an encoding of dissatisfaction, virtually always.

I believe this dissatisfaction is a deeply animal aspect of our psyche. It puts us as an animal in our situation. It is simply part of the whole arrangement. Part of what gives it its flavor is hooks for memory lines and so forth.

Why ticks?

Memetics says there exists a meme, which is partly made from the tick behavior. If the striatum is too permissive for some reason, the memetic landscape is changed. A meme can strategize to span across striatum and express behavior which will, in turn, activate its activators, and so forth in a happy behavior loop.

This shows us that the healthy memetic landscape probably has a bias to not make the same behavior over and over and things like that.

Elegant, nice hack: Use the same organization to orchestrate internal behaviors, 'cognitive moves'. Stuff like having a train of thought, deciding to remember something, guiding one's imagination and so forth.

Why is there no name for this, what the hell?

Names? 'internal behaviors', 'thought moves', 'guided cognition', 'stuff like imagination', 'stuff like using working memory', 'internal muscles', 'muscles of the mind', 'mental avenues to take', 'though-muscles', 'mind movement', 'mind behaviors', 'internal storylines', 'narrative muscles', 'narrative moves', 'internal cognition', 'reflexive cognition', 'attention', 'stuff like remembering', 'moving one's inner eye', 'moving in thought space'

This then sheds light on basal dementia (Corticobasal degeneration). Roughly, if the striatum is gone but perhaps the gas pedals (GPi) are still 'off' by default. You stop being able to think and move.

It was always very dubious to frame the basal ganglia purely in terms of 'behavior' as in muscle movements.

• Other input outputs of striatum?
• If the medial temporal is a Darwinian brain, what is the relationship between the striatum and medial temporal?
• What makes the tremors of Parkinson's?
• What is the detailed circuitry of the striatum->thalamus?
• What is the relationship between the cortex, thalamus, striatum and cerebellum?
• What is the nature of the 'commitment organization'?
• What is the nature of all elements I have left out, GPe, subthalamic nucleus and so forth?
• Is there 1 organization of cortex-striatum-thalamus repeated across the whole neocortex?
• What is the nature of nucleus accumbens and reward pathways?

• https://www.sciencedirect.com/topics/agricultural-and-biological-sciences/striatum

Hypothetical neuronal substrate that stores activity over longer time spans. The evolutionary driver for this is to keep cell assemblies alive for longer.

• make many small neurons
• slower neuron tick rate, because the activity just needs to stay around
• make autapses or tight self-activating cell assemblies

From simple software engineering reasoning, we can be sure about some aspects of a flash-drive memory:

Data must be loaded, in a way that enables retrieval later on.

Data must be stored, frozen for all we care. For all we care that is allowed to be implemented by a USB stick (flash drive).

There must be a useful retrieval operation.

You don't want to remember all the things all the time, it is useful for the storage to be relatively isolated, and have some regulated way of saying 'Give me a memory… with criteria a,b,c …'

Retrieval can be further split into a query and a result.

Finally, the rest of the cognition machine can use the result for making a remembering situation. See A High-Level Story for Midterm Memory Retrieval for a cognitive-level idea about remembering.

I think it will always have to work like that if the data is not accessible in the network (long-term memory), then the network will have to load the data. The cognition machine and the flash drive stand in a similar relationship as the cognition machine and the sensors.

It seems parsimonious to assume that the flash drive simply evolved as a sub-population of an existing conceptron (assembly calculus neuronal area).

Evolutionary early:

t1


   +------------------+
   |                  |  'online' cognition area
   |  +------+        |   ('fast' area)
   |  |    A |        |
   |  |  A   |        |
   +--+------+--------+
   |  |   A  |        |  slow sub population neurons
   |  +------+        |
   +------------------+
         neuronal area whole


A - Cell assembly spreading across the whole area


-----

   reset the online cognition area or new sensor inputs

-----

t2


                  sensors
                 |
                 |
                 |
                 |
   +-------------+----+
   |         +---v--+ |
   |         |    B | |
   |   ^     +------+ |
   |   |              |
   +--++-----+--------+
   |  ||  A  |        |  slow sub population neurons
   |  +------+        |
   +------------------+
         neuronal area


B - Cell assembly with support from the sensors
A - The slow place part of A survives the reset,
    flash drive A part is now another input to the online area

Evolutionary derived:

+------------------+
|                  |    online cognition area
|                  |    (conceptron)
|                  |
|  |         ^     |
|  |         |     |
+--+---------+-----+
   |         |
   |         | Load and retrieve organization
   |         |
   |         |
+--+---------+----+
|  v         |    |  flash drive
|                 |  (formerly slow neurons)
+-----------------+

Note that online cognition area and flash drives are allowed to specialize. The online cognition area doesn't have a driver to implement mid-term memory on its own. We can expect this element to be a good 'fast' cognition area, then.

So 'automatic' load and retrieval would be the evolutionarily preserved operation.

The evolutionarily more derived specialized organization (where the flash drive is more isolated) has the job of regulating this load and retrieval.

It is like we have a substrate that is eager to spread. And our job is to regulate how it spreads. You might imagine this with water pressure and valves. But in some ways, it is closer to imagining this in terms of a wildfire. The wildfire just wants to spread into the available forest. This makes load and retrieve trivial, you only need some stretch of forest between 2 places and cell assemblies will spread.

We need some kind of network valve, then. To say what goes between the flash drive and the conception. Note that this valve is in principle the same kind of stuff again, a piece network that is ready to be populated by cell assemblies.

Here is a speculative idea, called association-box:

retrieval:

             +----------+
             |          | conceptron
             |          |
             |         -+-------------------+
             +----------+                   |
                                            |
                                            |
                                            |
                                            |
 +------------+  +---------------+  +-------+-----+
 |            |  |               |  |       |     |
 |            |  |               |  |       |     |
 |            |  |    A?         |  |       v     |
 |   F        |  |         <-----+--+--- Q  Q     |
 |     -------+--+-->            |  |        Q    |
 |            |  |               |  |             |
 |    F       |  |               |  |             |
 |            |  |               |  |             |
 |            |  |               |  |             |
 |            |  |               |  |             |
 +------------+  +---------------+  +-------------+


  flash drive     association box      'query area'
                  a-box                'q-wires'


Q - Query cell assemblies
F - Flash drive cell assemblies
A? - Potential overlap cell assemblies of a-box

This is very similar to the m-box of the explanation structures above. There is some intermediate level between the 2 areas, and with enough support from both areas, you get cell assemblies stretching all 3. But without it, you don't.

Result case:


             +----------+
             |+---+     | conceptron
             ||   |     |
   result    ||^  |    -+-------------------+
             +++--+-----+                   |
               |                            |
               |                            |
               +-----+                      |
                     |                      |
                     |result-wire           |
                     |                      |
 +------------+  +---+-----------+  +-------+-----+
 |            |  |   |           |  |       |     |
 | +----------+--+---+-----------+--+-------+---+ |
 | |          |  |   |           |  |       v   | |
 | | F        |  |      A  <-----+--+--- Q  Q   | |
 | |   -------+--+-->            |  |        Q  | |
 | |          |  |               |  |           | |
 | +----------+--+---------------+--+-----------+ |
 |            |  |               |  |             |
 |            |  |               |  |             |
 |            |  |               |  |             |
 +------------+  +---------------+  +-------------+


  flash drive     association box      'query area'
                  a-box                'q-wire'


FAQ - A flash-drive-association-query spanning Assembly

Whatever cell assemblies are on inside a-box are the results, so then it is a matter of having some a-box -> conceptron wiring to get the a-box results into the cognition machine (for remembering).

We see that would be useful for a-box to have a very tightly controlled inhibition.

1. If a-box is active too much, flash drive activation would leak into the result wires. And presumably, there is so much flash-drive activation that this is a liability, potentially causing epilepsy. It is not surprising then that hippocampus and related structures are associated with epilepsy.
2. A-box is useless if it is active too much. It would be equivalent to confabulating mid-term memories.
3. A-box would be evolutionarily built to be some sort of valve.
4. Note that there doesn't need to be that many special things about a-box, it should be a relatively generic citizen of the rest of the conceptron.

Having frequent resets of a-box perhaps, and regulating it very tightly, would be evolutionary drivers.

Note that these are high-dimensional spaces with sparse activity.

      +---------+---+-----------------------------------------------+
      |         |Sit|                                               |  conceptron (cortex)
      +---------+---+-----------------------------------------------+
                |   |
                |   |
      +---------+---+-----------------------------------------------+
      |         |FFF|                                               |  flash drive
      +---------+---+-----------------------------------------------+

Sit - Situation cell assemblies
FFF - Flash drive cell assemblies

In this drawing, we pretend we can collapse the high dimension into a single dimension. Also, we align them topographically. In reality, these are allowed to be distributed cell assemblies in the network.

How would load work with an a-box?

Perhaps it is allowed to be the same thing.

Load:


             +----------+
             |          | conceptron
             |          |
             |         -+-------------------+
             +----------+                   |
                                            |
                                            |
                                            |
 +------------+  +---------------+  +-------+-----+
 |            |  |               |  |       |     |
 | +----------+--+---------------+--+-------+--+  |
 | |          |  |   A           |  |          |  |
 | | F      --+--+---->    <-----+--+--- S  S  |  |
 | |          |  |     A         |  |        S |  |
 | |          |  |               |  |          |  |
 | |  F    <--+--+-----          |  |          |  |
 | +----------+--+---------------+--+----------+  |
 |            |  |               |  |             |
 |            |  |               |  |             |
 +------------+  +---------------+  +-------------+


  flash drive     association box      situation area
                  a-box

FAS - A flash-drive-assication-box-situation cell assembly

If we have a hypothetical situation area, could span cell assemblies across flash drive, a-box and situation area.

Just one alternative, example:

Situation areas, part of conceptron.


                    +----------+
                    | ^     ^  | conceptron
                    +-+-----+--+
                    | v     v  | situation areas
                    +---^------+
                        |
 +------------+  +------v--------+
 |            |  |               |
 | +----------+--+-------------+ |
 | |          |  |   A         | |
 | | F      --+--+---->        | |
 | |          |  |     A       | |
 | |          |  |             | |
 | |  F    <--+--+-----        | |
 | +----------+--+-------------+ |
 |            |  |               |
 |            |  |               |
 +------------+  +---------------+


  flash drive     association box
                  a-box

FAS - A flash-drive-assication-box-situation cell assembly

This situation area would be useful if it represents a hyperdimensional point in 'time-space-conception' space.

As the situation of the animal changes, the situation area represents new cell assemblies, and the a-box activity moves forward.

The activity in a-box would open a relationship between flash drive and conceptron. And a-box would not differentiate between the current situation and a memory query.

This would mean that every time you retrieve from a flash drive, you have the chance to modify the memory in the flash drive. In the limit, this would mean you are re-creating the memory in a flash drive for each retrieval. Presumably, this depends on implementation details like the amount of plasticity in flash drives and so forth. This seems to map to empirical observations from cognitive neuroscience of memory, afaik.

Pointing to the fact that there is indeed some element in the system that doesn't differentiate between query and load.

—

Perhaps déjà vu is triggered by some 'situation' at activates a lot of flash-drive<->a-box cell assemblies? Then some 'result' wires are on, even though the system was not in query mode.

Perhaps this would flip some interpretations into thinking that a memory is being retrieved. But confusion, because no query was set?

The temporal extent of déjà vu might then perhaps be a hint about some temporal aspects of the information processing of this arrangement. Perhaps after a few seconds the 'time-situation' moves forward, and the familiarity with the situation moves forward, too.

                           +----  at t1, the situation moved forward
                           |
                           |
                  t0      t1
      +---------+---+----+----+-------------------------------------+
      |         |Sit|--> |Sit'|                                     |  conceptron (cortex)
      +---------+---+----+----+-------------------------------------+
                |   |    |    |
                |   |    |    |
      +---------+---+----+----+-------------------------------------+
      |         |FFF|    |F  F|                                     |  flash drive
      +---------+-+-+----+----+-------------------------------------+
                  |
                  |
                  +-- unusual large cell assembly


Sit - Situation cell assemblies
FFF - Flash drive cell assemblies

And the strange overlap of situation and memory would be over.

A mode for retrieval and a second mode for load.

• From the neuroscience of the hippocampus

[…]

—

The neuroanatomy of the hippocampus says 'I need to be isolated'. Curled up with the extra ventricle and so forth.

—

It is interesting that empirically, cell assemblies can be re-ignited with single neuron activations. [See Rafael Yuste presenting this work].

It would mean that single target k-lines [Minsky] might suffice as retrieval implementation.

That is, simply make a bunch of random wires that go from flash-drive to Conceptron. Even activating single neurons will already ignite cell assemblies.

• If a flash drive has an evolutionary driver for slow activity, perhaps the reason for theta rhythm in the hippocampus is that there is some other limiting factor, saying that theta is after some criterion the slowest possible.
• I think this means that when building an artificial conceptron with a flash drive, we can think in terms of 'very slow' or 'frozen'.

I:


+---------+     +-----+    +-----+
|      O  |     |  X  |    |  X  |
+---------+  ,  +-----+ ,  +-----+


      t0           t1         t2

      ^
      |
      |
      |
 [synaptic input]

      t0



          ---->  shifted by 0.5 time
II:

+------------+    +-----+    +-----+
|         O  |    |  X  |    |  X  |
+------------+ ,  +-----+ ,  +-----+

    t0  t0.5     t1.5       t2.5

          ^
          |
          |
          |
     [synaptic input]



O - Inactive
X - Active

That is the time of the input determines the time of the firing. This allows a mechanism that abuses the fact that if the input to multiple neurons is in sync, the resulting neuron firing is in sync, too.

This might or might or might not be how cerebral pyramidal cells work.

As we shall see from the mechanisms shortly, you might at first wonder about the absence of any 'synchronizing' elements. It might be a feature, not a bug.

—

I have since realized this is studied under the topic of Phase resetting.

This would depend on the reactive phase property.

Step 1:

(This also applies to neurons in a single area).

Let's say you have 2 areas with neurons in 2 phases:

      +---------+           +-------------+
      |         |           |             |
      |  Xa     |           |   Xb        |
      |     Xa  |           |          Xb |
      +---------+           +-------------+
                area A                area B



Xa - neurons firing in phase a
Xb - neurons firing in phase b

Step 2:

They both span neuronal ensembles into a shared nucleus C.

      +---------+           +-------------+
      |         |           |             |
      |  Xa     |           |   Xb        |
      |  ^  Xa  |           |     ^    Xb |
      +--+------+           +-----+-------+
         |      area A            |   area B
         |                        |
         |                        |
         |      +-------+         |
         +------+-> X <-+---------+
                |    X  |
                +-------+
                       nucleus C


Xa - neurons firing in phase a
Xb - neurons firing in phase b
X - neurons in nucleus C (perhaps with 1/2 tick rate)

Let's say that C is a paced nucleus, which has neurophysiology that keeps it in a certain phase. All we need to require is that the neurons in C can be excited from A and B, but only fire from an 'intrinsic' phase-giver. (This is biologically plausible as far as I can see. A theta rhythm pace nucleus exists if I remember correctly). We can easily imagine implementing this with cycling gradients of chemicals or so forth.

Observe that X is allowed to be an associated neuronal ensemble, spanning A, B and C.

Then, you can extinguish the activity in A and B, and subsequently reignite from C, where now everything is in phase because it comes from C.

Step 3:

Extinguish activity in A and B:

I:


      +---------+           +-------------+
      |         |           |             |
      |         |           |             |
      |  ^      |           |     ^       |  (empty, but not for longer than 2 neuron time steps)
      +--+------+           +-----+-------+
         |      area A            |   area B
         |                        |
         |                        |
         |      +-------+         |
         +------+- X   -+---------+
                |    X  |
                +-------+
                       nucleus C


X - neurons in nucleus C survive

II:



      +---------+           +-------------+
      |  X!     |           |     X!      |
      |         |           |             |
      |  ^      |           |     ^       |     re-ignite, same phase
      +--+------+           +-----+-------+
         |      area A            |   area B
         |                        |
         |           !            |
         |  !   +-------+   !     |
         +------+- X   -+---------+
                |    X  |
                +-------+
                  ^    nucleus C
                  |
                  |
                  |
                tick!



X! - spanned neuronal ensemble, same phase (from C)

I:


               +------+
               | ^    | empty
   +------+    +-+----+
   |    ^ |      |
   +----+-+      |
        | !      | !
        |        |
        |     +--++
        +-----+   |
              +---+
                 input nucleus


II:


               +------+
               | ^ X! |
   +------+    +-+----+
   |  X!^ |      |
   +----+-+      |
        | !      | !
        |        |
        |     +--++
        +-----+   |
              +---+
                 input nucleus

               ! t0


X! - in phase, from the input nucleus

If we have the operation available to suspend input nucleus, we can have the following arrangement:

III

               +------+
       +-------> X    |
   +---v--+    +-+----+     X ensembles still alive, still in their phase
   |  X   |
   +------+

              +--++
              |   |
              +---+
                 input nucleus


    [ t0, t1, ____ ,  ]

                    put the nucleus on hold for a little...


IV:



               +------+
       +-------> X    |         X in phase 1
   +---v--+    +-+----+
   |  X   |
   +------+



   +-----+            +------+
   | A   |            |  ^ A |  A in phase 2, depends randomly on the hold period
   +---^-+            +--+---+
       |                 |
       |         +-------+
       |         |
       |         |
       |         |
       |        !|
       |      +--++
       +------+ A |
              +---+
                 input nucleus

                 !ta
                  ^
                  |
                  |
                  |
 [ t0, t1, ____ ,  ]



 [ O, X, O, X, O, X, O ]   timeline X
            |
            |
            +----|    phase difference
                 |
                 |
 [ O, O, ______, X, O, ]   timeline A

                 ta - the load time for neurons A


X - still in phase from input load time t0
A - in phase from load time ta

Observe these 2 cases: If you suspend in exactly 1 neuron time step time or a multiple of it, you will go exactly in the rhythm of X. In this case, igniting A after the suspense time will make X and A go in sync. But if you do not care about the rhythm of X, and you suspend for any stretch time, then the A inputs will arrive out of sync with X.

Note that X and A could be allowed to exist in the same area, in this case, neurons would fire out of phase in the same area (empirical question).

We can imagine building the following into a Braitenberg vehicle:

move    +-> [ eyes ]
        |     |
        |   +-v-+
susp.   +--||   | input device
        |   +--++
        |      |
     +----+    |
     |    |    |
     +----+    |
 eye motor d.  |
               |
               |
          +----+--+
          |    v  |
          |       | neurons
          +-------+

• Input device receives sensor data from the eyes.
• Input device a paced nucleus again, so all its wires are on at the same time.
• Neurons are a network of excitatory units that support neuronal ensembles.
• Eye motor device moves the eyes of the vehicle.
• At the same time (with a branching wire to be sure), send a suspend signal to the input device.
• The suspend signal should suspend the input device roughly as long as the eye movement takes. (would be ok if it takes slightly longer, but shorter might be a problem).
• Sensor data changes rapidly during the movement and is relatively static in between movements.

        +-----------------------+
        |                [B]    |
        |                       |  visual field, actual
        |                       |
        |  [A]                  |
        +-----------------------+


sensor
        (A)              (B)
         |                |
         |                |
         |                |
inputD +-+-+            +-+-+
       +-+-+ Pa  [___]  +-+-+ Pb
         |   |            |   |
     +---+---+        +---+---+
     |   v   |        |   v   |
     |   A  <+        | A B  <+
     +-------+        +-------+
neurons

        t0               t1

            -------->
          eye movement,
          suspend input device



P - eye position encoding
A - neuronal ensemble representing the visual object A and Position a.
B - neuronal ensemble encoding B and position Pb, in a different phase from A

Let's also add the position P together with the input device. When the vehicle moves it's eyes, the input device sends position and eye sensor data to the neurons, this is synchronous because the input device has a controlled phase.

When the vehicle moves its eyes, we suspend the input device at the same time. The ensembles are allowed to stay active in the neurons. We see ensemble A, representing sensor data about the A object, composed with the position data, is allowed to stay stable in the network.

After the eye movement is done, at t1, new sensor data is coming into the neurons. If the eye move time interval is allowed to be any kind of interval, i.e. non-discrete, the fresh neurons B will be virtually guaranteed to have a different phase from A.

With this mechanism, the eye movement and input apparatus of the vehicle is like a moving conveyor belt that represents object data, it doesn't move around topologically, but it moves around in time-phase-space, and it ignites representations of the sensor data + and position data at different places in this time-phase space.

This mechanism would create ensembles of synchronously firing neurons, which encode position and sensor data together. This is one idea of how to make a visual field. Since everybody else in the system could see what position goes together with what sensor data.

"Auto composes" because this mechanism is automatic, but it depends on a 'reactive phase' property of the neurons.

Observe that this theory also fits this:

If nobody cares about the synchronicity, this would come as an artifact of the mechanisms of the input nucleus. It might then be that you observe sync. activity in the brain but it doesn't serve much of a functional role in the system.

—

• Maybe you don't need to 'suspend'?
• If the machine has a gate on the input nucleus, then perhaps attention can be a compose mechanism.
• If we can require 'phase reset' wires, then the thalamus could innervate with phase reset wires and doesn't even need to be in sync or phase itself.
• Then, whatever is being paid attention to, i.e. what activity we allow into the cortex, would be in sync, because we are resetting the phase together with the 'new' inputs.
• The phase reset wires don't need to be a thalamus concept.
• Perhaps something like the logic of Modeling a threshold-device would oscillate whenever there is a lot of activity coming in.
• This would be like implementing a bouncy substrate when you put an activity in, by paying attention, it bounces. And perhaps the bouncing is a phase reset implementation.

I have seen it said mysteriously 'it is not clear how synchronous activation does XYZ'. Where XYZ is then binding, awareness, consciousness etc. If you don't have a theory of the functioning of the machine, saying 'sync activation' is not even wrong. There is not even anything that can be wrong if you don't have an explanation (a theory) that says what kind of piece of mechanism you are looking at.

Synchronous activation only does something if something else in the system behaves differently, depending on whether there is synchronous activation, or not.

• Neurons that are only above the firing threshold when there are synchronous inputs.
• In a high-dimensional network, since you can listen to everything a little bit, your neurons can tell you what is active together.
• This depends on the time integration properties of the neurons. Presumably, there are different kinds of neurons in the brain.
• I.e. what is the time window for 2 things to count as synchronous?
• At the same time this window should be small so you can differentiate the synchronous populations apart.
• The power of a sync. population scheme comes from the fact that now you have composable elements.
• I.e. if you have a bunch of musical notes or building blocks, you want to be able to put them together in all kinds of ways, this would be the main point for a compositionality.

Talk.

He mentions the work of Wolf Singer (below), too.

Cortical neurons are multifunctional, they have 'mixed selectivity'. In other words, neurons are part of different ensembles.

Interesting that in Hydra these things are separated into modules ⁵².

The data support a model where pattern completion may arise at this near decomposability of circuits into their subordinate functions and information representations and also suggest a flexibility for this feature, depending on circuit stability, given that, as Hydra’s circuits mature, this feature is lost. The flexibility of processing in the mammalian cortex potentially leaves this feature open, creating an inherent hierarchy at the overlap of this functional decomposition.

—

The base mechanism [Singer, Abeles]:

1. Neuronal net where neurons can have different phases.
2. Compose ensembles (temporarily) via sync. activation.

E. Miller calls this 'Neurons that hum together'. (I don't know why he then says wire together there).

–> Some kind of nice on-the-fly composability was required for brain software that composes explanation structures from elements

(I.e. the means of combination of the software paradigm.)

An ensemble B is a temporarily allocated, coherent subpopulation in the network, firing at rate b-rate. Let's call B ensembles that fire in the beta wave range, and G ensembles that fire in the gamma wave range.

I didn't gather what Miller thinks how this sync. activation is created.

They say that ensembles go in beta oscillations top-down, encoding 'prediction'. So [B1, B2, B3, ...], call it beta. Prediction error ensembles are presumably a set of ensembles [G1, G2, G3, ...] which only ignite if the beta prediction is violated at sensory/earlier cortical areas. (Call it gamma).

The model:

'beta'
top-down            +---------------- shallow cortical layers
prediction          |
                    |
       | +-----+----+-+   ^
       | |     |    | |   |    <----- high cortical area
       | |     |      |   |
       | |     |      |   |
       | |     |      |   |
       | |     |      |   |
       | |     |      |   |    <----- early cortical area
       | |     |      |   |
       v +-----+------+   |
           |
           |           'gamma'
           |           bottom up
       ----+           prediction error
 deep cortical
 layers

When Miller talks of beta, gamma and theta in this talk, I have 3 superheroes in my mind that battle it out.

"Beta is predicting. Beta is suppressing Theta…"

The level of abstraction is very high. Since Miller is lumping the ensembles into their frequency bands.

I see the danger of this landing a bit on wonkyness to other neuroscientists. I suppose it is obvious to Miller that the mechanisms are open questions.

I like that the whole point of frequencies is to separate the ensembles, so I split this again:

                  ??? override, extinguish?
                B <----+
                       |
        +--------------+-+
        | +--+ +--+  +-- |
        | |B1| |B2|  |G1||
        | |  | |  |      |
        | |  | +--+  |  ||
        | |  |           |
        | |  |------>|  ||   sensor input
        | +--+  ???  + -+| <-----------------
        +----------------+



B1, B2, - 'top-down' ensembles, each in their beta frequency
G1, Gn  - 'bottom up' ensembles

• Presumably, different beta ensembles spread to different cortical areas. But it would be quite interesting if that spans globally.
• What is the mechanism that makes a top-down ensemble 'suppress' a gamma 'error signal'? Whatever it is, it is a Socratic wire implementation. Confusion and Socratic Wires.

The setup:

      top-down ensemble
   +---+
   |   |
   |   |
  ++---+----------+
  ||   |          |
  || B |          |
  |+-+-+          |
  |  |            |
  |  +--[b] <-----+--------- [input]
  |  |            |
  +--+------------+       sensory area (for the sake of argument)
     |
     |
     |
     |
     |
case 1: input roughly supports B so B stays active

      top-down ensemble
   +---+ :(
   |   | It is sad now. No support from the sensory area
   |   |
  ++---+----------+
  ||   |          |
  || B |          |
  |+-+-+     [g]  |
  |  |        |   |
  |  +--[b]   +--+--------- [input]
  |  |   :(       |
  +--+------------+
     |
     |
     |
     |
     |
case 2: input doesn't support B, but g

Idea 1:

• Since B is not supported by the sensors,
• there is a sudden drop in overall activity in sensory-area
• This drop makes some threshold-device oscillate,
• This could mean that the thalamic input gate is opened wide
• Or, that B was inhibiting its alternatives at layer 6 (-> cortico-thalamic), and now this inhibition is suddenly gone.
• A rush of activity comes in, activating even more [g]
• And this is somehow in gamma? Thalamic bursting mode?
• The system flips into a second mode, a gamma signal mode,
• which is an ensemble that spreads through the shallow cortical layers?
• and somehow this is now undermining the support for B on all cortical layers/areas?
• This must be a very parallel mechanism, following thalamocortical circuits which are parallel (?)
• Then G would gain sudden support in the network, while B is loosing support.

    top-down ensemble
    :(
 +---+                      B not inhibiting gamma nucleus anymore
 |   |
 |   +------------------------------|    ???
++---+----------+            --       [ gamma nucleus ]
||   |          |                         |
|| B |       !  |                         |
|+---+      [g] |                         |
|            ^  |  !!              ++     |
|            +--+--------- [input] <------+
|               |
+---------------+

          [ G ] :D

Gamma nucleus?

• Maybe claustrum somehow orchestrates excitation (threshold-device-like) or makes phase resets (somehow gives a signal to go in gamma? - that sounds strange to me)
• Maybe Layer 6 cortical->thalamic inhibition? (and/or Layer 6 -> TRN)
• Perhaps 'fresh' neurons in the thalamus go into burst and that makes gamma activity?
• Then thalamic relay neurons would suddenly receive sensor input and their inhibitory modulation would drop at the same time,
• They would be fresh relay neurons, we could biochemically implement them bursting now
• Or some other mechanism

It is aesthetically pleasing that in this mechanism the strong gamma frequency can override the weaker beta. But beta has the benefit of already being established.

There must be mechanisms or wiring that prevent the 'top-down' ensembles from falling into gamma frequency by themselves. It would then be a memetic landscape trick, that 'established' ensembles go in the mediocre 'power' of beta, which has the chance to be 'overruled' by a Socratic input override mechanism, using gamma frequency to establish a sudden swap from B to G in attractor space.

Perhaps this can be achieved by biochemically making thalamic relay neurons 'tire' of making gamma/bursts. You would then be an ensemble stretching the organization, but your relay neurons at the thalamus give a low hum of beta only.

Back to the superheroes for a sec:

                     +-------------- broadly speaking inhibitory
                     |
                     |
                     |
     Beta            |         Gamma
                     |
   +---------+       |      +---------+
   |         |-------+-----||         |
   |         |              |         |
   |         |              |         |
   |         |              |         |
   |         ||-------------|         |
   +---------+              +---------+
     --+-                     ---+--
       |                         |
       +-----------+-------------+
                   |
                   |

          the balance of power        ^
     ^                                |
     |                                |
     |                                |
     |                                |
the benefit of being             the benefit of being 'fresh'(?)
established                      the power of gamma? (~5-10 faster than beta)

If beta is too strong, your top-down expectations rule how you perceive the world - hallucinations, schizophrenia, … ?

meaning++

If gamma is too strong, your expectations are overruled by reality too much, presumably, your world is less infused with meaning. - autism?

meaning--

From memetics, we know that if any of the mechanisms fail, the ensembles will happily exploit the circuits to be active more.

During hallucinations, perhaps the mechanism that prevents 'established' ensembles from going in gamma is broken. Perhaps the mechanism that makes unexpected inputs go in gamma is broken. etc.

• Perhaps the absence of a (newly established) beta ensemble is a confusion signal
• It is interesting that theta and gamma are phase-aligned

Miller lit:

• I didn't mention his traveling wave data
• Also, working memory has a position. Frontal activity is on during working memory, is either the left or right hemisphere, depending on the location of the stimulus. This follows the left-right split of the visual field. This is quite astounding. The default assumption might have been that visual inputs somehow distribute on hemispheres, but this says that this stays split. That working memory has a location, too somehow perhaps.

Beyond sync. ensembles.

The time step composition idea of neuronal ensembles:

|------------------------------------------------|
     theta freq.


|------------------------|
     alpha freq.

+-----------+-----------+
| A         |  B        |      2 beta freq. fit into 1 alpha freq. and so forth
+-----------+-----------+


|-----------|-----------|
     beta freq.


|-----|-----|-----|-----|
      gamma

...

Vaguely, the idea is that ensembles would be composed in time frequencies of multiples (of two?). That is ensembles A and B fire in beta, then, if B activates A again, the whole thing makes an alpha time step. You an also say the waves are cross-frequency coupled.

But what are the mechanisms that produce this composition? Surely, there must be answers from simple cybernetic reasoning of the neurons and the devices around them. (G. Buzsáki's work is probably relevant).

Something cool about this idea is how the ensembles are composed very similar to music; The aesthetics of this are neat.

Moche Abeles

György Buzsáki calls this a neural syntax.

Origin of syntaxis

First recorded in 1535–45; from Late Latin, from Greek: literally, “arrangement in order,” equivalent to syntag- (stem of syntássein “to arrange together”; syntactic ) + -sis -sis

From dictionary.com.

The core notion of syntax is the ordering or arrangment.

Buzsáki's recognition of the importance hierarchical organization of brain rhythms of different frequencies and their cross-frequency coupling

Wikipedia

This is a variation of the re-ignite scheme:

I:

        +------+
        | A    |
        |   +--+----+
        |   | C|    |
        +---+--+    |
            |     B |
            +-------+


A and B - ensembles firing out of phase
C - overlapping A and B


II:


           +----+
           | C  |   |------------- thought pump
           +----+

    reduce excitability, until only C remains.

III:

Re-ignite from C:


         A
         ^
         | +----+
         +-+ C  +--+
           +----+  |
                   v
                   B

        +------+
        | A'   |
        |   +--+----+
        |   | C|    |
        +---+--+    |
            |     B'|
            +-------+

A' and B' - same ensembles but in phase (from C)

Observe that step II in principle would be allowed to go down to a single neuron. Or, with intrinsic firing rate, and on-the-fly excitability (Alternative Plasticity Models), you might even simply extinguish the whole ensemble, then it could be re-ignited from either a single random neuron or from intrinsic firing rate that goes in-phase together for some unspecified reason.

I realize that C doesn't need to be in the overlap either, if A and B are associated in the network, then a kernel of activity without overlap would perhaps activate them both, this time in phase with this kernel. Again, this depends on the hypothetical 'reactive phase' property.

Note that it is not clear that synchronous activation or even a composition means much. It might be more or less artifacts of the mechanisms of this thing. It might be we are looking at the car engine vibrating and wondering what the vibration does for the acceleration of the car.

I see that it makes a lot of sense to associate and compose hyperdimensional data structures. I label this 'expectation structure', it is a bit like asking for a banana and getting the gorilla together with it. In a network where things like bananas and gorillas are dynamically associated, depending on the situation. This is useful for instance, when the position encoding of an object goes together with the what encoding. (Getting A Visual Field From Selfish Memes). And this might already be the whole mechanism that is needed (without the need for sync. or rhythmically composed activation). If X is on, then as a consequence, A is on. That is, if the system pays attention to position X, then A becomes more relevant in the system, too. And this is as fast as the neurons go. This might be enough to represent the notion that what is located at X is A. And since A has the chance to inhibit its alternatives, it is only A and not some other object.

The point of course is to say it might just be a 'bind'. But perhaps the system mines synchronicity to make a bind. Perhaps both of these things happen in a layered way.

• Do neurons fire out of phase in the same area?
• If they do but it is multiples of each other, that might just come from a thought-pump, or simply activity shifting around.
• (to see why this is true, imagine a neuronal ensemble with a center, which fires at every tick, and some surrounding activity that flips back and forth between a bunch of neurons - they will all fire apparently in 1/2 the frequency).
• Do neurons have or do not have the 'reactive phase' property?
• What phase-maker nuclei exist and what is their relationship to the brain?
• What sync-listener neurons exist?
• (otherwise, we can say sync activation probably just doesn't matter).
• Perhaps other syncing-up schemes exist, perhaps neurons firing together naturally sync up as clocks do or something. (Irrc. The electronic field of the neuron activations is a candidate mechanism for this. Wild but what can you do?). If this is true, this would enable a completely separate or alternative set of mechanisms than what I said here.

• Why directed traveling waves in PFC?
• Presumably 'working memory loaded' must be represented at layer 5 (affordances), somewhere. Or everywhere more or less?
• Presumably the affordance pay attention to x is either identical to the 'working memory loaded' affordance, or tightly linked.
• Would it be fair to assume that the presence of working memory changes the memetic game completely? One would almost think that once this is there, the memes that don't play to be coherent globally are not significant or something.

• Idea: This might be extremely significant. Given the enlargement of prefrontal (and hippocampus) of humans.

  It might say that humans started having global coherent neuronal memes, that those relate to 'consciousness', language, symbolic reasoning, understanding, cultural memes and even personhood and creativity.

  Perhaps some mutation in PFC, like higher attenuation (making it harder to 'spread' into), made global coherent ensembles more likely. This gave gradual selective advantage, and gradually got improved with a bigger PF.

  Another piece that points in such a direction Pulvermueller: Semantic grounding of concepts and meaning in brain-constrained neural networks, in which Pulvermueller argues that language allows for "~ globally coherent assembles". Because the network dynamics have words functioning 'higher grounding' (a kind of anchor in the middle for the ensembles).

• Here is a further idea:
• If you lower the excitability globally now (let's say this is another button to press via an orchestration mechanism).
• Then you increase the challenge of challenger-node dynamically, and because this is a measure of global coherence, I feel like this would say the ensembles need to fit the network exceptionally well. I.e. this is a mechanism to find precise ideas that fit well (G. Palm).
• This sounds a little bit like an abstract thought to me, where you try to have a minimal and sufficient explanation structure.
• See also . I have speculated that a threshold device might be responsible for Hofstadters 'abstraction ceiling' brain-feeling.
• Don't migraines feel like some kind of muscle of the mind is tensed up? Perhaps whatever nucleus is enabling this move has some kind of epilepsy during migraines.

—

Notes that produced the idea:

Traveling waves in the prefrontal cortex during working memory

During baseline conditions, waves flowed bidirectionally along a specific axis of orientation. Waves in different frequency bands could travel in different directions. During task performance, there was an increase in waves in one direction over the other, especially in the beta band.

One fragment of mechanism that comes to mind would be Attenuation. This forces ensembles to 'move' around. Together with a topology (or 'geometry') in the network where closer neurons are connected, you get traveling amebas of activity.

This doesn't explain why the activity is regular in its direction during working memory vs. nonworking memory conditions.

It would fit with the requirements of working memory that we have high attenuation and perhaps low or no plasticity. Because you don't want your network to be permanently shaped by the working memory items.

But what is the implementation of 'holding in mind' and 'erasing'?

Perhaps you can re-ignite the same ensembles temporarily, presumably from Thalamic inputs. (This would fit experimental outcomes where whatever thalamus goes to PFC is essential for working memory). (this is an easy prediction to make anyway because of M. Sherman's driving inputs).

Erasing is then terminating the 'hold in mind' process.

• You could do that randomly every say 500 ms
• Then somebody else in the network needs to re-establish a 'holding in mind' if they still care
• You can implement a 'tiring' mode to the 'holding in mind' neurons, so if the same neurons fire for x amount of seconds, it becomes harder and harder for the holding in mind ensembles to stay active. (i.e. the ensembles would need more and more support from the network).
• This has certain biological reasoning appeals, too. Because you don't want the animal to be stuck with a single idea. You only want single ideas if they are worth it.

Here is a small twist:

What if working memory is implemented by making a region that is simply ridiculously hard to spread into, i.e. ensembles need a lot of support from the network to spread into it? (perhaps simply with low excitability, but why traveling waves?).

You automatically force the memes that want (and can) spread into there to be very competent in gathering the forces, moving the brain into a state where they are supported - sort of globally.

Not because the PFC is such a great region. But because it a region. And you always will get memes that use a region. Because the competitors that would would outdo them. Conversely, you only ever care about a region because you have competitors that would get support from that region.

So it is the tip of the iceberg creme de la creme of memes that can spread into working memory.

They must be amazingly useful buttons to the cognitive user, setting large cascades of effects in the brain in motion, that will give support from much of the network.

Perhaps at the level of working memory, we start having a little bit of leaky abstraction, we must 'hold in mind' a piece of info rather clumsily. This is probably a feature, not a bug, it says that those memes need to be implicated with the user intent and so forth. Which makes this even harder.

[ The subtext here includes Dennetts 'fame in the brain', subliminal vs. distributedly supported and "global workspace" by Baars ]

• I am assuming that Thalamus (MD?) -> PFC is necessary for working memory (i.e. activity in PFC). This comes from M. Sherman's driving inputs.
• People with schizophrenia have smaller Thalami and are less good at working memory tasks⁵³
• Let's roll with that tip of the iceberg gathering the forces memes idea:
• Then perhaps, PFCs function as being a 'hard to spread into' region is diminished, and all kinds of memes spread into it.
• Perhaps this would make such patients more distractable and so forth, where random ideas manage to go into working memory, but then such working memory disruptor memes are something more like local winners. Not like the usual strong iceberg memes with their 'global' support.

Large memes, very smart memes, that know how to use much of the computer to be successful - to be thought again. They spread into mid-term memory, they hijack where you look, they make you feel this or that way and so forth.

They represent 1:1 the neuroscience of 'the belief immune system' [Didn't find the research links right now, at least one researcher is using such terminology, quite sure].

Considering the logic of our inhibition model, where 1 cell assembly 'wins out':

        -----------------
         \             /
          \     A     /                   meaning-level
           \         /
            \       / <----->   B
             \     /   inibit
              \   /
             --\-/---------------------
                X                        sensor-level


A - Stable cell assembly

A Stands in competition with meme B:

   meme A                 meme B (competitor)

+-----------+     alternatives                       high meaning level
|           |   |-----------|
|       A   |
|           |
|     +-----+--+            +- -- - -+               medium meaning level
|  |  |     |  |                     |
+--+--+-----+  | |-------|  |
   |  |        |               b     |
   |  +--------+            +-- -- --+               sensor level
   |
   |                +-----+
   +------------->  | c   |------|
    activate        +-----+
                          alternative

A good meme

1. Activates its activators, that is how it is constructed in the first place.
2. Activate the alternatives of competitor memes, on every level of meaning.

Meme A here will activate c, even though c might not have to do anything with A at first glance. This will happen if c is an alternative to whatever sub-cell assemblies would support B, an alternative to A.

If meme A doesn't make this move, then it is just not a good meme and is discarded.

The dress:

If you don't know it, it either appears blue gold or blue black. Funny, it used to be white-gold for me back when I first saw it. Now it is blue black. And I cannot flip into seeing it white-gold, either.

I think what happens here is there is a further layer at play:

1. sensor level (looks blue)
2. meaning level (I see a blue dress)
3. psychology level (I understand the scene)

3 Says that you don't randomly flip something fundamental as the color of a dress. It is a larger situation, that a higher order meme is creating for the rest of the meme-machine.

You successfully predict yourself to competently parse the scene without going crazy. And only the memes that are well-supported by the situation will survive.

• It seems like the hemispheres only have some trickles of information flow left.
• Perhaps with assembly calculus we can make a computational definition of split/merged cognition machines:

that is the extent of the (theoretical) maximal cell assembly of the network. If it spans the 2 hemispheres, we can roughly say there is 1 cognition software running. If it doesn't, then it is 2.

• It is funny that 'alien hand syndrome' is conceptualized as a motor-system problem so to speak.

When in reality the reason for this is crazily profound.

• I think it is almost obvious that there are 2 'selfs' and so forth in a split-brain patient.

(everything speaks for it, nothing speaks against it).

• Split-brain is interesting, but it tells us how the system looks in an unnatural configuration.

What we learn from considering split-brain and lateral neglect and so forth is like shadows, which can tell us about the nature of the intact arrangement.

[Lecture 8. The Thalamus Structure, Function and Dysfunction]

• The thalamus is an ipsilateral concept. I.e. it looks like the input organization is separate.
• If the Thalamus Matrix system provides context (perspective) to the Cortex, this organization would roughly correspond to some kind of '2 perspectives mechanism'.

• Cortex is connected via Corpus Callosum (CC), at least roughly to the same region on the other side.

  CC connects functionally equivalent cortical areas in each hemisphere with the exception of primary sensory and motor cortices, which are sparsely connected transcallosally (Innocenti et al. 1995). The common understanding is that the sensorimotor cortices (S1, M1 and premotor cortex) are connected monosynaptically via the CC only for areas representing the body midline area (Innocenti et al. 1995).

  Functional anatomy of interhemispheric cortical connections in the human brain

  The prefrontal and temporoparietal visual areas are connected interhemispherically by poorly myelinated, small-diameter, slow-conducting fibres (Lamantia & Rakic, 1990; Aboitiz et al. 1992).

  (from here again)

  This is a challenge to my 'stupid wires' model. If we assume the cortex implements an ideal assembly calculus, there would not be a reason to have slow-conducting fibers.

  I would postulate that the presence of slow-conducting fibers only makes sense if the information processing has something to do with time delays. The computational reasoning for this is clear: There must be an evolutionary driver for making timeless calculations as fast as possible. (per definition, a timeless calculation doesn't concern itself with the time passing. See functional programming, What is a calculation?). The only reason to put delays into your information processing is that your computation has something to do with time.

  The other alternative is that somehow those fibers are on their evolutionary way out. Or that a slow conduction was sufficient, so there was no driver for myelination. (This would be rather surprising, why myelinate all the fibers in Cortex but then leave some out?).

• With a Hebbian substrate and a thalamus, you could do the following:
• Connect roughly to the homotopic side, at the area that receives the analogous thalamic relay inputs, with random wires:

                        C<->C'
          +---+-------------------------+---+
          |   |                         |   |
      +---+---+-+                  +----+---+-+
      |   |   | |                  |    |   | |
      |   | ^   |                  |    | ^ | |
      |     |   |                  |      | | |
      +-----+---+                  +------+---+
            |  C                          |   C'
            |                             |
            |                             |
            |                             |
      +-----+--+                    +-----+--+
      |        |TH1                 |        | TH1'
      +--------+                    +--------+
      |        |                    |        |
      +--------+                    +--------+




TH1 - Thalamic relay nucleus, side 1
TH1' - Analogous relay nucleus, other side
C - Cortical area, receiving inputs from TH1
C' - Contralateral cortical area with inputs from TH1'
C<->C' - A few random wires 'interhemispheric and homotopical'.

If this is the arrangement, you will observe that via Hebbian Plasticity, we will select wires that find some kind of symmetry in the information representation between the 2 thalamic relay inputs. Braitenberg 1977 mentions such symmetry detectors.

Simply imagine that from many random wires, the ones that connect to 2 pieces of network that is active on both sides at the same time, will be on. From then on, those symmetry wires will pattern completely to the symmetrical activation on both sides [Cell Assembly literature].

• What is symmetry in derived meaning spaces?
• The hunch is somehow that this would be a useful arrangement to 'share information' between hemispheres. (which must be the function of CC either way).
• The 2 sides are functionally connected (see literature on functional connectivity networks).
• Consider that organizations would have been possible:
• Random wires between cortex hemispheres
• Wire Thalamus -> contralateral hemisphere
• Wire contralateral hemisphere -> Thalamus
• Or wire contralateral hemisphere -> other subcortical structures.
• The corpus callosum is curiously absent at v1. Exactly the area where we would have expected to gain use from 'symmetry detectors' [Braitenberg 1977]. (all primary sensory and motor cortices are sparsely connected transcallosally).
• In general, this arrangement seems to mix aspects of converging information flow with separate information flow, but why?
• Why decusssions?
• Decussation seems to be a neocortical concept, but not enforced (audition).
• What is the nature of hemispatial neglect?
• What is the other 'Wernicke' doing? There is some literature on this. Iain McGilchrist mentions that the right side has a different vocabulary and so forth; So perhaps it is not true that only the left is doing language.
• If the hemispheres implement some kind of '2 mutually complementary perspectives' mechanism, then what is the computational-level explanation of how and why?
• Perhaps the best way to make neo-neo cortex is to make yet another artificial sphere, trihemispheric, or quadrihemispheric.
• It would mean we have strong notions of what the basic circuits are, then we can simulate another symmetrical cortico-cortical connected element.

Doesn't it feel like you are 'inside' a memory during a strong déjà vu?

My idea is that whatever is saying 'Here comes a memory, please fill in the blanks, Mr. Cortex (step 0)' is accidentally on. And the system falls into an interpretation that says I am enacting right now a memory.

I.e. Déjà vu is remembering being on, without a memory coming in, but the interpretation of cortex is for a moment that the sensor input is the memory. And the rememberer resources try to fill in the blanks.

Déjà vu is the interpretation of being inside a movie of one's mind a little bit.

—

Perhaps this is a hint about the circuitry. The relative rarity of déjà vu might give a hint that it is usually not the same input stream circuitry where memory 'context' comes from and where sensory input comes from. But it is also not impossible to interpret one place as the other.

—

This of course all fits with the idea that Cortex is so vast. That Cortex has a tiny amount of sensor inputs compared to its interpretation. So it is always the interpretation that makes cognition, perception, imagination, memory, and dreaming … They are created by the vast interpretations of the Cortex and its nuclei.

—

Of course, step 2 depends on how fine-grained the memory coming from the hippocampus is. Presumably ruminating on an emotionally salient event, perhaps every few minutes, such memory would become highly detailed.

—

A strong déjà vu is almost existential, it is a very intense 'mind-moment'. Perhaps only psychedelics and things like out-of-body experiences are stronger. For this reason, I am very skeptical about inducing artificial déjà vu in the lab.

I think what you measure is something like vague familiarity or something. At best, the notion is 'very low-level déjà vu' and at worst, it is utterly confused and doesn't have much to do with an actual déjà vu.

-—

*) perhaps via the thalamic matrix system

-—

This preceeds The Slow Place: Evolutionary Drivers For Mid-Term Memory I changed my mind and think that it would perhaps make more sense when déjà vu is triggered by out of context query result information states.

I want to move up from the neurons, like seeing the forest instead of the trees. Just like a stone can have a temperature, neuronal substances can represent information. So you can sort of imagine a magic stone substance that can be loaded up with information.

In general, it looks a bit like the cortex is a 'substrate that supports memes'. It can represent `vast` spaces of meaning, sort of everything possible.

+----------------------------------+
|                                  |
|  ^                               | cortex 1 ('everything possible machine')
+--+-------------------------------+
   |
   |
   |
   | [brain nuclei]
 organize, drive, extinguish activity - shape the memetic environment

From AI we know that 'everything possible' is not useful unless I have something that selects and explores this space.

My main view then of what the rest of the brain is doing is that it is creating a memetic environment, in which only some memes (presumably useful for biological success), survive.

So you get this view of the brain shaping the memetic landscapes, to say what kinds of memes are possible. And the cortex fills the activation spaces⁵⁸

That is from the 1977 Braitenberg Cortes paper:

1. Distinguish an A-system of cortico-cortical connections from a B-system.
2. A-system is the apical dendrites of pyramidal cells in upper layers [1 and 2?], that listen presumably to everything else in the brain.
3. B-systems are local connections, the basal dendrites of pyramidal cells,
4. With some careful reasoning you conclude that the connectivity in the B-system is ~0.1. (you are connected to 10% of the local network).
5. From statistical reasoning, it looks like you can make the square root of N (number of neurons in cerebrum), 'compartments'. Each compartment listens to everybody else. It is intriguing that size and count overlap with that of a 'cortical column'. Further, if every compartment listens to all other compartments, then probably the activity in a compartment should be correlated; Intriguingly, the orientation columns we know look like they have this property.

thoughts ->

(The software engineering approach dictates that we let our imagination go wild sometimes. Let's assume for a moment you would get this or that kind of computational properties of the system, then …).

From this then you would get the hypothesis that each compartment (map to cortical columns) receives a holographic encoding from the rest of the system. So each part gets a representation of the whole via the A-System, the network would not mean we throw away the messiness of the connections of the network. It would still have the cell asssembly properties, Hebbian plasticity and so forth. So each compartment would learn to represent different things, depending on the overall context. But - you could store and manipulate the overall context easily and this would be useful. I.e. that is the state of which compartments are active.

This is very intriguing because it would be a bridge to hyperdimensional computing. [HD/VSA]. In this hypothesis, you could store an encoding of the overall state of the cortex in roughly 10⁵ neurons. Btw the numbers are very large. In the literature, the 'blessing of dimensionality' comes at 10⁴.

We would be able to do HD computing, the fundamental computations are trivial to implement. Simply align 2 input vectors and do either addition or multiplication. There are many ways we could imagine a hypothetical circuit or nucleus, or the cortex itself to do such math.

From HD computing you get wonderful properties, in meaning-spaces. (See HD/VSA).

The fundamental operations are bind and bundle.

With bind you find the contrast of 2 points in meaning space, it corresponds to the surface of a rectangle, made from width and height.

With the bundle, you find the 'between' between 2 points in meaning space. You might say the essence.

—

The 'perspective', 'zoom', and 'inside perspective' move:

1. Find the essence of a cloud of meanings (with bundle)
2. Use this essence as a perspective point. I.e. you bind all the points, highlighting the differences between the points.

This would be a beautiful on-the-fly perspective mechanism, blowing up the meaning spaces around a point that matters.⁵⁹

You would be able to temporarily and totally dynamically make the brain into a brain that is good at representing 'this kind of meaning space'. And that is allowed to be completely arbitrary and so forth.

This fits me with the observation that a different perspective completely changes the kinds of thoughts we think.

Via a zoom like this, we could move around in meaning-space. From outside, there is a cloud of meaning points that all look similarly far apart, after a zoom, it would be like sitting at the center of this meaning cloud. And seeing the differences between the meanings.

—

The notion of contrast and perspective is one of the challenges to purely connectionist models. At first glance, I would think everything is mud if everything is sort of associated with everything else.

—

If I would know that thee TRN was segmented, then I would go like holy shit that fits.

This would fit with a circuit/mechanism where the thalamus would have slaps of roughly 1/2 * 10⁵ neurons, each making wires to the cortex, going through a segment of TRN, with 1/2 * 10⁵ wires going through it. Half, because the thalamus is a unilateral concept (1 hemisphere).

This would fit, if you would want to gate 1 holographic encoding at a time. Because at first glance it would be a bit strange to ever separate one encoding out. (This consideration depends on the idea that TRN is a kind of activation gating 'attention' mask around the thalamus).

This is just 1 of many things that could be true though. Perhaps the Thalamus isn't encoding a context at all. And if it does, then maybe it is useful to have 'partial attention', after all. (The brain would be more complicated).

—

If the thalamus would represent the cortex context in this way, you might expect that there is a hippocampus -> [thalamus-HD-encoder-nucleus] -> cortex circuit; And the basic midterm memory retrieval mechanism would be to reinstantiate the cortex context. (See below musings on hippocampus, future).

'thalamus-HD-encoder-nucleus' should be roughly 1/2 * 10⁵ neurons large. It would be one of the Anterior nuclei, then. If the simple 1:1 reasoning of 'what is connected to what' can be applied here. (Which is not clear).

Or, there are slaps of 1/2 * 10⁵ neurons in such a nucleus. Where 1 slap would correspond to 1 context context.

—

Intuitively, this might be incredibly sparse somehow. Let's say that a tiny subset of compartments active is already meaningful. Speaking against the idea that you would need to encode all 1/2 * 10⁵ neurons.

I have seen this being mentioned the cerebellum might be doing HD computing. Initial thoughts.. things that don't fit:

1. The cerebellum from all angles doesn't look like it is part of the most interesting aspects of cognition. In particular, it looks like it is doing fine-grained timing calculations, required for movement (Braitenberg On The Texture of Brains). At the very least you have to concede that the current literature conceptualizes it as some kind of movement computation-making computer.
2. Perhaps the regular structure of the cerebellar cortex invites a hypothesis about it doing hypervector math. The issue here is that the parallel fibers are thin. They look like they have an evolutionary pressure to be slow (see Braitenberg). This doesn't bode well for a model of the cerebellum where you want to do timeless calculations involving the parallel fibers. (Every fiber in the cortex that does computation seems to have a driver to be fast, see Pyramidal cell activity above).
3. The whole reason to consider HD computing in the first place is to explain challenging modes of thinking: succession, causality, directional relationships, contrast, perspective, essence representations (symbol representation) and so forth. It does not make sense then to point to the cerebellum to implement this, since we know neuroanatomically, that the cerebellum is not essential for the most interesting cognitive processes (language, thought and so forth).

—

Thoughts after reading Pentti Kanerva's seminal Distributed Sparse Memory:

Talk as intro: Pentti Kanerva - Computing in Superposition in the Classical Domain

(I'm a big fan of this stuff)

• The same Albus+Marr theory of memory is postulated for Hippocampus function, this would redeem sparse distributed memory (SPM) for a neocortex memory module, then neocortex would be a user of the memory interface that the Hippocampus provides, and the Hippocampus would implement (perhaps among other things) an SPM.
• "Perhaps the regular structure of the cerebellar cortex invites a hypothesis about it doing hypervector math" is simply wrong. My apologies. A hypervector math would be done somewhere else. Cerebellum is postulated to be an SPM module.
• The non-timeless nature of parallel fiber transduction is still interesting. And perhaps this is implementing a k-fold sequence memory as Kanerva mentions in SDM.
• In this scheme, the bit locations of the content matrix would have (random) delays (0,1,2… k-delay)
• You might call this a 'delay lines k-folded sparse distributed memory'

What does make sense is to consider olfaction and the computations it implements.

Vempala is mentioning this here as a stepping stone to think about 'assembly calculus'.

Perhaps it is simply the nature of making information processing on olfaction sensors that simply gives the system a nudge to high dimensionality. This then would give us the evolutionary driver for the cortex, and its HD computing implementation.

1. Take some tertiary olfactory ganglion
2. It represents olfaction information states, high dimensions
3. Make a few neurons sluggish, or make some neurons go back and forth, or duplicate the whole array
4. Now the input to the next time step of the computation is allowed to be not only the sensors but the current 'situation' that the network represents.
5. From biological/cybernetic reasoning, we see the evolutionary driver for analyzing bigger and more fine-grained situations.

The evolutionary story of the cortex would then be summarized as sniffing with more and more context.

• https://www.pnas.org/doi/abs/10.1073/pnas.1406077111?url_ver=Z39.88-2003&rfr_id=ori%3Arid%3Acrossref.org&rfr_dat=cr_pub++0pubmed