Philosophy of Design
What makes a design good? Not "correct", not "fast": good, the kind of good that makes a programmer grin when they read the code. These are my working notes on that question. They wander through software, biology and Leibniz, because the question does.
Simplicity, and what lies beyond it
Simplicity is the ultimate sophistication.
Usually credited to Leonardo da Vinci, but there is no record of him saying it. The idea goes back at least to Clare Boothe Luce (1931), and Apple made the line famous in 1977.1
Design is simplifying, not complicating. That is the first half. The second half: the best design is not merely simple, it is evident. You look at it and it could not have been otherwise.2 Self-evidence is the true swag, beyond even beauty.
Elegance
Elegance is the property of good design. In my constructor theory of adaptation and design (see below), I make this formal: elegance is the property of good design as a creative process can judge it. Judging it requires explanatory universality: the judge must be able to explain why the design works, and why its possible alternatives would work less well.3
In other words: will the program for creativity, the one Deutsch says AGI has to be,4 contain some kind of elegance heuristics? I suspect so. A creative process that cannot tell an elegant conjecture from a clumsy one has nothing to steer by among the infinitely many conjectures it could make.
Neal Stephenson put the feeling better than any textbook:
Elegance?
Pardon me, Your Honor, the concept is not easy to explain – there is an ineffable quality to some technology, described by its creators as a concinnitous, or technically sweet, or a nice hack – signs that it was made with great care by one who was not merely motivated but inspired. It is the difference between an engineer and a hacker.
The Diamond Age (1995), Miss Pao answering Judge Fang.5
Some properties I would put on the list:
- Elegance is pragmatic, useful and delightful. Vitruvius asked of a building firmitas, utilitas, venustas: durability, convenience and beauty.6
- Elegance is mysterious. It hangs together with truth, scientific discovery and art.
- In mathematics, an "elegant" program is sometimes defined as the shortest program with a given input/output behaviour. Is that what we mean in software? I don't think so. More on that below.
A rich simplicity?
Isn't good design not only simple, but also rich in its simplicity?
Lego has a dozen or two important bricks, not two. Particle physics is elegant, and it does not have two kinds of things either, it has a handful. I don't buy the idea that elegance is mere compression. Something more is happening: expressivity.
Leibniz: simple rules, rich world
Leibniz said God chose the world that is "simplest in hypotheses and richest in phenomena".7 That is the shape of every good design I know:
| Rule level | Content level |
|---|---|
| natural selection | the tree of life |
| the hardware | the software |
| the Lisp evaluator | the program running in the REPL |
| the Lego bricks | the castle |
Leibniz also separates two kinds of truths:8
- Truths of reason (Vernunftwahrheiten): necessary, their opposite is impossible. Why does a NAND circuit compute NAND, and why can you build a universal computer from it? A NAND circuit that does not compute NAND is nonsense. If that were up for grabs, anything would be.
- Truths of fact (Tatsachenwahrheiten): contingent, they could have been otherwise. We resemble fish during embryology, not because of a law of nature, but because of history. Why does the elephant have a trunk? Because history.
And here is the twist: it is a law of nature (evolution by natural selection) that the alternatives are possible. The contingent stuff has a necessary reason for being contingent.
Adaptation needs possible alternatives
This is the core of my constructor theory of adaptation and design, for genes and for memes (background in Memes and Adaptation). Constructor theory states laws of physics as statements about which tasks are possible, which are impossible, and why.9 I want to express adaptationism in those terms, and one thing is clear from the start:
For a design to be an adaptation, its alternatives must be possible.
If no other trunk could have been built, the elephant's trunk is not an adaptation, it is a necessity, a truth of reason. Selection can only choose among variants that can actually arise. Where nothing else is possible, nothing is selected, and nothing is adapted. I take this as a fundamental conjecture, and I say it follows naturally from Dawkins: cumulative selection is non-random survival among variants, so the variants have to be physically possible in the first place.10
Chiara Marletto has argued that replication and natural selection are possible under laws of physics that contain no design, provided those laws allow digital information media.11 To my knowledge, nobody has yet written down adaptation itself as a statement about possible tasks: the task "build the alternative" must be possible. That is the step I want to take. Elegance then has a place in the same theory: a design is judged good against its possible alternatives.
Open questions:
- How far do the alternatives have to reach? Is one possible variant enough, or does adaptation need a whole neighbourhood of possible designs?
- Are there designs in software that are truths of reason, where no alternative is possible, and are those the "evident" ones?
Why science works at all
Despite its complexity, the world can be described by simpler rules. Leibniz already saw a version of this: put dots on a page at random, and there is always some formula whose line goes through all of them. When the formula is as complicated as the dots, it explains nothing.12 Hermann Weyl drew the conclusion: if any function counts as a law of nature, the concept of a law becomes empty.
A programmer knows this in their bones. There is always a program that outputs a given string of N bits:
"001010110011"
It uses the same amount of memory as the string itself. The program has taken on the shape of the data. It is the maximally useless scientific theory: "the data exists". Science, in contrast, finds rules that are much smaller than what they describe, and that is why it predicts.
François Chollet calls the bet behind this the kaleidoscope hypothesis: the endless novelty of the world is recombination of a small number of "atoms of meaning".13
Levels
This interplay of simplicity and complexity shows up everywhere once you look.
We can describe what a program does in a few sentences, while an epic story of bits, electricity, cache misses and memory lookups plays out underneath. You cannot look at the bits and say what the software is doing. That is a level violation, and it is a real affliction in computational neuroscience.
In complexity science, emergence means that a higher level of description is possible: you can talk in simple terms about a thing, even though it is made of many interwoven parts. Software is the quintessential example.
Physiology was one of my favourite subjects in biology, because it is a language of design and function. We call the heart a blood pump. That abstraction buys leverage: when it pumps too slowly, whatever blood flow is for will suffer. Causal reasoning, for free. It seems obvious, but only because biology is an awesome science. Consider how clueless we are when we look at the neuronal activity of the brain, and how clueless we would be looking at the electricity in a computer without knowing what an operating system is.
A Kolmogorov hypothesis of elegance?
Is the most elegant program the shortest one? No.
Kolmogorov complexity as the elegance heuristic is a hypothesis, and a popular one. Jürgen Schmidhuber holds that an observer finds beautiful whatever their own encoder can describe briefly;14 Max Tegmark considers algorithmic complexity as a measure of simplicity over mathematical structures.15 I think it is what David Deutsch calls the mathematicians' misconception: the idea that mathematics alone decides what is simple.16 Different laws of physics make different transformations simple and different computations tractable. Already a quantum computer would give different answers about what is short and what runs fast.
It is the wrong way around. The classical computer does not say what runs fast or what is simple. Reality says what is simple.
- The best source code is usually not the shortest. It uses the material at hand, with sensibleness and straightforwardness.
- It has artistic swag: concinnity, technical sweetness.
- It makes clear that, in the infinite space of possible programs, the programmer chose one way, without coercing anybody else to think or behave in a predetermined way.
- It leaves an open future for the creative problem solving of the programmers who come after.
Lego again: you would not make a brick with a bespoke joint that only bespoke bricks fit into. You make a brick that composes with all the other bricks, open to the whole symbolic system of Lego design.
Maybe "short" plus "frugal", measured against real-world resources, gets closer to what programmers mean by elegant.
In biology, my feeling is that elegance has to do with the range of possible uses of a thing. An elegant design is one that lets you win in the future. It plans for the unplanned, because problem solving in the real world is open ended. Gregory Chaitin argues that looking at evolution only as adaptation to an environment misses exactly this: its open-ended creativity.17 The problems of creativity and open-endedness are at the cutting edge of theoretical biology, too.
So elegance is a creative problem, and creative problems are open ended. There is no "most elegant" solution to anything. The same way we can always find better theories of the world [Popper], we can always find new ways of solving a problem.
Good design is about winning (in the future), resourcefulness, usefulness and fun.
Counting things: parsimony and abundance
Imagine someone comes to you and says: "Evolution by natural selection is not parsimonious, it needs such a large count of generations." They would be confused indeed. They would have fallen for the runtime footprint fallacy.
The theory of evolution by natural selection is a three-line algorithm. Writing down the software design does not mean writing down the state of all successive generations. The theory does not enumerate the generations.
We find the same situation with Everettian quantum theory, "many worlds".
| Software | Theory of evolution | Many worlds (Everettian QT) | |
|---|---|---|---|
| Theory (algorithm) | The program (source code, design) | The natural selection algorithm | The quantum mechanical equations |
| Factual question | Does hardware exist to run it? | Did life have enough time to evolve? | Is reality large enough for the worlds? |
| Runtime entities | Memory, state, execution trace | Generations, organisms | Worlds, branches of the multiverse |
| Parsimony of the theory | Small, elegant design | Three-line algorithm | Few simple equations |
| Abundance at runtime | Vast compute and memory | Large count of generations | Large count of worlds |
| The fallacy: "not parsimonious…" | "…it uses too much memory" | "…it needs too many generations" | "…it needs too many worlds" |
For evolution it is obvious how absurd the argument is. Not even wrong. For many worlds, the exact same confusion somehow disables people's minds.
The only question is whether reality is large enough to run the algorithm. If it is, nothing more is required.
A software design is not more complicated because it needs a lot of memory at runtime. A large count of entities is not complicated if the description of the entities is small. The reverse is closer to the truth: with a lot of memory available, we can write simpler algorithms. We live in the space age, we live in abundance [Hickey].18
To me the grandeur, the size, the abundance of reality is beautiful. Like when Giordano Bruno proposed that the stars are distant suns.
The cognitive psychology of design work
It is easier to do cognitive neuroscience on ordinary reasoning, but humans are experts in things. I claim the most interesting cognition happens when people reason inside their field of expertise. How does a designer think?
- You can have a feel for the future uses, and the future fun, of a thing, in the context of elements that do not exist yet but will form a harmonic whole.
- One version is the element that simply fits many things well. It is a safe pick.
- The other is "I predict future fun, given this element": a vague hunch that some unknown future element could exist. A wild move, as they say in chess, that turns the game into fresh, unexplored positions.
- In terms of Gestalt: perceiving some parts of a future whole might already give us a feel for the whole.
Open questions:
- Is the feel for future fun learnable, or is it the residue of having seen many futures play out?
- Can a design be evident to one programmer and opaque to another, and is it then evident at all?
- Where does an LLM sit here? It has seen a lot of futures play out.
Elegance detectors
David Deutsch asks why flowers are beautiful.19 Flowers evolved to signal to insects ("I have nectar for you"), yet humans find them beautiful. Why? Deutsch's guess: the gap in knowledge between flowers and insects is so large that the signal has to be hard to forge, and the easiest way to send a hard-to-forge signal across a gap that large is to use objective standards of beauty. And we recognise it because we face the same problem: in knowledge and creative individuality, one human is like a species. So human artists signal across the same scale of gap. The theory of that objective beauty is still waiting to be written.
An artist creates knowledge, gets closer to the truth, by conjecture and criticism, trying version after version of a piece. Art and design are a dialogue of the artist with the piece [Glanville].20 Beethoven's sketchbooks are full of discarded versions, and they really are wrong versions of a symphony.
Elegance is a criterion for good scientific theories, and for mathematical proofs. Beauty and truth have some relationship. Why? This makes elegance a subtle but somehow obvious problem, one of the problems. The missing theory of elegance should unify epistemology, science and the arts, and it seems related to creativity, personhood, consciousness and AGI.
So the human brain should have adaptations: elegance detectors. Perhaps they play a central role in the brain's ad-hoc epistemology.
My guess, building on Braitenberg's cell assemblies: an idea is elegant to the brain when it explains many things with little activity, with few neuronal ensembles. When the brain finds a good idea that fits well, overall activation would rise suddenly, even where excitability is low. In German: Gedankenblitz, a lightning thought, a flash of insight.
Perhaps this is also the biological basis of humor. It would make sense to have evolved elegance detectors that reuse existing circuitry, and train the neuronal nets with some kind of reward signal.
Aristotelian physics is a marshmallow. It doesn't cut the world at the joints; it is a list of special cases: the natural place of rocks is the ground, so they sink in water; the natural place of wood is above water. Compare Newtonian physics: a few simple entities (force, mass, acceleration) explaining a huge amount of stuff.
Butterflies
Butterflies are the flowers of the animal kingdom.
Valentino Braitenberg, Information: der Geist in der Natur.21
Literature
- Vitruvius: firmness, commodity, and delight
- David Deutsch, The Beginning of Infinity
- Why Are Flowers Beautiful?
- Ranulph Glanville
- Stuart Halloway: Simplicity Ain't Easy
Footnotes:
Quote Investigator: Clare Boothe Luce, Stuffed Shirts (1931), "the height of sophistication is simplicity"; the exact wording in print by the 1950s; Apple II brochure, 1977. The earliest Leonardo attribution Quote Investigator found is from 2000.
Stuart Halloway, Simplicity Ain't Easy, and later Evident Code at Scale. The idea echoes C.A.R. Hoare's 1980 Turing lecture: designs "so simple that there are obviously no deficiencies" versus designs with "no obvious deficiencies".
David Deutsch, The Beginning of Infinity (2011): people are universal explainers, able to create any explanation that can be created. The use of it for elegance is my own.
David Deutsch, Creative blocks, Aeon (2012): an AGI is a program for creativity, and it will not come from tuning narrow AI.
Neal Stephenson, The Diamond Age, or, A Young Lady's Illustrated Primer (1995), Part One.
Vitruvius, De architectura I.3.2; "durability, convenience, and beauty" in Morgan's 1914 translation. Henry Wotton's 1624 paraphrase, "commodity, firmness, and delight", is the famous one. Text.
Leibniz, Discourse on Metaphysics (1686), §6. Bennett's translation renders it "simplest in general rules".
Leibniz, Monadology (1714), §33: vérités de raisonnement and vérités de fait.
David Deutsch, Constructor theory, Synthese 190 (2013); Deutsch and Marletto, Constructor theory of information, Proc. R. Soc. A (2015). A task counts as possible if a constructor can perform it to arbitrarily high accuracy and keeps the ability to do so.
Richard Dawkins, The Blind Watchmaker (1986), ch. 3, on cumulative selection; and The evolution of evolvability (1989), on how embryology limits which variants can arise.
Chiara Marletto, Constructor theory of life, J. R. Soc. Interface (2015).
Leibniz, Discourse on Metaphysics §6; Hermann Weyl, The Open World (1932). Gregory Chaitin found the passage through Weyl and reads it as an early form of algorithmic information theory: Chaitin, "Leibniz, Information, Math and Physics".
François Chollet, AGI-24 keynote and It's Not About Scale, It's About Abstraction (MLST, 2024).
Jürgen Schmidhuber, Low-Complexity Art, Leonardo (1997); Driven by Compression Progress (2009), where beauty is relative to the observer's compressor and interestingness is compression progress.
Max Tegmark, Is "the theory of everything" merely the ultimate ensemble theory? (1998), and The Mathematical Universe (2008), which credits Schmidhuber for the point.
David Deutsch, Constructor Theory, Synthese (2013), §2.8: "Different laws of physics would in general make different functions computable… They could also make different physical states and transformations simple – which determines which computational tasks are tractable."
Gregory Chaitin, Proving Darwin: Making Biology Mathematical (2012). His metabiology models evolution as unbounded creativity rather than as filling a genome with information about the environment until it stagnates.
Rich Hickey, The Value of Values, JaxConf 2012: "we're moving into what I would like to call The Space Age", with memory and disk grown "a millionfold" (transcript).
David Deutsch, The Beginning of Infinity (2011), ch. 14, "Why Are Flowers Beautiful?": "the easiest way to signal across such a gap with hard-to-forge patterns […] is to use objective standards of beauty"; and "a human being, in terms of knowledge content and creative individuality, is like a species". Also his talk.
Ranulph Glanville, "Try again. Fail again. Fail better: the cybernetics in design and the design in cybernetics", Kybernetes 36 (2007), building on Pask's conversation theory; also his RSD3 keynote (2014).
Valentino Braitenberg, Information: der Geist in der Natur (Schattauer). My translation.
Generated by Claude on Benjamin Schwerdtner's behalf.