So if you want to simulate a mammalian neocortex in part or in whole, in software, how might you go about doing it? It's fairly uncontroversial to describe the human brain as one of the most complex structures that we know about. I've heard the comparison to a football stadium filled with spaghetti.

But it's not that chaotic. Over the past decades, neuroscientists have painstakingly studied central nervous systems and found structure and regularity where it seemed daunting to find. Some organizational principles we do know about are high degrees of symmetry, modularity, and hierarchy.

One approach that could be taken would be to try to code each and every feature, every neuron and connection, from scratch. Another might be to borrow some ideas from biology, and use the power of evolution to construct a simulated phenotype from an artificial genotype. This allows you to represent the model in much more compact form, and to experiment with parameters at the genetic level, inducing large-scale changes in the phenotype. Evolutionary algorithms are also very good at optimizing parameter spaces that are very large. That just means that if you have a lot of things that need tweaked, evolution is good at it.

I've applied evolutionary algorithms to artificial neural networks before, using the NEAT algorithm. NEAT was a step forward, but it has severe limitations. First of all, it uses what's called a direct encoding. This means that every feature in the phenotype is encoded explicitly as a separate gene in the genotype. There's a 1-to-1 mapping between genes and traits. But a moment's reflection reveals that this isn't analogous at all to biological systems.

There are roughly 100,000,000,000 neurons in your brain and about 20,000-25,000 genes in your DNA. So every neuron simply can't be encoded as a separate gene. So how does DNA produce a structure with 100 billion neurons and trillions of connections? It uses a form of indirect encoding, a 1-to-many mapping between genotype and phenotype. You don't have genetic encoding for every hair on your head. You have the information for building that structure, and then you have regulatory DNA that determines when, where, and how many are built. (To any biologists in the audience...if I'm butchering this or grossly oversimplifying, let me know).

So the way I'm going to encode my neocortical model is through the use of a hierarchical, iterative representation. For example:

[neuron n=12][connection]

The pseudocode above describes a single neuron with 12 connections. The genes to the right encode lower-level features, and n is like a regulatory region describing the number of lower-level features. With this representation, you can just keep adding terms to the left, and build increasingly large, complex, hierarchical structures. For example:

[minicolumn n=85][neuron n=10][connection]

This describes a network composed of one minicolumn, which contains 85 neurons, each with 10 connections.

[sheet n=100][minicolumn n=85][neuron n=10][connection]

By adding another single term, we can create a sheet composed of 100 of these minicolumns. Now, I'm leaving out a lot of information for simplicity's sake. There are regulatory aspects that determine the spatial layout, how lower-level features are packed into a given region, and so on...but hopefully you get the idea.

I believe that the evolution of human intelligence involved the formation of a precursor module, something akin to the cortical minicolumn, that performed a very useful task (invariant pattern learning, inference, and prediction). I think this module allowed for the easy scalability of mammalian brains, enhancing existing functions (e.g. visual processing) and adding new functions (e.g. human language) through the mutation of regulatory sequences affecting the number and distribution of minicolumns.

That's the idea, anyway. In my model, mutations will affect the number and distribution of features. We'll see how it goes.

Next time I'll talk about how learning and development take place in the model.