What's your earliest memory?

For me, probably from when I was 4 or 5 years old, but their very sketchy snippets. Things start to get a little more solid from 7 on. This is pretty common, and the name for it is infantile amnesia, basically the inability to remember anything in the first few years of life.

Chris Chatham discusses new research looking into a linguistic-based explanation for this phenomenon. One working hypothesis is that humans primarily encode memory in terms of their language, and since babies haven't developed language yet, they can't store long-term memories. You can go read the experimental set-up, which uses memories of colors, but in a nutshell they found evidence that 2 year-olds can encode memories dealing with color that they can't put into language. A small percentage of these children were able to link the word to the memory after they'd learned it, but in the time span of the experiment (2 months), most couldn't.

Personally I don't think infantile amnesia has anything to do with language. If one buys into the hierarchical memory view of the neocortex I described here in recent posts, what's basically going on is that a baby is learning the lowest levels of the hierarchy first, the base of the cognitive pyramid. In the visual cortex, this means lines, edges, points...visual primitives. In audition, this means learning to segment all the incoming noise into discrete sounds with particular properties like pitch and timbre.

Including other modalities such as touch, these are the basic sensory building blocks that all of your experience is built on. Because concepts at the next layer up the hierarchy are built from the foundation, you can't learn things like basic shapes (squares, circles, etc.) without having first learned lines and curves. The pyramid is built from the bottom up. Basically this means that you can't possibly encode an episodic memory, a scene such as a dinner you had when you were only a few months old, because you couldn't even properly parse the visual and auditory information all around you, what William James called "a blooming, buzzing confusion".

Once your neocortex builds enough of the lower layers of the pyramid, you've got the foundation to start encoding spatio-temporal sequences. And you've got the ability to start learning concepts at the top of the pyramid, the most abstract representations we have: language. So I think the onset of more solid memories and language use are correlated, but not causal. My guess is that humans not exposed to language (e.g. raised by wolves or something), would have a similar development of memory as a linguistic human, though I would expect the linguistic person to have somewhat earlier and somewhat better memories on the whole. I'm sure language help encoding immensely, once you learn it. My guess is just that even without language, a human and many other animals, encode early memories just fine, at the point at which they have a perceptual foundation.