I thought this article about studying happiness scientifically was pretty interesting.

When people think of "science," they naturally think of atoms, planets, robots — things they can touch and see. They know that subjective experiences such as happiness are important, but they believe that such experiences can't be studied scientifically. That belief is dead wrong.

What does it take to study something scientifically? One word: Measurement. If you can measure something, you can study it scientifically. Can we measure a person's subjective emotional experience? You bet. People can tell you with both words and actions what they are experiencing — what they are seeing, hearing, smelling, thinking, and feeling—and these reports are the essential data on which the science of experience is built. If you don't think such reports are reliable or valid, then you should feel free to discard my research papers.

This isn't just an issue with happiness, but with a great deal of cognition. How do you study what someone (or some thing) is thinking or feeling? Behaviorism (focusing only on external behavior to infer internal algorithms) was rejected as an approach to studying cognition half a century ago.

But should we buy what this guy is saying? He likens the study of happiness to optometry. Theoretically you might be able to make someone a pair of glasses without their subjective report, by some sort of excruciating analysis of their eye structure. But it's a hell of a lot easier to make an adjustment on a machine, have them look through, and ask them if the picture is better or worse.

In this case, why would the person lie? They presumably want to purchase a pair of glasses that will let them see better. But what about self report for studying other types of cognition, like memory, awareness, or emotion? People's memories are notoriously bad, and easily manipulated. A famous study showed two groups of subjects a video of a car accident. Later, one groups was asked "How fast were the cars going when they hit each other?", the other "How fast were the cars going when they smashed into each other?" Later, when asked whether they remembered seeing broken glass, the group that was asked the "smashed" question reported seeing broken glass more often. Same video, subtle manipulation, and the person's memory is altered.

So just how much can you trust techniques like self report when it comes to cognitive functions? Well, you kind of have to. You have to be clever with experimental design, and hope that brain imaging technology keeps trucking along at the brisk pace it's kept for several decades now.

Anyway, go read the whole article, it's interesting.