Don't know if you've seen it, but some bloggers, like Charlie Stross, are linking to the Iraqi Body Count Project, supposedly an attempt to compile worthwhile or meaningful statistics on the number of civilians killed in the current war with Iraq.

I noted criticism of the IBCP by other bloggers, like Oxblog, but finally decided to check it out for myself.

This morning there was a new listing in the IBC database:

20Mar-03Apr - Nasiriyah - air raids
minimum: 226
maximum: 240

And they list the sources. In this case, there are three: The Christian Science Monitor, The Independent, and The Telegraph.

So I went ahead and tracked down the actual articles. Here there are, with the relevant sections quoted.

From the Christian Science Monitor:

Doctors said they had treated 900 injuries in the past two weeks. They said US aerial raids had killed 250 civilians, all of whom had been brought to the hospital.

From The Independent:

Doctors claimed that up to 250 people had died and that many had been temporarily buried in a park in the city, waiting to be interred in the holy city of Najaf when the road became safer. But a visit to the park revealed just 12 shallow graves, though local people said that up to 50 people in that area had been killed in bombing attacks.

And from The Telegraph:

Doctors from the city hospital, who pleaded with the marines for protection, said they had treated 900 injuries in the last two weeks. They said aerial raids had killed 250 civilians, all of whom had been brought to the hospital.

Now then, here's the method of "data extraction" used by the Iraqi Body Count Project:

Maximum deaths. This is the highest number of civilian deaths published by at least two of our approved list of news media sources.

Minimum deaths. This is the same as the maximum, unless at least two of the listed news media sources publish a lower number. In this case, the lower number is entered as the minimum. The minimum can be zero if there is a report of "zero deaths" from two of our sources. "Unable to confirm any deaths" or similar wording (as in an official statement) does NOT amount to a report of zero, and will NOT lead to an entry of "0" in the minimum column.

Now I'm no statistician, but I can't seem to figure out how they got a minimum number of casualties as 226 and a maximum of 240. Unless I'm looking at the wrong stories. But these are the sources quoted for the dates mentioned, and these are the only stories I can find that mention civilian casualties.

Notice that The Independent mentions that only 12 shallow graves were observed, and local residents claimed 50 dead. But this is only one source, so according to the IBCP these numbers are unreliable and can't be used as a minimum. If either of the other sources had such a sentence, I presume we'd have a minimum of 12 and a maximum of 250. But since the only consistent number in all three sources is 250, why isn't the minimum 250 and the maximum 250?

Any mathematicians out there?

But of course anybody with about a 9th grade education can see that this methodology is a specious and unreliable way to actually assess civilian casualties.

The IBCP lists reputable news sources, as if they are confirming the actual numbers. But they use the numbers simply quoted by secondary sources in the stories, who may or may not be reliable. "Doctors say" is good enough for the IBCP. There is no confirmation from the reporter or from any other independent source. They're not quoting the Red Cross or Human Rights Watch. Does this really seem like a good way to try to get at the truth? By data mining news stories for numbers and quoting them no matter how substantiated or reliable the original source?

Unfortunately, I would not be shocked to find that civilian casualties were much higher than those currently quoted by the IBCP. Nor would I be surprised to find they were much lower (though I would probably guess they are higher, if anything). Most of the news pouring out of Iraq right now is extremely unreliable.

And besides the ever-present fog of war, there are other factors that are making civilian casualties in the current conflict difficult to accurately estimate. For one, we have dozens of reports of Iraqi soldiers dressing in civilian clothing. There have also been suicide bombings carried out by civilians. The line between combatant and civilian has been significantly blurred.

The truth of anything close to reliable numbers of civilian dead will probably not be known until most of the dust settles. As I've said, I wouldn't be surprised if the numbers were higher.

But I wouldn't put an ounce of confidence in the ticker rolling over at the Iraqi Body Count Project. It's a horribly sloppy, highly questionable way of making any kind of accurate assessment of the deaths of Iraqi civilians.

(And finally, on a note of irony, the most recent entry by Charlie Stross as of this writing is entitled "A digression into statistical lies".)