From "Slavery and the Founders: Race & Liberty in the Age of Jefferson" by Paul Finkelman (2nd ed.):

"One difficulty... of a serious nature" made election by the people impossible (according to James Madison). Madison noted that the "right of suffrage was much more diffusive in the Northern than the southern States; and the latter could have no influence in the election on the score of the Negroes." In order to guarantee that the nonvoting slaves could nevertheless influence the presidential election, Madison favored the creation of the electoral college.

Oh, man. That supports everything I've ever thought about the electoral college - it doesn't make sense, it's fundamentally anti-democratic, and needs to be replaced. Interestingly, the man who wrote this book is also our guest lecturer this week. Should be fascinating.