Sunday, September 03, 2006

It's the Data, Professor

Today’s Week in Review section of the NY Times has an article on mid-term elections by David Greenberg, who is ‘assistant professor of Journalism and Media Studies and History at Rutgers University’. Greenberg’s column focuses on 6 particular mid-terms, but reaches no solid conclusions. An excerpt:

The party occupying the White House almost always loses seats in midterms. One theory political scientists give to explain this tendency is called “surge and decline.” It notes that most presidents have coattails when they are elected, carrying the party’s candidates into Congress. But in other years those legislators have to run without the presidential surge, and many lose.

But the extent of a party’s losses can vary. A different theory of voting behavior, which considers midterm elections to be a referendum on the president, helps explain why. Evidence suggests that when voters support the president — especially his handling of the economy — his party loses fewer seats than when they’re unhappy. Some scholars have also noted that midterm turnout is highest among people who want to punish the president, which helps to account for his party’s ill fortunes.

Of course, not all midterms are created equal. Some occur in the aftermath of scandal. The 1974 elections, held just months after Richard Nixon resigned over Watergate, were a windfall for the Democrats, who gained 48 House and 4 Senate seats.

Other times, the economy influences the outcome. Recessions exacerbated Republican losses in both 1958 and 1982.

This is the drift of Professor Greenberg’s non-thesis. Drivers of mid-term elections are the ‘surge and decline’ phenomenon, or voter unhappiness, or the economy, or war, or scandal, or something else, or none of these.

This is too bad, because while making such a muddled and hedged analysis, Greenberg completely misses the most important trend in House elections. In the spirit of Edward Tufte, why don’t we just take the radical step of simply looking at the actual data. As Tufte explains, a solid presentation of relevant data speaks volumes and adds clarity to any analysis. In his book 'Visual Explanations', Tufte disscects the bureaucratic failures leading to the disasterous 1986 launch of the space shuttle Challenger and writes:

'When assessing evidence, it is helpful to see a full data matrix, all observations for all variables, those private numbers from which the public displays are constructed. No telling what will turn up.'

With help from Wikipedia and only about 15 minutes of work, I have taken Tufte's advice and plotted some actual data. Let’s have a look and see what turns up.

Here is a time-series plot of all Congressional elections from 1942-2004 with the Y-axis plotting the net change in seats between the 2 political parties. Just looking at the chart tells you right away that something significant has happened during the last 20 years or so. Recent elections have resulted in far smaller changes in party alignment. Clearly this is the most significant trend in the data and one that merits serious research and commentary. One could enrich this elementary chart by also graphing the re-election rate of incumbents or other variables. Clearly ALL recent Congressional elections (not just mid-term elections) resulted in less change in party position than was the norm before 1986. Mid-term has nothing to do with this trend. The 1994 election is a notable outlier. Ironically, Professor Greenberg’s data-poor analysis does not even mention the 1994 election.

You can also tell by a quick inspection of the chart that only 1 out of the last 10 elections changed the party makeup of the House as much as will be required in 2006 to tip the House back to the Democrats (a change of 15 seats).

One lowly blogger could develop this time-series chart in a few minutes. What did the Professor and the New York Times provide as a graphical supplement for with their article? They provided this poor graphic, which Professor Tufte would accurately call ‘chart-junk’. I’m sure the Times and Professor Greenberg spent far more time making this pathetic chart than the 15 minutes I spent creating the time-series trend. Which chart informs more? Clearly the time-series does.

It speaks to the sorry state of the Sunday Times and of the academy that they didn’t produce better work than this.

No comments: