The Great Exit Poll Debate (Bruce O'Dell)

by Bruce O'Dell

I recently encountered yet another exit poll debate and related discussion and felt the need to comment once more on the topic of the 2004 exit poll controversy.

Limitations of the available exit poll data sure haven’t hampered the debate
Shortly before resigning from the organization I co-founded in 2004 to scientifically study the accuracy of US elections (US Count Votes) I reached the conclusion that all the available public exit poll data was consistent with widespread electoral fraud, with widespread bad polling techniques - and, worse yet, an unfathomable combination of the two.

My argument at the time: that publicly available exit poll data alone cannot in itself prove that the election of 2004, or certain elections in 2002, 2000, or 1996 were fraudulent, greatly displeased those who had concluded otherwise, including the pseudonymous "TIA" cited in Lindeman's article above.

The other conclusion that I reached - namely, when you look outside the narrow domain of exit polls, you see considerable support for the possibility of widespread covert vote manipulation - unfortunately failed to register with those on the opposite side of the exit poll debate from "TIA".

Limited data, limited conclusions
Lindeman and Liddle insist on application of the scientific method to first identify testable hypotheses, and then apply them to the available data. The problem with exit poll data is that the data available to the public is aggregated at the state level and is truly inadequate to test any hypothesis involving voting where it occurs - at the precinct level.

It is true that Liddle was given privileged access to E/M precinct-level data on a non-disclosure basis, and from that data drew a number of conclusions - that, unfortunately, could not be independently reproduced.

“Exit poll errors don’t correlate with precinct partisanship”… say what?
One of her key conclusions was that when looking at the secret precinct level exit poll data, exit poll errors relative to the official tally were not at all correlated with partisanship - as some studies by US Count Votes based on aggregate data seemed to indicate.

One model of vote fraud assumes "vote padding" occurs only in partisan strongholds where it can be more readily explained. Warren Mitofsky and others touted Liddle's finding as a rebuttal of any possibility of fraud, since exit poll discrepancies favoring Kerry happened everywhere to the same degree.

Somehow, all over the country, in Democratic and Republican strongholds and evenly-matched precincts as well, Kerry voters were overall just as equally inclined to take an exit poll more often than Bush voters (or, Bush voters were equally less likely inclined to participate in an exit poll - the data is consistent with either interpretation).

If there's been a coherent behavioral explanation for such oddly-uniform behavior, all across the country, on the part of people participating in an exit poll that is not a wordy restatement of "because", I've sure missed it - although I admit I largely stopped following the exit poll "debate" last summer.

Actually, the publicly released fraction of Liddle's secret precinct-level analysis was just as consistent with widespread covert vote manipulation as with inexplicable polling behaviors – or a mixture of the two..

The “Big Picture”
As long as we are confined to the narrow domain of exit polls and in the absence of full disclosure of the data, these questions were, are and will remain unanswerable. If we look outside the realm of exit polls briefly in order look at the "big picture", what do we see?

We now know for certain there are a host of systematic vulnerabilities in the all the components of electronic vote counting technology - DRE, op-scan, and tabulator - that tally the vast majority of the US vote. Given the nature of end-to-end voting systems and considering how they are "tested", deployed to the field, operated, and "examined" after the fact, widespread covert vote count corruption is certainly technically feasible.

Nor would it require a "massive conspiracy" to implement, even on a national scale. Changes to the master copy of voting software are replicated to thousands of machines in what amounts to an industrial process; the behavior of thousands of devices can be changed if just one person corrupts the master copy. Worse yet, all too many voting machines have built-in modems. These are stated to be secure by the equipment vendors, but the most secure military, government and industrial networks ban installation of modems, no matter how purportedly secure, due to the inherent security risks.

Reported vote-switching was not random
Setting aside the large number of reported irregularities involving purged registration rolls, provisional ballot irregularities, "dirty tricks", misallocation of voting equipment, and so on, consider just the one-sided nature of anecdotal reports of visible vote switching. For example, 87 of 94 vote-switching reports indicated voters who saw their DRE vote visibly switched from Kerry to Bush (in the EIRS sample).

Common sense would tell us hypothetical individuals perpetrating widespread covert vote count corruption would be highly motivated to do so in such a way as to remain undetected. One could argue that visible vote-switching is an argument against the systematic fraud hypothesis, or at the very least evidence of poor but innocent programming practices.

“Denial of service”?
But I believe that visible vote-switching could be a deliberate "denial of service" tactic. By "denial of service" I mean a deliberate tactic intended to make the voting process itself so slow and unreliable that voters simply give up and go home. Imagine the rationale for a covert vote manipulator to visibly switch the vote from one candidate to the other: if not noticed (often apparent only on the final "review your vote" screen), the vote is switched successfully. If it is noticed, the voter will probably panic - at least attempt to revote, possibly many times - slowing the entire voting process at that precinct. The net result, regardless of the final outcome, is to reduce the number of votes for candidate "A" in that location. The fact that the voting software itself is not accessible to independent review - and in fact can be programmed to modify itself at the end of the election - means that such an exploit would never be detectable after the fact, even if investigated.

The fact that visible vote switching occurring in multiple jurisdictions, involving software and equipment from multiple vendors, all overwhelmingly favored one candidate indicates to me that these were not random programming errors.

Given the highly charged partisan nature of the 2004 election, I simply find it impossible to believe that Bush supporters would not publicize incidents where their e-vote appeared to switch to Kerry. (Or were there "reluctant Bush vote-switching reporters" just like the exit poll "reluctant Bush participants"?)

Less-discussed aspects of exit polls
There’s a host of other exit poll topics, but eventually you hit the same wall with all of them: the detailed data you need to do a truly thorough analysis has not been made public The data required to audit the integrity of the official precinct level vote total is entirely missing or suspect due to the passage of time.

Therefore the possible explanations remain poor polling, official vote count manipulation, or a combination of both.

But there is more.

National exit poll sample
For example: why did the huge 2004 national exit poll sample of 12,000 voters (or 14,000, depending on what E/M data you believe) predict essentially the reverse of the official national popular vote? That large sample predicted a Kerry win by 3%, instead of a loss by 2.5%. Simple statistics tells us a sample so large is extremely unlikely to be that far off by chance; but once again, the explanation is either "bad polling", because Kerry voters behaved differently than Bush voters more or less exactly the same way everywhere, or "bad vote counting" - or a combination of both.

Southern state exit poll trends
Or, if you look at E/M's published state-level exit poll accuracy data, since 1992 you see a pattern in many southern states where exit polls are increasingly inaccurate. They appear to overstate the Democratic vote more and more in every election cycle - or there is an increasing pattern of inaccuracies in the official tally favoring Republicans. Explanation? Overeager African American exit poll responders? Shy Republicans? Vote count manipulation? Take your pick.

Recent trends: toward greater inaccuracy?
Or, consider that although since 1996 exit polls have become increasingly controversial, for decades in the US, election cycle after election cycle, exit polls produced no "surprises". E/M admits to one missed call which did not go public in the 1990s; some people speculate it was the 1996 Nebraska US Senate race, where Hagel unexpectedly did so well against Nelson - and of course, where the votes were tallied by a company, ES&S, that Hagel had such recent, close, unacknowledged ties to...

What happened to Florida exit polls in 2000?
Then there was the Florida debacle in 2000, now apparently firmly believed by everyone to be a problem with the exit poll; but Warren Mitofsky himself was quoted in 2001 as saying more or less "if I had the same data [exit poll and AP trial precinct actual results] I'd call the election the same way". It appears that the exit poll discrepancy was centered in a few counties - such as Palm Beach, Broward and Volusia - where people seem to have accurately told the exit pollsters how they thought they had voted. (See an absolutely fascinating article from the Columbia Journalism Review here. )

What happened to nationwide exit polls in 2002?
After that embarrassing incident, VNS vowed to totally revamp for 2002. But depending on whom you believe, either poor internal computer programming at VNS took its toll (as reported here) and/or poll taking technique reached its nadir; or multiple Senate and Governor races that went Republican in the official tally were actually called for the Democratic candidates in the exit poll (as reported here).

No exit polls were officially released that year – in fact, VNS went out of business. It was replaced by Edison/Mitofsky and the National Election Pool, who once again vowed better polling techniques must and shall be put in place for 2004. And the result: the worst exit poll inaccuracy ever publicly admitted to.

How do we explain that sequence of events? Either exit polling, a once rather tame and predictable business with no notable track record of controversy has somehow become increasingly unreliable lately; or the vote count, with accuracy usually taken for granted overall with local exceptions, like Chicago in the 1960s, has inexplicably become increasingly unreliable lately - or, a combination of both.

So whether you conclude that it's human behavior or the integrity of the vote counting process that's changed appears to be a matter of personal preference, and to some extent, partisan ideology. If your candidate wins, most people don't complain too loudly. But it certainly appears that the rate of exit poll controversies is proportional to the degree to which increasingly sophisticated "black box" electronic voting equipment has been deployed in recent years.

...and what will happen in the future?
I admire Liddle and Lindeman, and certainly have enjoyed collaborating with them in the past. I believe some of their work has done a valuable public service by debunking overinflated, mathematically erroneous claims that exit poll data "proves" this or "disproves" that model of fraud. I think where I part company with them is in my willingness to contemplate the prospect of widespread vote count manipulation.

We have a voting system wide open to manipulation, and where there is every motivation, means and capability to evade detection - other than by the very imperfect means of exit and public opinion polling.

Under most circumstances, Liddle and Lindeman's insistence on avoiding a Type I error (i.e., concluding that systematic vote manipulation is occurring, when in fact it is not) would be commendable. But when the integrity of the American Republic is the "null hypothesis" it is a highly-misguided Type II error to insist the burden of rigorous proof lies totally with those attempting to show that severe election systems vulnerabilities are actually being exploited. Especially so, given that simple, secure and effective countermeasures, including election auditing, in-precinct hand counted paper ballots, and so on, are so readily available.

Computer security analysts presume all known vulnerabilities are actually being actively exploited and ethically feel required to immediately put counter measures in place, both to stop an exploit if it is occurring, and to keep it from happening if it has not. We do that because when protecting billions of dollars of other people's money (much less the fate of the American Republic) a Type I error of fixing or preventing a problem that may not have happened yet utterly pales in significance next to perpetuating a Type II error - failing to stop a crime in process.

That remains undone.