O'Dell Testimony for Hand-Counted Paper Ballots, Texas Legislature, 04.04.07

My name is Bruce O’Dell, and I am a self-employed information technology consultant based in Minneapolis, Minnesota. I have twenty five years experience specializing in the design of very large scale computer systems with extraordinary requirements for security and integrity. For example, as an employee of American Express, I led a project to design a computer security service to authorize access to financial systems across that company and with other financial institutions throughout North America. In 2005 I was the architect in charge of integrating existing systems with a comprehensive new company-wide security environment at one of the 20 largest public companies in America. So I would like to thank the committee for the invitation to share my perspective on electronic voting, as someone accountable for the security and integrity of computer systems which safely handle billions - or trillions - of dollars of other people’s money.

Since the heady days of the 1960s, a new, multi-billion-dollar electronic voting industry with world-wide growth aspirations has emerged. Whether the original drive to automate our voting was driven by genuine desire to improve elections, naive faith in progress, blissful ignorance of the potential threats, bad technical advice or coldly calculated self-interest, that industry is now so entrenched it has now become almost impossible to question the original decision to apply computer technology to voting.

This problem has been decades in the making, but was brought to a head by the “Help America Vote Act” of 2002 (HAVA). Passed in the aftermath of the disputed presidential election in 2000, HAVA was intended to improve the process of voting in America. But as a direct result of its enactment, a new wave of secret and proprietary computerized voting technology has taken almost total control of American elections.

With thousands of reported problems nationwide affecting newly-deployed electronic voting equipment in the subsequent elections of 2002, 2004 and 2006, it is clear that HAVA has had precisely the opposite effect to its stated intention, and Texas has certainly seen its share of so-called glitches. As an information technology professional I am dismayed that all this has been allowed to happen with the blessing and active participation of so many of my colleagues, many of whom make their living promoting e-voting technologies. Billions of dollars have been spent on new voting equipment in the absence of what I would consider adequate disclosure of the true costs and risks to policy makers and the general public. This is a disservice to those who must rely on IT professionals to assess the technologies they do not understand.

The proposed Texas House Bill 1364, “AN ACT relating to the accuracy, security, and reliability of certain electronic voting systems”, acknowledges the very real risks of e-voting and it seeks to mitigate them by mandating a more rigorous testing protocol for Texas’ DRE (Direct Recording Electronic touchscreen) voting equipment, along with enhanced tracking of access to voting equipment and the chain of custody of electronic storage media.

To avoid reprising HAVA’s harmful side effects despite the best of intentions, and to help restore and maintain public trust and confidence in our electoral system, this committee needs to look much more closely at the fundamental issues of software and voting.

Ensuring the integrity of systems is the hardest of all challenges in computing, and in too many cases, my profession has failed to adequately protect our employers and the public.

One of the primary reasons why trustworthy technology is so hard to achieve is that the mind-boggling complexity of real-world systems provides an enormous number of potential points of vulnerability. Voting hardware is deployed at more than 180,000 precincts and in more than three thousand counties in the US - more than 7,000 polling places just in the state of Texas. The physical logistics of moving all that equipment out to the field and getting election results back to the central tabulators for the official canvass is challenging, and as we will see, House Bill 1364’s procedural safeguards, while well-meant, are utterly inadequate to safeguard the integrity of the end-to-end system.

Not only are there thousands and thousands of devices, there are thousands of individual hardware and software components within each voting device. This includes proprietary software developed by voting equipment vendors, mass market consumer products like Microsoft Windows, and a host of highly complex, very specialized software - with no visible behaviors - supplied by a long list of other vendors, many of them offshore.

In addition to all the devices and their individual components, we must also consider the collective actions of the thousands of people who participate, directly or indirectly, in designing, programming, testing, distributing, manufacturing, installing, maintaining, configuring, operating, transporting, monitoring, repairing and storing the millions of hardware and software components that collectively add up to our system of electronic voting.

House Bill 1364 calls for testing and controls to be applied to several aspects of the end-to-end voting process, but no amount of testing alone can conjure trust in the overall system.

It is well known in the information technology profession that computers are ultimately "black boxes" - you cannot actually see what bits are really present and executing; and all methods to attempt to do so require other software that itself has the same problem, in an infinite regress. There is no workaround.

The only way to know what is running in a computer at any given moment is to observe its behavior: give all possible inputs, measure its corresponding outputs, and then check to see if the inputs and outputs you observe match the specification.

It is reasonable to ask if computer software is always tested before use, why bother to produce an “audit trail”? Unfortunately, you really have no guarantee that a given computer program's behavior as measured, say, at 10:00 AM will have any relationship to the same program's execution at noon. Computers have clocks and can tell time, and can easily be programmed to behave differently at different times, on different dates – or under an endless variety of different circumstances.

When it comes to systems processing high-value transactions of interest to potential criminal embezzlers - like money or votes - the inherent limitations of point-in-time behavioral testing make it unacceptably risky. Some kind of computer behavioral monitoring system is required to record a vulnerable system's inputs and corresponding outputs while it is processing critical transactions. This would provide all the information needed to enable a human auditor or another automated auditing system to spot processing errors or manipulation of the transactions.

I know that many people make an analogy between computerized banking and computerized voting. For example, Michael Shamos, a noted expert in the field, advocate of computerized voting, and a long-time paid consultant to states on the certification of their electronic voting systems has stated:

“Why should voting systems be held to a standard of perfection when nothing else in society is? Nonetheless, electronic voting watchdogs insist that election equipment must be perfect or it is totally unusable. The analogy between voting systems and the bank is particularly apt because (1) the chance of a system being tampered with successfully is low; (2) even successful tampering does not necessarily result in the wrong candidate being elected; and (3) only a small portion of the vote is cast on one machine.”

Unfortunately, not only is there good reason to dispute each of his three assertions, computerized voting and computerized banking actually have almost nothing in common.

One reason why electronic financial transactions are as secure as they are (by which I only mean that embezzlement is the exception and not the rule) is that while financial transactions are private, they are hardly anonymous; you need to prove your identity to all the other counterparties involved. Each counterparty gets and keeps their own independent records of the transaction, all counterparties are strongly motivated to spot discrepancies and compare their records with others, while procedures relating to resolution of financial disputes are legally mature.

Why are voting systems so difficult to protect? In contrast with banking, voting is a private and anonymous transaction. Applying the conventional counterparty-based financial auditing mechanisms to voting transactions as they occur would compromise the confidentiality of the vote and voter and would in fact be illegal.

To meet the standards of banking, not only would multiple independent copies of audit records fully describing the voter’s identity and ballot choices need to be generated and shared with multiple parties, 100% of those transaction records would be routinely audited and the results compared across organizations. In voting, on the other hand, only a relative few states routinely audit their paper ballot records (if they have any) and then only a few percent of the precincts are checked. If a bank audited only a few percent of its accounts - or none at all, its customers would flee, regulators would shut it down, and the Board of Directors would face possible jail time.

Although some computer scientists feel they've identified some magical all-electronic means of auditing the accuracy of DRE internal vote totals, ultimately there is no reliable means to do so. At the moment of creating the electronic audit record, the computer could be programmed to electronically assert you input “Smith for Governor" even though you actually input "Jones for Governor". Every all-electronic internal DRE auditing scheme, no matter how elaborate, would from that point on then simply record a lie with every appearance of the truth.

The only way voters can protect themselves from such a consistently-told electronic lie is with some kind of corresponding tangible, visible receipt that could be used as a proof you really voted for Jones. Unlike in banking, we cannot give a voter a receipt or a monthly statement; the best we can do is create an anonymous receipt that says the equivalent of "Someone Voted for Jones", have the voter verify the accuracy of that assertion, and then deposit it with the electoral authorities to retain for future auditing or recounting.

The risks of errors and covert manipulation are inherent to the use of computer software. Human nature being what it is, those risks are ever-present in all systems that process high-value transactions - especially those involving money or voting. So to achieve trustworthiness, auditing of an independent ballot record would always be performed.

Both the accuracy and integrity of any paper ballot record must also be assured.

“Accuracy” means that every voter checks that the paper record accurately records their intent. Yet studies show an abysmally low rate of detection in the field of problems with voter-verified paper audit trails (VVPATs) created by DREs; election outcomes can never be known to a greater accuracy than the rate at which voters accurately verify their intent. Paper ballots (whether tabulated by hand or optical scan) have much higher accuracy than VVPATs since the audit record is the same thing as the vote-casting record and inherently demands much more scrutiny by the voter.

To ensure integrity, no one must be able to alter, delete, or substitute paper ballot records after they are verified by the voter. Immediately after the election, traditional paper-based audit and control concerns take precedence. In general, the more time passes since creation and the further it travels from point of origin, the more risk there is of manipulation or destruction of paper records.

Unfortunately, there is no such thing as perfect security; the best we can do is to mitigate the risks as best we can. In recognition of this inherent problem, the Canadian system of counting paper ballots in-precinct on election night - in concert with their absentee/early voting procedure - is highly secure. The paper flow is always under observation, and ballots are immediately counted in front of multiple adversarial counterparties - namely the political party representatives.

Admittedly, even rigorous paper-handling processes are not perfectly secure - but on the other hand, in the last 600 years of general use of paper records, we have figured out some pretty good procedures. Yet I doubt that many jurisdictions in America handle paper election records with the level of custodial care that we find, say, in handling real estate collateral in the mortgage-backed securities market, much less in Canadian elections.

So as a practical matter, I'd have to conclude that simply having a VVPAT offers ultimately no assurance of practical "auditability" - the records in the field are only as accurate as the rate at which people actually verify them, and with the passage of time are increasingly unlikely to have a clear, secure chain of custody. The same applies to optical scan ballots.

There are additional practical problems with checking the trustworthiness of an electronic vote tally after the fact. Since paper ballot records are typically not recounted unless margins are very close, brazen theft would be rewarded in practice. No candidate losing by a large margin wants to challenge an election and force a recount. Political culture being what it is in America, they quickly get labeled as "sore losers" who "waste the public's money and the government's time" on pointless recounts, and who use "conspiracy theories" to compensate for their inability to admit they lost.

Even when recounts of paper ballot records occur, recent experiences in Ohio and Washington state clearly reveal fundamental flaws in both approach and execution. Recounts are "broken" and existing spot-audit protocols are subject to the same limitations, as well.

Not only are there fundamental limitations to our ability to prove the trustworthiness of any complex real-world computing system, voting itself deserves the strongest degree of protection. Many of my colleagues (perhaps more so, for those gaining financially by their involvement with electronic voting industry) seem to utterly miss the essential point: computerized voting systems should be classified as national defense systems demanding a much higher standard of protection than conventional applications - including mere banking software.

Undetected widespread covert manipulation of computerized voting systems is the functional equivalent of invasion and occupation by a foreign power. In either case the people lose control of their own destinies, perhaps permanently. Undetected covert manipulation of voting systems could even be worse than mere invasion, since the “electoral coup” would appear to occur with the illusion of the manufactured consent of the governed, and there would be no “tanks in the street” to galvanize resistance.

Voting systems used in American federal elections grant regulatory powers over the world’s largest economy, disbursement authority for the federal procurement budget, control of the composition of the Supreme Court and federal judiciary, and command of the world’s only superpower military. Texas would be among the world’s wealthiest nations if it were an independent country, and it is clear that the financial rewards for covert control of state elections are vast as well.

Yet despite the fact that our computerized voting systems represent the most irresistible target for insider manipulation in the history of the world, they are not currently given even the same level of protection as systems I’m familiar with in banking and financial services, nor even to computerized gaming equipment in Las Vegas. This is a national scandal, and a disgraceful failure on the part of my profession. House Bill 1364 continues the unfortunate historical pattern of misunderstanding and underestimating the seriousness of the threat to computerized voting systems while putting in place ineffective countermeasures.

Independent inspection and certification of source code has no real benefit. If a malicious insider at Diebold or ES&S truly wanted to corrupt vote tabulation logic, they would hardly put it in the official release handed over for review. There’s simply no reason to trust that any software delivered for inspection bears any relationship whatsoever to the logic that actually runs on voting devices in an election.

The language in House Bill 1364 regarding “check and verification check” is also suspect. Since real-world computer systems involve complex inventories of hundreds or even thousands of application program modules, firmware, device drivers and operating system components, static inspection alone will never be able to reliably determine what those components will actually do at any given point in time. There’s simply no reason to believe that given executable corresponds to the given source code, and no way to truly know what the executable is doing - except by running it. Static inspection is not a security measure.

Nor can we test security into software. It is a truism in my profession that the purpose of testing is to find “bugs” - not to indicate that a piece of software contains no flaws. It’s a subtle point, but what it really means is that if I’ve found 100 errors, there is simply no magic oracle that will then tell me “well, that’s all, we’re done, no more bugs”. If it was possible to test quality - much less security - into any piece of software Microsoft Windows would be the bug-free, highly secure platform we all know it to be, since Microsoft has the world’s most sophisticated automated testing tools, thousands of paid testers, and hundreds of thousands of people worldwide who volunteer to help. Yet even so several critical Microsoft security defects have been reported every month for the last several years. But not to pick on Microsoft; Secunia, a Danish company, has nearly a seven hundred page listing of security issues in popular software; in every case these flaws were discovered after completion of formal testing.

As socially-responsible professionals we must openly acknowledge the inherent limitations of our ability to ensure voting is as trustworthy as a critical national security system should be. We cannot and should not ask the public to simply trust the outcome of any testing and certification process, no matter how many “experts” say so.

In fact, there is a fascinating study from 2001 (interestingly enough, published shortly before HAVA was enacted) which concluded that not only were hand-counted paper ballots the most accurate of all vote counting methods, measuring by residual vote rate, but that every single technological “innovation” of the last century - lever machines, punch cards, optical scan, DRE - actually measurably decreased the accuracy of the voting process. Their conclusion:

These results are a stark warning of how difficult it is to implement new voting technologies. People worked hard to develop these new technologies. Election officials carefully evaluated the systems, with increasing attentiveness over the last decade. The result: our best efforts applying computer technology have decreased the accuracy of elections, to the point where the true outcomes of many races are unknowable.

There is an entire industry which is predicated on the belief that computers are better than people when it comes to counting votes, yet the precise nature of the problem that electronic voting was intended to solve remains unclear. The balance of evidence indicates that while voting by DRE may well be wide open to insider manipulation, and in practice has been plagued by glitches and inaccuracies, at least it’s far more expensive than the alternatives. Even with optical scan balloting, the effort required to hand-check machine tallies undermines the rationale for automation in the first place.

The fundamental question - why should machines tally our votes in secret - remains unanswered. Other than for the obvious financial benefit of the vendors, why should voting be forever defined as a transaction to be tallied in secret by machines, and never as a civic transaction to be performed by people in public view?

In the final analysis, I believe computer automation of voting will be regarded by future historians as one of the greatest blunders in the history of technology. Our choice now is to determine at what price - both in money and public good will - that realization will finally strike home.