Fourth World Logo Fourth World Media Corporation
  Embassy Services Products Resources   About Fourth World Contact  
logo

Update for November 2004:

Software Security and Usability Issues Create
Unnecessary Obstacles for Reliable Vote Count

If your child hurt himself with a hammer, your solution would probably not be to replace it with a power drill. It would be more dangerous, and still wouldn't drive a nail.

Yet that's pretty much the response from various elections boards across the United States to the usability issues of Florida's 2000 ballot.

Rather than simply hire usability professionals to solve the usability problem, they've chosen instead to spend tens of millions of taxpayer dollars completely replacing the entire voting system that had served the nation well through dozens of elections with new technology that is largely untested, arguably unsecured, often has no accountable paper trail, and doesn't address the usability issues that were at the heart of the issue in the 2000 elections.

And it doesn't make voters any more at ease to learn that the CEO of the leading vendor of these new voting machines has publicly expressed a commitment to deliver votes in a key swing state to one of the presidential contenders.

Many voters are just as confused by the new computer-based systems as they were by the traditional paper-based ones, and sadly that's the least of concerns in this next election.

Far more critical is that this untested technology is reported to be riddled with security holes, exposing the fate of the nation's leadership to simple hacking. The source code for the software that drives the system is proprietary and has not been subject to public review.

In the 2002 elections, electronic voting resulted in thousands of lost votes in California, Florida, New Mexico, and elsewhere. California is suing the manufacturer of its voting machines, saying the company’s equipment left its elections open to hackers and software bugs, and the state has officially banned all paperless e-voting machines.

US voters have become so concerned about having a fair election that the nation that sometimes calls itself "the beacon of democracy" now has to have international observers overseeing the elections process, much as is done in third-world countries.

While allegations of other forms of voter fraud are beyond what technology can address, it should be a simple matter to employ technology that can count a vote, and do so in a way that is reliable, accountable, and auditable.

The paper-based system that was proven effective for decades suffered only from minor usability issues. But rather than employ usability experts to fix the problem, voters are now faced with newer, larger problems. And they're paying millions to e-voting machine manufacturers for the priviledge.

Having a reliable voting process is not a partisan issue. But neither is it a technology issue, and turning it into one has not given voters any greater confidence in the process.

While the integrity of the November 2004 elections appears primed for controversy, perhaps the only thing we can be assured of is the revenue e-voting manufacturers will enjoy from this debacle.

Meanwhile, the fate of the free world hangs by an Ethernet cable transferring easily hackable data....

For more:

Black Box Voting: Ballot-tampering in the 21st Century

IT Analysis: Electronic Voting and the US Election

PC Magazine: Is E-Voting Fundamentally Flawed?

Electronic Voting Raises Concerns About Presidential Election

Computer scientists worry about electronic voting machines

Electronic voting sparks worries over lost Florida votes

E-Voting: Is The Fix In?

Can Voting Machines Be Trusted?

Voting Machine Controversy

Diebold Wines and Dines Election Officials

Rage Against the Voting Machines

Trust, transparency, democracy

New Scientist: The great American e-voting experiment

Time to Recall E-Vote Machines?

E-Voting Undermined by Sloppiness

International team to monitor presidential election

Russian Officials To Monitor U.S. Vote


From November 2000:

Usability Issue Contributes to Election Confusion

Florida Voters Sue Over Ballot Design

The ballot for West Palm Beach and other sections of Florida was described as confusing, prompting complaints from several hundred voters that they may have accidentally voted for Pat Buchanan when they had intended to vote for Al Gore. Being primarily a perceptual issue, in addition to those who have filed complaints there may be many who made the mistake without being aware of it. The exact number of affected votes cannot be known.

The Problem

The reported problem with the ballot design was that voters wishing to vote for the second candidate on the list actually needed to punch the third hole on the form, with the second hole designating Buchanan.

This problem was said to have been compounded by having had a sample ballot distributed in advance which looked visually similar but had two columns for the punch holes, one for each side of the listings. In the sample ballot, a punch in the second hole from the top would have indicated a vote for Gore.

Another contributing factor may be the age of the majority of those who have submitted the complaint: the affected counties have a disproportionate number of retired people. With a demographic in that age range, a designer would do well to take vision anomalies into account.

Somewhere in Florida a government worker with a copy of PageMaker is having a very bad day.

20/20 Hindsight

As a part of the design process, the proposed layout for this ballot was submitted to Florida state officials as well as authorities in both parties The design apparently met with approval, but there is no indication in the press thus far that anyone in the review process has a background in usability, nor even that such considerations were even explicitely part of the review.

Most usability professionals would likely agree that reliance on "managerial review" without direct user testing is unlikely to yield truly useful results. With any design deliverable, the review process and the user testing process serve very different purposes, with the participants of each looking for very different things. Moreover, few managers have the background in user-centered design to even begin to anticipate user needs, let alone be able to conduct objective user testing.

If you've ever done usability for large organizations, you know how it goes with "managerial review": each stakeholder is looking only for specific aspects which are important to them (spelling, font size, visual prominence), and without explicitely calling usability issues to their attention they are unlikely to consider such things on their own.

In the absence of user testing or even heuristic evaluation, as with many areas of design the format of ballots in Florida is supported by a set of published guidelines. Like the Human Interface Guidelines for Macintosh and Windows operating systems, exisiting documentation on a system anticipates common design questions and attempt to provide solutions in advance based on current research. In Florida, the guidelines would appear to anticipate the potential ambiguity of a layout with two facing pages, as they are said to suggest a single-column layout which places the punch hole in a consistent position to the right of each candidate.

Legal specialists are said to be revewing the ballot design for its compliance with the specifications, but it doesn't appear that any usability professionals have yet been employed in that review.

Preventative Measures for Next Time

Given how rarely any form undergoes formal usability testing (the IRS reportedly tests some form designs, but I'm not familiar with such evaluations for other government agencies), it may not be reasonable to assert that a need for formal user tests should have been identified by the design team, no matter how desirable it may appear in retrospect.

But as with any system, adhering to existing standards will almost always server the end user well. If the ballot is indeed a departure from guidelines suggesting a single-column design, it is less interesting to me whether or not the guidelines carry any legal weight than that they appear to be good ones, recognizing the importance of the two core principles of good interface design: consistency and simplicity. Such principles become especially important in systems used only infrequently, as the user has little chance to acclimate to any inconsistencies within the interface. A ballot system used only once every two years.

In the absence of guidelines, researching other good designs would provide a good solution. Most other counties across Florida and through the most of the US use a single-column format, obviating the issue of confusion over the two-column layout altogether.

We don't yet know if the number of votes in question may turn out to be significant enough to make the difference in this Presidential, but with the current lead only 215 votes apart as of this writing the real impact is unknown.

No matter how this issue is resolved, no single event has brought as much visibility to a usability problem as this one, and it is perhaps the strongest argument our profession has seen for the importance of user testing.

More Info

For a little background in design considerations for ballots, I couldn't find any of the Florida state guidelines online, but I did find this interesting snippet from the Vermont Commission to Study Instant Runoff Voting. (If you happen to have a URL for Federal or Florida state ballot design guidelines, please drop me a note and I'll add it here.

For the general news info, see these stories at MSNBC.com and CNN.com.

There's a great take on this usability angle added late today at AskTog.com.

This article by Dan Briklin has one of the most detailed usability discussions on the matter.

The issue is also the cover story at Jakob Nielson's UseIt.com.

Larry Magid's article at Upside covers the usability aspect as well.

And this article from the Sun-Sentinal quote Buchanan as supporting the suggestion that the ballot design played a role in the number of votes he received in affected counties.

This article from Salon says that county official knew about ballot confusion by the middle of election day and distributed a memo to workers about it.

Dr. Greg Adams of Carnegie Mellon posted this statistical analysis to illustrate the scope of the problem.

John Marden, a statistics professor at the University of Illinois, Urbana-Champaign, posted this interactive Java applet which further shows the broad variance.

Still not convinced that usability can play such a critical role in the outcome of significant events? Check out Tog's article "When Interfaces Kill", describing just one of many examples how good interface design, in hardware or software, can even make the difference between life and death.