For novel ideas about building embedded systems (both hardware and firmware), join the 35,000 engineers who subscribe to The Embedded Muse, a free biweekly newsletter. The Muse has no hype and no vendor PR. Click here to subscribe.
By Jack Ganssle
I'll present my Better Firmware Faster seminar in Melbourne and Perth, Australia February 20 and 26th. All are invited. More info here. The early registration discount ends January 20.
California's recall election will be tallied by a mix of voting machines, from punched cards to the latest in high tech wizardry. Anyone following the comp.risks forum knows of the furor over electronic voting machines.
That's a strong statement, but applies to any product that does not fulfill its mission. In the case of voting, the only important feature is trust. And few computer scientists feel the devices deliver an accurate count.
Vendors claim their machines work correctly and are tamper-proof, citing the Federal Election Commission's standards. Well, check `em out at http://www.fec.gov/pages/vssfinal/vss.html. Any computer jock with the faintest knowledge of building good code will be appalled.
The FEC mandates compliance to a primitive set of firmware standards which are woefully incomplete and simply wrong. One rule limits IFs to a max of 6 levels of nesting. My rule is 3, since none of us are smart enough to understand all of the permutations that explode with each extra nesting level. It's almost impossible to design tests to exercise all of the possibilities engendered by so much nesting.
And "test" is the name of the game for certifying voting machines. The standards propose various and inadequate testing requirements at the expense of design and code analysis. We know that testing is indeed important, but testing never guarantees correctness. Various studies suggest that a testing regimen checks about half the code in a typical product.
The FEC's mandates are much too weak to eliminate miscounting machines. It's time for a different approach.
Let's get the mob involved.
Don Corleone would never tolerate gambling machines that might rip off the 5 families of New York. Neither do state lotteries and casinos. They know how to instill trust in their products, trust that though everyone loses, customers know by how much. Customers would flock to other casinos at the faintest hint of a cheating machine.
Outside contractors verify the integrity of all gaming machines, electronic or otherwise. They do this so thoroughly that granny hasn't a care in the world when she pulls the lever of the one-armed bandit.
One such outside auditor is Gaming Laboratories International (www.gaminglabs.com) (GLI). To certify a new device, or even a software upgrade, vendors send GLI all of the source code, all of the tools needed to build the code, maybe a development computer, and even an in-circuit emulator if that's how you debugged your code. Expensive? You bet. Accurate? It sure seems so.
GLI tears the design apart, digs into the guts, finds back-doors impossible to isolate via testing and insures the customer will lose by exactly the amount specified. Tests check both functionality and threat resistance. Technicians zap every square inch of the gaming machine with a 27 KV prod - because cheaters often try to rip off the devices using ESD to confuse the electronics. GLI jimmies the coinbox, and generally simulates all of the attacks observed by those hidden cameras in the casino's roof. That's regression testing of a whole new order.
Gaming machines using Flash must physically disconnect the write line; GLI recommends cutting the PCB track. That's a lesson the FEC needs to learn.
Change the code - even just one line - and the whole process repeats. The FEC has no such requirement.
Testers even spill liquids on the machine, emulating the tipsy patrons swilling free booze. That's worthwhile for voting machines, too, as an altered state of awareness might be the best way to vote in the California gubernatorial circus.
If a gaming auditor certified voting machines, elections wouldn't be so much of a, uh, crap-shoot.