By John Washburn
This article was posted at Washburn's World and is reposted here with permission of the author.
I have been a long time critic of the 2002 Voting System Standards (2002 VSS) and of the 2005 Voluntary Voting System Guidelines (2005 VVSG). In fact, both sets of standards are virtually worthless. There are two reasons for this. First, the requirements enumerated in the standards are, in and of themselves, much too weak for something as vital as administering an election. Second, both sets of standards have an explicit loophole that allows almost all the requirements - weak as they are - to be ignored. This second objection was first brought to my attention two years ago by Howard Stanislevic.
We now have proof that this loophole is used by the labs in order to �pass� systems that don�t meet the standards. In the most recent certification test report submitted to the Election Assistance Commission (EAC), SysTest labs (one of the labs accredited by the EAC and the NIST) recommended certification for a voting system. SysTest recommended the system certification even though their findings showed 79 specific failures to meet the standards.
As you read this article, keep in mind that the standards actually allow 77 of these 79 failures to be ignored!
Last February, SysTest labs wrote its certification test report for a new voting system manufactured by Premier Election Solutions (formerly Diebold). The report listed the 79 problems the lab found during testing. Even so, SysTest recommended the system be certified by the EAC. Section 6 from the test report reads: SysTest Labs has successfully completed the testing of Premier�s Assure 1.2 voting system. It has been determined that Assure 1.2 successfully met the required criteria of the Federal Election Commission Voting System Standards April 2002.
Based on testing scope and results, as detailed in this report, it is SysTest Labs' recommendation that the EAC committee grant certification of Premier�s Assure 1.2 voting system.
I sent a letter to the EAC objecting to this recommendation for certification - not because of the 77 "acceptable" failures, but because of the two failures that require denial of certification.
Except for the last two findings on this list, there is nothing that would prevent ANY NIST-accredited test lab from recommending this system for certification by the EAC. None of the 2002 VSS requirements violated by the first 77 findings are enforceable requirements!
Some definitions are in order.
There are almost no enforceable requirements in either the 2002 VSS or the 2005 VVSG. This is because both standards have a conformance exception clause (loophole) which covers nearly every requirement listed in either standard. The exemption paragraph is found in section B.5 of Appendix B of Volume II of the 2002 VSS and the 2005 VVSG . From the 2002 VSS, paragraph B.5 reads [emphasis mine]: Of note, any uncorrected deficiency that does not involve the loss or corruption of voting data shall not necessarily be cause for rejection. Deficiencies of this type may include failure to fully achieve the levels of performance specified in Volume I, Sections 3 and 4 of the Standards, or failure to fully implement formal programs for qualify [sic] assurance and configuration management described in Volume I, Sections 7 and 8. The nature of the deficiency is described in detail sufficient to support the recommendation either to accept or to reject the system, and the recommendation is based on consideration of the probable effect the deficiency will have on safe and efficient system operation during all phases of election use.
The first 77 of the 79 uncorrected deficiencies listed above fit within the conformance exemption (loophole) defined in both sets of standards. Some of these 77 violations are also violations of federal law, and yet they are allowed by the standards. For example:
But, since none of these specific 18 non-conformances nor 59 of the remaining 61 non-conformances involve vote loss or corruption of voting data, none of them prevents the lab from recommending certification.
The last two finding involve the corruption or loss of voting data. So, even with the loophole of paragraph B.5, the system cannot be recommended for certification. SysTest erred when it recommended the system for certification, but only because the voting system was so broken that even the B.5 loophole would not allow the system to be recommended for certification.
SysTest's report demonstrates that with the broad nature of the loophole found in paragraph B.5, there is almost no failure that would prevent a system from being certified by the EAC, or that would have prevented qualification under the previous NASED/ITA program . So, it should come as no surprise to anyone that we read about failure after failure, in election after election, by voting systems "certified" by the NASED Voting Systems Board .
The 2002 VSS and 2005 VVSG are made toothless because of the conformance exemption found in paragraph B.5. But, as broad and eviscerating as the loophole is, it is not a blanket exemption.
The report I examined was written by SysTest about a Premier/Diebold system. But, the problems with the 2002 VSS and 2005 VVSG apply to systems from all vendors, not just Premier/Diebold, and they apply to all of the NIST-accredited test labs, not just SysTest. SysTest found 79 non-conformances to the 2002 VSS, 18 of which also violate federal law. But, because of the loophole paragraph, the system as tested, but containing only the first 77 of the 79 non-conformances, is certifiable.
The problem with the whole national certification program of the EAC is that the program is built on standards that are standards in form only, without substance. Nearly every voting system currently in the field and used in elections since November 2006 was "certified" to the 2002 VSS under the previous NASED/ITA system. Nearly every voting system to be sold to counties and municipalities until 2012 (and perhaps beyond) will be "certified" to the 2005 VVSG.
The labs are quick to state that they do not certify equipment. The EAC is quick to point out that the testing is done by the labs and not by the EAC. For years the voting equipment vendors have claimed that the 2002 VSS and 2005 VVSG are "comprehensive and rigorous". This real world application of the 2002 VSS to an actual voting system by a NIST-accredited laboratory should demonstrate how toothless and ineffective the 2002 VSS and 2005 VVSG are.
As hard as this is to believe, the EAC certification program is a substantial improvement over the qualification program administered by NASED. With EAC certification, it is possible to find out the known deficiencies that are present in the system used to administer your election. How many known deficiencies are lurking within the equipment certified by NASED and currently used in an election near you? You are not allowed to know! Neither are your local election officials.
Certification test reports, listing the deficiencies of the systems certified by NASED, will never be published. This is because the reports are considered trade secrets by the vendors. The reports are also considered trade secrets by the ITA testing labs that tested the systems. Most importantly, the reports are also considered trade secrets by the chairman and every member of the NASED Voting Systems Board.
This means that systems with known deficiencies certified by NASED were sold to unsuspecting counties and municipalities. With the old NASED/ITA program, these local election officials were left in the dark and never had the opportunity to learn of known deficiencies before they used the voting systems in elections. Instead, they get to learn of the deficiencies as the systems fail during the administration of real elections in the real world.
At least, under the EAC certification system, local election officials have a chance to learn how their voting systems will fail before they use the voting system to administer a real election. This is how the certification process has "improved" under the EAC.
Is this sufficient improvement? Certainly not. Listing the known deficiencies in obscure, technical reports on a government website has some small benefit. Better still, don�t certify systems that violate the standards. Under the standards though, the NIST-accredited labs can recommend certification and the EAC can certify systems which violate most of the requirements of the standards. In my opinion the EAC should not certify such systems nor should the NIST-accredited labs make such a recommendation. But, the standards as written and enforced almost never prohibit certification of a voting system.
The central problem is that both standards (the 2002 VSS and the 2005 VVSG) allow lax and probably uneven enforcement of nearly all of the system requirements listed in the standards. This is because the loophole created by paragraph B.5 is nearly without limitation, moreover, paragraph B.5 provides no meaningful guidelines on when to enforce and when not to enforce a system requirement found in the standard. It is possible the same problem may be found in two different systems and one system is certified and the other is not.
Until this core loophole in paragraph B.5 is closed:
As long as a voting system does not lose or corrupt vote totals, the voting system can violate any or all other requirements listed in the standards and still be certifiable.
This is why the "standards" (2002 VSS and 2005 VVSG) used to test and certify voting systems are all form and no substance and will be without substance for the next several years.