Consumer Reports hired a firm to create 5500 new variants of existing viruses to see just how good AV software was at protecting you from new viruses.

And of course the big "security" companies like McAfee were angry at this, since the obvious conclusion to the test would be that their software was hardly infalliable.

http://redtape.msnbc.com/2006/08/consumer_report.html

Consumer Reports recently conducted one of the most thorough tests ever of antivirus programs. But to really put these security programs through the paces, the magazine hired a firm to create 5,500 new viruses, using them to test the antivirus software products for their ability to detect unexpected threats.

Now antivirus companies are crying foul, saying the magazine ignored a long-standing principle not to invent new viruses.

"Creating new viruses for the purpose of testing and education is generally not considered a good idea,” wrote Igor Muttik of McAfee's antivirus lab on a public company blog this week. “Viruses can leak and cause real trouble." The entry helped touch off a firestorm.
I found this remark at Slashdot to be the most fitting and amusing:
Soon they'll propose testing car safety by doing test crashes! Or testing fire retardants by trying to set them on fire. Damn those Consumer Reports fools!
Anyhow, these are the results (found at Slashdot):
BitDefender Standard - 87
Zone Labs ZoneAlarm Antivirus - 85
Kaspersky Labs Anti-Virus Personal - 82
Norton Antivirus - 80
Norton Antivirus for Macintosh - 80
McAfee ViruScan - 77
Trend Micro PC-cillin Internet Security - 75
Alwil Avast! Antivirus - 68
F-Secure Anti-Virus - 66
Panda Software Titanium AV - 64
CA/eTrust EZ Antivirus - 57
PC Tools AntiVirus - 41

BitDefender at #1 is *very* impressive. I always knew they were good but not that good. eTrust has a pretty disgusting showing and I will be changing my opinion of them from here on out. No matter how efficient the software is I won't use it if it can't detect new viruses. I'm definitely going to be spending more time researching BitDefender's offerings. I already use their virus defs in my GFI implementations but it's probably time to run them for everything.