New malware tests find poor detection rates
Some products struggle with new malware, finds VB.
By John Dunn | Techworld | Published: 16:59, 04 August 2009
Many Windows Vista anti-virus programs struggle to detect new and unusual malware, Virus Bulletin's state-of-the-art Reactive and Proactive (RAP) tests have found.
The latest figures report an average detection rate for the period between February 2009, when the tests were first introduced, to the end of July.
The resulting ‘RAP Quadrant' shows that several well-known products fall in to the lower left hand quarter of the graph, including PC Tools' Anti-Virus, Fortinet's Forticlient, and CA's Internet Security Suite, all of which achieved detection levels below 50 percent on both axes when configured in their default mode.
Even the best performers, including those from Kaspersky Lab, BitDefender, Sophos, Check Point and Microsoft, showed mixed performance across some aspects of the RAP test regime.
The February to August quadrant can be viewed on the Virus Bulletin website.
Virus Bulletin is best known for its VB100 Certification, which rates software products against the independent but limited WildList collection of malware samples. The RAP is an attempt to pioneer more demanding tests that measure how products react to new malware sets in each of the three weeks prior to a pre-defined test deadline (the Reactive dimension) and in the week immediately following it (The Proactive).
Generally speaking, the older a sample, the more easily it will be detected due to vendors obtaining their own copy and using it to update a product's signature database. This shows the effectiveness of a vendor's ‘rapid response'. The proactive samples, by contrast, will be far less likely to be have been detected, and therefore this part of the test measures the underlying heuristic capabilities of a product to spot a new or unknown threat without looking it up.
"We saw some particularly poor detection of emerging threats and the products in question have a lot of work to do if they are to provide acceptable protection for their customers," said VB test director John Hawes, who also praised the performance of several other products in the same tests. "All products should be aiming for this position and we hope to see an improvement in RAP scores in the future."
At the moment, the RAP scores had no bearing on the established VB100 Certification and were only indications of performance, he said.
What constitutes a good result is simply a consistently high score relative to other products. The assumption is that no product can possibly detect 100 percent of new threats given their rapid mutation, huge volume, and variety of attack methods, including exploiting flaws in specific software products. As ever, anti-virus is not a barrier against all possible attacks but a percentages game.
