IT Jobs
TippingPoint IPS struggles in new security tests
Vulnerability detection falls below 40 percent.
By John E. Dunn | Techworld
Published: 16:04 GMT, 09 September 09
Testing outfit NSS Labs says it has seen a sharp drop in the security performance of one of the leading intrusion prevention products (IPS) products on the market, the TippingPoint 10.
Running 622 known exploits gathered from a range of Windows and Linux applications and operating systems going back several years in some cases, the TippingPoint 10 in default configuration blocked only 247 (39.7 percent) of them. In terms of the most serious ‘attacker-initiated' exploits, the box performed only slightly better, detecting 205 (49.8 percent).
This iffy security performance contrasts with similar IPS products the company recently tested from McAfee and IBM, which both scored "in the 95 percent range" when pitted against the same family of exploits, said NSS Labs' Rick Moy.
Despite the having a price tag in the region of $4,000 (approx £2,400), the TippingPoint 10 lies at the entry-level point of the company's intrusion prevention appliance range, and is designed to serve smaller or remote offices. According to Moy, the core security technology would be the same across all TippingPoint products, however, and he also doubted that adjusting the default configuration would have made any difference.
"Five years ago it was the leader of the pack and it got a Gold award," said Moy. "Something has clearly slipped in the mean time, and competitors have surpassed them. This should give IT security buyers a good think about the risks of buying based on brand name, marketing and historic performance data."
Throughput was one bright spot, with the TippingPoint 10 easily achieving its stated packet inspection performance level of 20Mbit/s, even under heavy test loadings.
"We found the TippingPoint 10 to be stable and reliable, handling our extensive reliability tests without failure. Management is easy to use and suitable for small office environments, where simplicity is a must," the post-test notes said.
"In summary, with a protection rating of 39.7 percent, the TippingPoint 10 is unlikely to stop the majority of attacks, but it does offer stable performance for a small office."
So why has the TippingPoint declined on the NSS Labs' tests? Moy was unsure, but suggested that the company had possibly not been investing the same resources in the time-consuming and complex business of coding signatures.
TippingPoint Technologies was bought by 3Com in 2004, but has ploughed a fairly arms-length furrow as the premier IPS vendor ever since. Probably its most controversial moment was the inauguration in July 2005 of its Zero Day Initiative (ZDI), under which independent researchers were paid by the company for reporting security vulnerabilities. These are then added to its IPS database.
It seems unlikely that the ZDI program would have had any bearing on the NSS results given that reported vulnerabilities are unlikely to make up more than a small subset of the company's signature updates.
TippingPoint was unable to offer comment at the time of going to press. The full NSS Group report on the TippingPoint 10 can be downloaded from the company's website.
.gif)
Add your commentComments
Mumbo | Published: 18:32 GMT, 07 October 2009
Actually, ISS Proventia IPS's "demonstrated an average security effectiveness of 98.6% over a 3 month period" during an identical test by NSS 5 months ago. Read here: http://nsslabs.com/2008/ibm-iss-gx6116-intrusion-prevention-system-achieves-nss-labs-gold-award-and-certification.html
Tyler Durden | Published: 21:03 GMT, 30 September 2009
How about a test showing how each vendor fared *after* the rules were applied? I don't know about TP, but I do know that Proventias used to be god awful without being tuned.
mike.politik | Published: 15:36 GMT, 21 September 2009
TippingPoint sounds like a bunch of whiners. NSS PWND their "IPS" and now they are upset.....welcome to security TP!
Bob Walder | Published: 09:39 GMT, 19 September 2009
Joel-Thank you for your (humorous) note of sanity! That is exactly how some vendors would like us to test! The fact remains, we test ALL vendors in an identical manner (including encouraging engineer participation and appropriate tuning for the test traffic/prototcols/apps in use) and some vendors are able to achieve in excess of 90% in the security effectiveness tests - Tippingpoint didn't.
Joel Shoemaker | Published: 17:49 GMT, 18 September 2009
My network is unique as the attackers always tell me when they are going to conduct their port scans. After this they tell me exactly the exploits they use, if they fragment the packets, send the attack on a different port, obscure it, the time they attack and the IPs they attack. I turn on the correct signatures and tune it just right to block them. Nothing bad ever happens, no false positives and no false negatives.
Erhan Dolak | Published: 07:58 GMT, 18 September 2009
Bob, I think you need at least say how many signatures are enabled during testing. Recommended means safe and no false+ as far as I know. Voip was just an example as you may already understand. Anyway, I am sure in near future you will repeat the test with correct methodoly.
Bob Walder | Published: 09:08 GMT, 17 September 2009
I will clarify one point: ALL vendors are provided with a very precise indication of the traffic (though not the exploits) which will be on the test network. All vendors are then able to produce a default/recommended policy for that traffic. NSS engineers also validate this and will enable batches of sigs if missing. If we don't run Apache traffic, then we won't run Apache exploits. We are not looking to "trick" any vendor into failing-what would be the point? Tippingpoint was tested in this way
Bob Walder | Published: 08:11 GMT, 17 September 2009
Erhan-you need to actually READ the methodology. You have no idea how NSS tested until you do and are just speculating (i.e. did they actually USE VOIP traffic?-if not your point is irrelevant). Re "congratulating" Tippingpoint-the salient point is that other vendors who were tested under identical conditions fared much better. If 40% coverage is good enough for you then I wish the users of your network the best of luck.
Joel Snyder | Published: 07:32 GMT, 16 September 2009
Some comments have 'scrolled off' Techworld, but to NSS' defense: I run a competitive lab, yet have been a big fan of their testing program for many years. Compared to nearly ALL of the other public testing labs, Bob's is THE most thorough and honest I've ever seen. I haven't read this report yet, but in the past NSS has set the standard for what a solid product test should look like. (Gartner, on the other hand, does not actually test products, so they cannot be compared to a test lab.)
Erhan Dolak | Published: 10:53 GMT, 13 September 2009
Re:Bob Then you definetely know that you need to configure these products for your environment before sending production traffic. I know that nearly all voip signatures disabled in tippingpoint by default, if you have voip traffic,you need to enable them,same rule is valid for IIS and apache signatures. With default settings we should congratulate tippinpoint of catching %40 of your attempts. By publishing this kind of report you just loose community trust on your tests and methodology.