Tuesday, February 16, 2010

Web Application Scanners - Open source vs. commercial scanners

There is this ongoing debate within the web application security community relative to selection of web application scanners. With some good commercial scanners in the market and promising open source scanners, it is quite confusing for many developers and IT professionals to select one. If price alone was the consideration, it would be an easy choice, but what makes it harder is which product does the intended job. I have seen that in selecting one product, there is a tendency to feel like you are missing out on what the other product offers. It is like buying a car: regardless of how hard you negotiate to get the best deal, you always feel that the salesman got away with the better deal.

Let me begin with my thoughts on open source scanners. Two names that are frequently mentioned are WebScarab (OWASP) and Burp Suite (PortSwigger). Those who have worked with these products for some time firmly stand by them. To learn more about the products, I downloaded both them; of course, the free offer was not too bad, either. After few attempts, I quickly learnt that there is a steep learning curve associated with both of these products. Lack of a good set of documentation made it rather challenging to learn their usage and appreciate their effectiveness. As is the case with all open source products, support is provided by community in open forum. That may be of concern if your inquiry involves disclosing confidential information. Furthermore, if your questions are time sensitive, you can expect to be disappointed waiting for an answer.

Turning to commercial products, the one I have used extensively is Acunetix Web Vulnerability Scanner (http://www.ebusinessmantra.com/buywebsecurityscanner.aspx). Right "out of the box", the product is easy to install, comes with detailed documentation, and support from the vendor. That's a big check plus for commercial products. Literally within few minutes after download, I had a list of vulnerabilities in the application I tested. I certainly cannot say that for the open source products. What value do you put to your time? Do you have time to learn use of open source products by yourself, especially in face of deadlines? Can you afford to remain vulnerable while you figure out how to use them? Can you afford to remain vulnerable while you wait for someone to answer your question? These are the questions you have to ask yourself.

Also of note: all products, open source and commercial, come with their faults in that they all report false positives and false negatives. So, the results of each application, open source or commercial, have to be evaluated for its correctness. Some may ask, why spend money if the outcome is not assuring? As we saw earlier, it is the time you save to get to the outcome, not just the outcome. The other advantage of a commercial product is that one product contains features that tests for various parameters whereas open source products are typically test for specific vulnerabilities. With open source you have to have multiple products to tests for a variety of parameters, for example, having a product for port scanning and another for scanning, file checks, directory checks, perhaps Google Hacking Database (GHDB), and so on. Now for the learning curve associated with each product, I think I would prefer commercial product and that should be true for all who are serious about vulnerabilities in their application.

Lastly, updates and bug fixes - how often are updates, fixes, and patches issued for open source products compared to commercial products? My first hand experience with Acunetix is that updates are issued at least once every two weeks. Can we say the same for WebScarab and Burp Suite?

So factors to consider when comparing open source and commercial scanner are:
• time to learn effective use of the product,
• features,
• customer support,
• product maintenance
• and one factor to ponder: can you wait to remain vulnerable while considering the other factors?

Now don't get me wrong that open source scanners don't have its place, so next time, I will talk about when and where an open source products can be effective.