OWASP ZAP Tutorial - Part 2: Crawling

If you completed part one of this series, you have now successfully intercepted an HTTP request and response with OWASP ZAP. Don't you feel proud and empowered? I knew you would. But guess what? We're not done yet... not by a damn site. Now it's time to learn how to crawl a web application with ZAP. Now I know what you're thinking to your self. Your thinking, "Pablo, why do I need to crawl the site? Can't I just do an outright scan? Won't that crawl the site first anyway?" Well, the answer to your question is "yes," you can just scan the site, which usually starts with a crawl. But here is the deal, maybe you don't want to be noticed, especially if you are doing a test where stealth is important.

Have you ever worked on a blue team and had your email inbox explode with alerts because some jackhole decided to scan your company's website? If so, then you understand why running a full scan might not be desirable (you may also want to tune the tools that fired those alerts). Crawling a site is less noisy than a full scan and it also is less risky. Scans can knock down websites. Crawls are far less likely to do so. With a crawl, you can identify interesting components of a website and maybe even some directories that the designer didn't want you to know about (security through obscurity). So let's look at how to crawl with ZAP. We will build off of the work that was done in the last post.

Crawling

1. In ZAP, right-click on the URL of the application in the Target pane (left-side of the screen). In the options that appear, choose Attack > Spider (Spider's crawl, get it?).
   
2. In the Spider window, make sure that you are in the scope tab and that the scope is properly defined. For those of you that have read the last post, you know that holing up in an Equadorian consulate isn't a great career move. Leave the other options on this screen as is.
   
   
   
3. While still in the spider window, click on the Advanced tab. In this screen, we will want to make some changes to how deep the crawler goes and how many children it is allowed to crawl. Failure to do this can cause the crawl to last for a very long time, and it can also cause the application to run out of resources and crash. As a rule of thumb, I set the max depth to 3 (at least to start) and the number of children to 3 as well. You can fiddle with these settings to find what works best for you. Each situation is different, so these rules certainly aren't hard rules. Leave the other settings as they are.
   
   
   
4. Click the Start Scan button. While the crawl is running, expand the tree for your target and watch for any newly identified application locations or components.
   
   
   
5. Note that you can pause or stop the crawl at any time by using the crawl control buttons. You can also monitor the status of the scan by watching the progress bar and the percent complete. If the scan is taking far longer than you anticipated or wanted, stop the scan and modify the settings in step 3 (smaller numbers = quicker scans but fewer details).

Well, you have successfully crawled a site with OWASP ZAP. Pop the cork on the champagne, were don... wait, what? You still want to learn how to scan? Well crap. OK, check out my next post and we will hit that topic next.