Scanning a large website very often takes a long time. Using the default settings, Acunetix Web Vulnerability Scanner will first try to identify all the pages using various crawling techniques, and will then proceed to scan the pages that have been identified. While Acunetix WVS does employ various techniques to minimise the scan time, you would sometimes need to fine-tune the scan settings to avoid needless scan delays. In this article, we will look at techniques that can be used to minimise the time taken to scan large and template-based websites.
Scanning template-based websites
Nowadays, websites are mostly dynamic; they are built using a few templates, which are used to create the site’s numerous pages. Data is often retrieved from a database, and inserted into the templates to create the pages. Often, a large site makes use of less than 50 template pages, and often, the vulnerabilities exist and need to be fixed in the template pages.
This guide will assist you in identifying one of many pages using the same template and scanning only that page, rather than scanning all the pages using the same template. This will result in a faster scan. You can perform such a scan either by starting a new scan, or by using crawling results which have been saved from a previous scan.
Starting a New Scan
- Initiate a new scan using the scan wizard by clicking on the “New Scan” button.
- In the “Scan Type” page, select ‘Scan single website’ and enter the Website’s URL. Click Next.
- On the “Options” page, you need to select the option “After Crawling let me choose the files to scan”.
- Proceed through the wizard and start the scan. Acunetix WVS will proceed with crawling all the website, and might identify some vulnerabilities in the process.
- Once the crawler has finished, you will be asked to select the files you want to scan. At this stage you need to select one page for each template. In our example website, all the product pages are created from a single template. Therefore only product1.htm needs to be selected. The other product pages can be unselected since they are based on the same template.
- After selecting one page for each template, you should also remember to scan all the pages which are not based on the any template.
Note: You need to ensure that you identify all the templates at this stage, otherwise you run the risk of not scanning sections of your website. Be cautious and consult your development team if necessary.
- Click Ok to start the scan on the selected pages.
- Optional: After the scan has finished, you might want to save the site structure identified during the crawl so as to avoid crawling the website the next time you want to scan the same site (check note 1). Right click on the Site structure and select “Save Crawler Data”.
Using Saved Crawling Results
If you have saved the Crawler Data from a previous scan, you can re-use this data to scan specific portions of your website. Proceed as if starting a new scan, and in the Scan Wizard, select ‘Scan using saved crawl results’.
- Avoid using the same crawled results over and over again if pages are added and removed from your website frequently.
- Make sure that you select at least one page per template used. If you do not do this, you might end up not scanning part of your site for vulnerabilities. Consult your developers in order to identify all the templates.
- Further reading: How to Scan a Shopping Cart with an Automated Security Scanner
Scanning large websites
Acunetix WVS also provides the ability to split a scan. In order to do so you first have to configure different Scan Settings Templates with different Directory and File Filter rules. For example, if your website has ‘sub1′ and ‘sub2′ as sub-directories, and each directory is to be scanned separately, exclude ‘sub1′ from the first instances’ Scan Settings Template and exclude ‘sub2′ in the other instances’ Scan Settings Template.
To exclude sub-directories, navigate to the Configuration > Scan Settings > Crawling Options > Directory and File Filters node. You can read more about excluding directories in the section ‘Directory and File Filters’ in the Acunetix WVS user manual. To learn more about how to create new Scan Settings Templates or to modify existing ones, refer to the section ‘Creating, modifying, or deleting Scan Settings templates’.
Once the different Scan Settings Templates have been created, launch a scan, selecting the Scan Settings Template which excludes one of the sub-directories. Once the first scan is up and running, launch another instance of Acunetix WVS and from the scan wizard select the other Scan Settings Template you previously created.
In the example above, we scanned a website with 2 sub-directories. If you have more sub-directories, you can create several Scan Settings Templates each containing different directory and file filters.