FAQ: Is it possible to crawl a site manually?

It is possible to manually crawl your website with Acunetix WVS using a web browser. Using the resultant — and manually crawled — links, it is then possible to build a website structure that will be targeted during the security scan.  This is useful for scanning specific web applications that cannot be automatically crawled due to some strange coding ambiguities. The following procedure offers a reliable workaround:

1. Configure the web browser

Browser Proxy Configuration
Configure your web browser of choice to proxy all the traffic through the Acunetix WVS HTTP Sniffer tool, as shown in the above screen shot.  Presuming that the web browser is running on the same machine where Acunetix WVS is installed, set the proxy server IP to 127.0.0.1 and the proxy server port to 8080.

2. Start the HTTP Sniffer and browse the website using the previously configured web browser.

Acunetix WVS HTTP Sniffer capturing traffic

 

3. Once ready, stop the HTTP sniffer. Save captured data by selecting ‘Save Logs’ from the Actions drop down menu.

4. Import Logs to Crawler

Acunetix WVS website crawler with import from HTTP proxy
In the Site Crawler node, click the ‘Build Structure from HTTP Sniffer log’ button (highlighted in the above screen shot) to import the captured data into the Site Crawler.

It is also possible to import HTTP Sniffer logs to an already existing scan, or import multiple HTTP Sniffer logs into the same crawl. To do so, simply tick the option “Merge the log9s0 with the currently opened crawl results in the HTTP Sniffer Log import window as highlighted below.

Import HTTP Sniffer results to Website Crawler

5. Save the crawler import results by selecting ‘Save Results’ from the Actions drop down menu.

6. Launch the Scan

Acunetix WVS Scan Wizard to launch scan from saved crawl
Click on the New Scan button to launch the scan wizard.  In the first step of the Scan Wizard select the option ‘Scan using saved crawling results’ as highlighted in the above screen shot.  Proceed with completing the scan wizard to launch the automated scan against the manually browsed website.

Note:

Only the links you’ve manually crawled will be automatically scanned.  Other pages in the website, even those linked from manually crawled pages will not be crawled or scanned.


View all the Acunetix FAQs here.

  • Hello,

    I have a sniffer log file where it has different hosts. With ‘Merge’ option in Build structure, Is it possible to merge the sniffer logs with different hosts ?

    I also have a scenario, where, when i manually crawl the website it records many number of hosts (within the same sniffer log). Leaving behind some unwanted hosts, i would want to merge the requests from 2 different hosts.
    Acunetix version: 9.0

    Kindly advice me if i can achieve this or not ?

  • Leave a Reply

    Your email address will not be published.


    *