Manual Crawling using the HTTP Sniffer
Screenshot – The HTTP Sniffer
The HTTP Sniffer is a proxy server that enables you to capture and edit HTTP requests and responses exchanged between a web client (browser or other http application) and a web server. The HTTP Sniffer can be used to manually crawl sections of a website that cannot be crawled automatically by Acunetix Web Vulnerability Scanner. The captured data can then be loaded into the Crawler and used to launch a scan.
To capture live traffic, your web browser must be configured to proxy through the HTTP Sniffer and then export the logs to the Site Crawler. You can read more about this process from the following URL; http://www.acunetix.com/blog/docs/manual-crawling-http-sniffer/
The HTTP Sniffer can also be used to analyze HTTP traffic and to trap particular POST or GET requests that can be changed on-the-fly (manually or automatically) to emulate a ‘man in the middle’ attack.
Configuring Your Browser
To start capturing traffic, you must first configure your browser to use the Acunetix HTTP Sniffer as proxy server:
- From the Tools drop down menu select Internet Options
- Select Lan Settings from the Connections tab
- In the Connection section click on Settings and tick Manual proxy configuration
- Set HTTP Proxy to 127.0.0.1 and Port to 8080
- If you also need to capture SSL traffic, configure the SSL Proxy to 127.0.0.1 and Port to 8080
- Click OK to save all options and close all configuration windows.
Screenshot- Browser Proxy Server Settings
- From the Tools drop down menu click Internet Options
- Click on the Connections tab and then click LAN Settings button
- Tick the option Use a proxy server for your LAN
- In the Address input field, enter 127.0.0.1 and enter 8080 in the Port input field.
- If you also need to capture SSL traffic, click on the Advanced button and in the Secure Input field enter 127.0.0.0 as proxy address and 8080 as port number.
- Click on OK to save all settings and close all configuration windows.
Google Chrome uses Internet Explorer’s proxy server settings. Therefore to use Google Chrome, follow the procedure above and configure Internet Explorer.
Note: By default, the HTTP Sniffer proxy server listens on localhost (127.0.0.1) and port 8080. This limits the capturing of traffic to web clients running on the same machine.
The HTTP Sniffer options in Acunetix Web Vulnerability Scanner can be accessed from the Configuration > Application Settings > HTTP Sniffer node.
You can set the HTTP Sniffer to listen on all interfaces, so web client applications running on other machines can proxy traffic through the HTTP Sniffer for analysis. The HTTP Sniffer port can also be configured.
Capturing HTTP traffic
To capture HTTP traffic:
- Go to the Tools > HTTP sniffer node
- Click on the Start button to enable the HTTP Sniffer.
- From your browser, browse the website that you are interested in. All HTTP requests and responses will be listed in the main window.
- Click on a request or response to view the complete details. All the requests/responses will be displayed in the lower window pane.
- Click Stop when browsing is complete. Keep in mind that when the HTTP Sniffer is stopped, the web browser will lose its connection to the target URL.
- You can then save the browsing logs, and load them into the crawler. Click Save to store the logs.
Go to Tools > Site Crawler and click on the Build structure from HTTP sniffer log button. Browse to the sniffer log you just saved.
The crawler will build the structure. You can then right click on the site and scan it from within the Crawler, or save the crawl results and load them into the web scanner.
For more information about using the HTTP sniffer:
HTTP Sniffer Trap Filters
Through an HTTP Proxy trap filter, you can configure the HTTP Sniffer to intercept an HTTP request for it to be manipulated in real-time before it arrives to the server. You can do the same for HTTP responses.
Screenshot - HTTP Sniffer Edit Trap window
Creating a HTTP Sniffer Trap Filter
- In the HTTP Sniffer toolbar, click on the Edit traps button to launch the HTTP Traps window.
- Select a trap rule template, e.g. trap requests, and trap ASP or PHP requests. This will load up a preconfigured trap which you can edit.
- Alternatively you can create a new trap by first entering a description for the rule.
- Specify the rule type from the following 4 options:
- Include - Configure which HTTP requests and responses should be trapped.
- Exclude - Configure which HTTP requests and responses should excluded.
- Replace or change rules - Configure which HTTP requests should be automatically changed based on the given expression.
- Logging rules - Configure which HTTP requests or responses should be logged in the Activity window.
- The type of traffic that will be captured by the trap must also be configured. Traps can be set to capture all traffic, HTTP requests only, request headers only, etc.
- In the Regular expression option, enter a regular expression that matches the data you would like to trap.
- Once the new trap is ready, click on the ‘Add…’ button to save the new trap. This will add the trap and automatically enable it. You can enable/disable traps by clicking on the tick box in front of the trap rule.
- Click the ‘OK’ button to return to the HTTP Sniffer dialog and click on the ‘Enable traps’ button to activate the traps in the HTTP Sniffer.
The Trap Form
Screenshot - HTTP Sniffer Trap form
When an HTTP request or a response is trapped by the HTTP Sniffer, the HTTP Trap window will automatically appear to allow you to edit the captured data. Similarly to the HTTP Editor, the Trap Form editor allows you to edit headers, cookies, queries, and post variables. Click OK to allow the HTTP request or response through.
Editing a HTTP Request without a Trap
If you want to edit a HTTP request without setting up an HTTP trap, right click on a request or a response and select Edit with the HTTP Editor. Click Start in the HTTP Editor to send the HTTP request to the server.