How can I define my own URL rewrite rules?

When used by a website, URL rewrite rules need to be defined in Acunetix WVS to instruct the Crawler on how to recognize rewritten URLs, otherwise some URLs will be misinterpreted as directories — which will result in an incorrect scan.

In this FAQ we will detail one of the URL rewrite rules needed to scan the sample site http://testphp.vulnweb.com/Mod_Rewrite_Shop/, where /Mod_Rewrite_Shop/ uses URL rewriting technology.  Three rewrite rules are needed to scan this website; you can download the rewrite file to follow this exercise.

The below URL will request the web application to display product details for the product: ID 1.

http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-storage-d-link-dns-313-enclosure-1-x-sata/1

The above URL, is rewritten by the web server.  In fact, when the web server receives the request, it sends the following URL to the web application;

http://testphp.vulnweb.com/Mod_Rewrite_Shop/details.php?id=1

From the above, we can see that details.php has changed to /Details/ (looks like a directory), the product name is displayed as a directory, and id=1 has been changed to 1.

1. To set the above URL rewrite rule in Acunetix WVS, click on ‘Add Ruleset’ from the Configuration > Scan Settings > Crawling Options > URL rewrite node.
2. Click on the HostName to enter the website or web application URL.
3. Click on the ‘Plus’ sign button to create a new rule.
4. Since URL rewrite is only enabled for /Mod_Rewrite_Shop/ directory, select ‘Directory rule’ and specify the directory ‘/Mod_Rewrite_Shop/’.  If URL rewrite technology is enabled throughout the whole website, select ‘General Rule’.
5. In the ‘Regular Expression’ input field, specify a part of the URL including regular expressions (or a group of Regular expressions) which Acunetix WVS will use to recognize a rewritten URL.  E.g. Details/.*/(d+).  This means match everything after the Details/ directory, and after that matched string, match also a digit or more.
6. In the ‘Replace with’ input field, specify the URL Acunetix WVS should request instead of the rewritten URL. E.g. /Mod_Rewrite_Shop/details.php?id=$1.  The $1 will be replaced with the value retrieved from the first regular expression group specified in the ‘Regular Expression’ input field, in this case (d+).

Thanks to the above rule, when the Acunetix WVS crawler matches the following URL; /Mod_Rewrite_Shop/Details/network-storage-d-link-dns-313-enclosure-1-x-sata/1, instead it will request the following; /Mod_Rewrite_Shop/details.php?id=1.

Therefore in the ‘Regular expression’ input field, one must always specify a string including a group of regular expression that will match the dynamic URL.  In the ‘Replace with’ input field, one should specify the real URL Acunetix WVS should request, including $1, $2 etc which will be replaced with the content matched in the grouped regular expressions.

For more information about regular expressions, click here (http://en.wikipedia.org/wiki/Regular_expression)

View all the Acunetix FAQs here.

THE AUTHOR

Acunetix

Acunetix developers and tech agents regularly contribute to the blog. All the Acunetix developers come with years of experience in the web security sphere.