Limit Crawling to Address and Subdirectories Only

This option is useful to limit the scope of the scan to part of the web application. By default, the option “Limit Crawling to address and sub-directories only” is enabled for new targets.

This option will limit the scope of the scan up to the last forward slash (/) in the target address.

Note that any target URL WITH a path but WITHOUT a trailing slash will cause the crawler to consider the final part of the path to be a FILE and not a FOLDER; the result is that the parent folder of that file will be the real target URL. For example:

the target URL http://www.example.com/folder1/subfolder1/ WITH the option "Limit Crawling to address and sub-directories only" will scan items beneath /folder1/subfolder1/ (WITHOUT the option, you will be scanning the full domain)
the target URL http://www.example.com/folder1/subfolder1 WITH the option "Limit Crawling to address and sub-directories only" will scan items beneath /folder1/

🔍 Limiting Scan Scope - Examples

Example 1: Scan the full domain

Set target URL to http://www.example.com (WITH or WITHOUT trailing forward slash); in this case, the option "Limit Crawling to address and sub-directories only" will have no effect on the scope of the scan

Example 2: Scan only part of the site or domain

Set target URL to http://www.example.com/part1/ (WITH trailing forward slash), and set the option "Limit Crawling to address and sub-directories only" to ENABLED to limit the scope of the scan to ONLY resources beneath the /part1/ folder
If you DISABLE the option "Limit Crawling to address and sub-directories only", then any path specified in the target URL will be ignored and you will scan the full domain

Therefore, if your target URL is set to http://www.example.com/task/subtask, you can disable the option "Limit Crawling to address and sub-directories only" to instruct the crawler to also look for resources in http://www.example.com/task/ and http://www.example.com.

« Back to the Acunetix Support Page