Web mirroring

This script makes a mirror of the remote web site and extracts the list of CGIs that are used by the remote host. It is suggested you allow a long-enough timeout value for this test routine and also adjust the setting on the number of pages to mirror.