Suggested Settings is a feature that recommends changing certain settings when specific conditions are detected while downloading a site. If you apply these changes and download the site again, SiteSucker may be able to download more files and get better results.
You should be able to download most sites using the factory default settings. If that doesn’t produce acceptable results, choose Suggested Settings in the Settings menu. A window is displayed with all appropriate options turned on. Turn off any options you do not want, click the Apply button, and download the site again. The Suggested Settings menu item is only enabled if a change is recommended and only those changes that are recommended are enabled in the window.
The Suggested Settings dialog provides the following options:
Turn on ‘Ignore Robot Exclusions’
Check this box to turn on the Ignore Robot Exclusions option in the General settings. Robot exclusions allow website administrators to define what parts of a site are off-limits to robots like SiteSucker. By default, SiteSucker honors these exclusions, but this behavior can be overridden by turning on the Ignore Robot Exclusions option, allowing excluded files to be downloaded. (See Robot Exclusions for details.)
Turn on ‘Ignore rel="nofollow"’
Check this box to turn on the Ignore rel="nofollow" option in the General settings. By default, if the rel attribute equals “nofollow” in an HTML tag, then SiteSucker will not follow that link. However, if the Ignore rel="nofollow" setting is on, SiteSucker will download links that have the rel="nofollow" attribute.
Turn on ‘Include Supporting Files’
Set ‘Identity’ to ‘Web View’
Check this box to set the Identity setting to Web View in the Request settings. You may only get errors when downloading files from some sites, because the sites are particular about which browsers they will allow. To get around this problem, the Identity option can be used to “fool” a site into thinking that SiteSucker is an approved browser.
Turn on ‘Download Using Web Views’
Turn on ‘text/html’
Check this box to turn on the text/html option under the HTML Types tab in the File Type settings. If the text/html option is off, then webpages will not be scanned and nothing will be downloaded.