Limitations

SiteSucker is a relatively simple program and it has a number of limitations.

SiteSucker only examines CSS and HTML for URLs. When SiteSucker analyzes HTML, it looks for URLs in the style attribute and in the following tags:

  • <a>
  • <area>
  • <audio>
  • <base>
  • <body>
  • <embed>
  • <form>
  • <frame>
  • <iframe>
  • <img>
  • <input>
  • <link>
  • <meta>
  • <object>
  • <script>
  • <source>
  • <style>
  • <table>
  • <tbody>
  • <td>
  • <tfoot>
  • <th>
  • <thead>
  • <tr>
  • <track>
  • <video>

If a link is specified elsewhere in an HTML file, SiteSucker will not see it.

SiteSucker does not scan PDFs, Flash files (swf), QuickTime movie files (mov), or other media files for embedded links.

SiteSucker totally ignores JavaScript. Any URL specified within JavaScript will not be seen by SiteSucker and will not be downloaded. (SiteSucker will include a warning in the log file for any page that uses JavaScript if the Log Warnings option is on in the Log settings.) If JavaScript is embedded in HTML text and it is fairly simple, you might be able to extract URLs from the JavaScript by using the Patterns setting in the Webpage settings. See Settings in the SiteSucker manual for more information.

SiteSucker ignores forms for the most part, although it will download the action URL from a form tag and the src URL from an input tag, and you can use the web browser to submit forms. See Password-protected Sites in the SiteSucker manual for more information.

SiteSucker cannot download videos. See Frequently Asked Questions in the SiteSucker manual for more information.

By default, SiteSucker honors robots.txt exclusions, the Robots META tag, and the X-Robots-Tag HTTP header. As a result, any files or directories disallowed by robot exclusions will not be downloaded by SiteSucker. See Robot Exclusions in the SiteSucker manual for more information.