Instruct Alkaline not to index a particular set of urls. When specified, all urls matching the UrlSkip criteria, will still be spidered but not indexed. This means their content and meta tags will not be indexed, these pages will not appear in search results, but the links that they contain will still be followed. Unlike the UrlExclude directive, this option is processed after a page is retrieved.
With the global RegExp option enabled or a RegExp prefix, the parameter must be a list of regular expressions. This option has the RegExp UrlSkip extension since version 1.6.
This option was added in version 1.3 (09-Jul-2000).
To avoid indexing http://www.foo.com/bar/ files, but follow all links from these pages:
UrlList=http://www.foo.com/
UrlSkip=http://www.foo.com/bar/
|