FeedOtter Crawler

Feedotterbot is the generic name for FeedOtter's web crawler that simulates a user on desktop.

If you are using FeedOtter your website will probably be crawled by Feedotterbot. You can identify the Feedotterbot by looking at the user agent string in the request.

How Feedotterbot accesses your site

For most sites, Feedotterbot shouldn't access your site more than five to ten times a day on average. However, due to delays it's possible that the rate will appear to be slightly higher over short periods.

Our goal is to crawl as few pages from your site as we can on each visit.

Blocking Feedotterbot from visiting your site

If you want to prevent Feedotterbot from crawling content on your site, you have a number of options but this may limit various features of the FeedOtter product including; image detection, webpage to email bookmarking and auto-email generation features.

Verifying Feedotterbot

Before you decide to block Feedotterbot, the best way to verify that a request actually comes from Feedotterbot is to use a reverse DNS lookup on the source IP of the request.

Feedotterbot and all respectable search engine bots will respect the directives in robots.txt, but some nogoodniks and spammers do not.