About WorkhorseSiteChecker
If you found WorkhorseSiteChecker/1.0 in your server logs, firewall or analytics, it’s an automated monitoring tool run by Workhorse. It checks the websites we build, host or maintain for our clients, to make sure search engines can still find and index them.
It is not a search engine crawler, a scraper or a security scanner. It doesn’t follow links, submit forms, log in, or collect personal information.
What it checks
Once a night, for each site we look after, it checks:
- robots.txt: that it loads, doesn’t accidentally block all search engines, and hasn’t changed unexpectedly.
- XML sitemap: that it exists, is valid, and stays within search engine size limits.
- Homepage indexing: that the homepage isn’t set to “noindex” by mistake (or, for staging sites, that it still is).
- And more
If something looks wrong, our team is notified and follows up with the site owner.
What you’ll see in your logs
- User agent:
WorkhorseSiteChecker/1.0 (+https://www.builtbyworkhorse.com/workhorsesitechecker-bot/) - When: once a night, starting around 2:00 a.m. US Central Time
- Which pages:
/robots.txt, your XML sitemap file(s), and your homepage. - How many requests: usually a handful per site. A site with a large sitemap index can see a few dozen, because we check up to 25 of the sitemaps listed in it.
- From where: a dedicated Workhorse IP address. If your firewall already allows Workhorse, nothing needs to change.
What we keep
We store the results of each check: whether it passed, HTTP status codes, and a copy of your robots.txt so we can spot changes. Results are deleted after 90 days.
Questions, or think we shouldn’t be checking your site?
If you’d like to confirm our IP address, have a question about the requests, or think your site is being checked by mistake, please get in touch. We’re happy to help, and we’ll remove any site that shouldn’t be monitored.