RankHintBot
RankHintBot is the crawler behind RankHint. It crawls a website only after the site's owner (or someone with their permission) turns on a site audit in RankHint, to check it for technical SEO issues such as broken links, redirects and missing titles.
How to recognize it
- User agent:
Mozilla/5.0 (compatible; RankHintBot/1.0; +https://rankhint.com/bot)(always containsRankHintBot) - Runs on Cloudflare's network. There is no fixed list of IP addresses, so allow it by user agent, not IP.
How it behaves
- Obeys robots.txt, including
Crawl-delayand rules forUser-agent: RankHintBot. - Crawls one site at a time at up to 4 requests per second, with only a few requests in flight. The site owner can choose a gentler speed, and it slows to as little as one request every 10 seconds if the site asks.
- Only fetches HTML pages on the site being audited; skips images, files, admin and feed URLs.
- When refused, it stops immediately and waits. On "too many requests" it slows down and retries for up to about 20 minutes; on a firewall or bot-protection block it retries once or twice over a few minutes, then stops and tells the site owner what blocked it.
Allowing it
If the site owner asked you to let the audit in, allow requests whose user agent contains RankHintBot in whatever protects the site. Steps for common setups:
Cloudflare
- Security → Bots: if Bot Fight Mode is on, turn it off (on the free plan no rule can skip it). On Pro and above with Super Bot Fight Mode, set "Definitely automated" to Allow, or skip it with the rule below.
- Security → WAF → Custom rules → Create rule. When incoming requests match: User Agent contains "RankHintBot". Action: Skip, and tick all remaining custom rules, rate limiting rules, managed rules and Super Bot Fight Mode. Save it at the top of the list.
- Security → Events (filter by user agent "RankHintBot") can show which rule blocked it, though requests from other Cloudflare-hosted services don't always appear there.
SiteGround (Anti-Bot AI)
- SiteGround's Anti-Bot AI can't be allowlisted in Site Tools. Open a support chat and ask them to allow the user agent "RankHintBot" for your site (share the report with the times of the blocked requests).
- Alternatively, put the site behind Cloudflare (free plan works) and allow "RankHintBot" there; SiteGround then sees Cloudflare's addresses instead.
SiteGround firewall (behind Cloudflare)
- SiteGround's server firewall (its WAF / ModSecurity rules, not the Anti-Bot captcha) is refusing the crawler, and Cloudflare passes the 403 along. It can't be changed in Site Tools.
- Open a SiteGround support ticket with the times and request IDs from this report. Ask which firewall rule blocks the user agent "RankHintBot" and to allow it for your site. Requests from Cloudflare-hosted services like RankHint usually reach SiteGround from 2a06:98c0:3600::103, so they can also check that address.
Wordfence (WordPress)
- Wordfence → All Options → Rate Limiting → "Allowlisted user agents": add RankHintBot. Save.
- If you use Wordfence's country blocking, our crawler runs from data centers worldwide: allow RankHintBot or relax it during the audit.
Sucuri Firewall
- Sucuri dashboard → Firewall → Access Control → Allowlist: allow the user agent RankHintBot.
Host or security plugin (behind Cloudflare)
- The site is behind Cloudflare, but the refusal wasn't Cloudflare's own block page, so it most likely came from the web host or a security plugin and Cloudflare passed it along.
- Allow the user agent "RankHintBot" in the security plugin, or ask the host's support to allow it. The Ray IDs and times in the report help them find the requests in their logs.
Any firewall, host or plugin
- Allow requests whose user agent contains "RankHintBot" in whatever protects the site: host firewall, security plugin, CDN or server rules (e.g. nginx/Apache user-agent blocks).
- Our crawler runs on Cloudflare's network, which has no fixed IP list, so allow it by user agent rather than IP. Hosts that block data-center traffic may need this even if the site works in a browser.
Checking that it works
In RankHint, open the site's Site audit tab and click Test access. It sends three requests from the crawler's own network, a couple of seconds apart, and shows whether each one got through and, if not, what answered (status, firewall, request ID).
A request from your own computer with the same user agent, e.g. curl -A "Mozilla/5.0 (compatible; RankHintBot/1.0; +https://rankhint.com/bot)" https://example.com/, checks user-agent rules only. It can't reproduce blocks based on IP reputation or network, which is why the in-app test exists.
Blocking it
Add this to robots.txt:
User-agent: RankHintBot Disallow: /
Questions: travis@travisw.me