Blackhole Pro automatically detects, traps, and blocks bad bots. There are some cases, however, where you may want to always allow access for a particular bot or IP address. This may come in handy for testing purposes, proxy servers, caching plugins, and so forth. This quick tutorial explains how to whitelist with Blackhole Pro.
Note: The information in this post applies to both free and pro versions of Blackhole.
Whitelist bots by User Agent
Blackhole Pro blocks bad bots based on their reported IP address. To whitelist a bot by its reported user agent, visit the plugin setting “Whitelisted Bots”. There you can enter any strings that should never be blocked. That way you will never block important things like Google, Facebook, Twitter, etc. Any strings entered here will be matched against the reported user agent via regular expression. Separate multiple strings with commas.
By default, Blackhole Pro whitelists the following user agents:
a6-indexer, adsbot-google, ahrefsbot, aolbuild, apis-google, baidu, bingbot, bingpreview, butterfly, cloudflare, duckduckgo, embedly, facebookexternalhit, facebot, googlebot, ia_archiver, linkedinbot, mediapartners-google, msnbot, netcraftsurvey, outbrain, pinterest, quora, rogerbot, showyoubot, slackbot, slurp, sogou, teoma, tweetmemebot, twitterbot, uptimerobot, urlresolver, vkshare, w3c_validator, wordpress, wp rocket, yandex
Note: The default user-agent strings added for this setting ensures that the main search engines and other popular services never are blocked, so don’t make any changes unless you are 100% sure that you know what you are doing.
Whitelist bots by IP Address
To whitelist specific bots by their reported IP address, visit the setting “Whitelisted IPs”, and enter the IPs that you would like to always allow access. Any IPs entered in the Whitelisted IPs option will be matched against the reported IP address via regular expression. So you can do any of the following:
- Block an individual IP address, like
- Block a range of sequential IP addresses, like
- Block a range of IP addresses in CIDR notation, like
Separate multiple IP/strings with commas. Note that the plugin automatically adds your server IP address and local IP address, if it is available. If you are using anything like caching, load-balancing, or reverse proxy, make sure to add their respective IPs to the whitelist.
By default, Blackhole Pro whitelists the following IP addresses:
- The admin’s server IP address
- The admin’s local IP address
For more information, check out the article on whitelisting plugins.
Redirecting whitelisted bots
Along with the other whitelist settings, there also is an option to specify a custom URL to which all whitelisted bots will be redirected. Normally whitelisted bots can access and surf around your site just like any other visitor. This setting enables you to redirect all bots in your user-agent or IP-address whitelists. Important: recommended to leave this setting blank unless you know what you are doing.