.htaccess Block ‘Bad’ Bot in Apache

Issue: Excessive (or abusive, questionable, etc.) bots are accessing a website. In some cases the bots are degrading website performance. Some bots are using multiple IP addresses, however, they are using the same user_agent.

Solution: Use .htaccess directives to block access (error 403) to those bots, that identify their user agent. Example below:

# BEGIN block bots
BrowserMatchNoCase Ahrefs bad_bot
BrowserMatchNoCase aiHitBot bad_bot
BrowserMatchNoCase Apache-HttpClient bad_bot
BrowserMatchNoCase AppleBot bad_bot
BrowserMatchNoCase aspiegelbot bad_bot
BrowserMatchNoCase careerbot bad_bot
BrowserMatchNoCase CCBot bad_bot
BrowserMatchNoCase checker bad_bot
BrowserMatchNoCase Clickagy bad_bot
BrowserMatchNoCase coccocbot bad_bot
BrowserMatchNoCase crawl bad_bot
BrowserMatchNoCase Dataprovider bad_bot
BrowserMatchNoCase Datanyze bad_bot
BrowserMatchNoCase Daum bad_bot
BrowserMatchNoCase digg bad_bot
BrowserMatchNoCase discovery bad_bot
BrowserMatchNoCase dotbot bad_bot
BrowserMatchNoCase downloadbot bad_bot
BrowserMatchNoCase EveryoneSocialBot bad_bot
BrowserMatchNoCase ExoRank bad_bot
BrowserMatchNoCase facebot bad_bot
BrowserMatchNoCase FlipboardProxy bad_bot
BrowserMatchNoCase Go-http-client bad_bot
BrowserMatchNoCase Google-HTTP-Java-Client bad_bot
BrowserMatchNoCase GuzzleHttp bad_bot
BrowserMatchNoCase HeadlessChrome bad_bot
BrowserMatchNoCase http_request2 bad_bot
BrowserMatchNoCase InternetSeer bad_bot
BrowserMatchNoCase jack bad_bot
BrowserMatchNoCase JS-Kit bad_bot
BrowserMatchNoCase kraken bad_bot
BrowserMatchNoCase liebaofast bad_bot
BrowserMatchNoCase ltx71 bad_bot
BrowserMatchNoCase MauiBot bad_bot
BrowserMatchNoCase MegaIndex bad_bot
BrowserMatchNoCase MetaCommentBot bad_bot
BrowserMatchNoCase metadataparser bad_bot
BrowserMatchNoCase MetaURI bad_bot
BrowserMatchNoCase MJ12bot bad_bot
BrowserMatchNoCase nerdybot bad_bot
BrowserMatchNoCase NetLyzer bad_bot
BrowserMatchNoCase Nimbostratus-Bot bad_bot
BrowserMatchNoCase NING bad_bot
BrowserMatchNoCase okhttp bad_bot
BrowserMatchNoCase PaperLiBot bad_bot
BrowserMatchNoCase Prlog bad_bot
BrowserMatchNoCase Qwantify bad_bot
BrowserMatchNoCase uMBot bad_bot
BrowserMatchNoCase robot bad_bot
BrowserMatchNoCase RogerBot bad_bot
BrowserMatchNoCase SAMSUNG-SM-G920A bad_bot
BrowserMatchNoCase scanner bad_bot
BrowserMatchNoCase screaming bad_bot
BrowserMatchNoCase SEMrushBot bad_bot
BrowserMatchNoCase serpstatbot bad_bot
BrowserMatchNoCase sistrix bad_bot
BrowserMatchNoCase sogou bad_bot
BrowserMatchNoCase spbot bad_bot
BrowserMatchNoCase spider bad_bot
BrowserMatchNoCase ToutiaoSpider bad_bot
BrowserMatchNoCase tweetedtimes bad_bot
BrowserMatchNoCase ubermetrics bad_bot
BrowserMatchNoCase weborama bad_bot
BrowserMatchNoCase WebWasher bad_bot
BrowserMatchNoCase wget bad_bot
BrowserMatchNoCase woobot bad_bot
BrowserMatchNoCase yandex bad_bot
BrowserMatchNoCase yisouspider bad_bot
BrowserMatchNoCase zmeu bad_bot
BrowserMatchNoCase ZoominfoBot bad_bot
Order Deny,Allow
Deny from env=bad_bot
# END block bots

Leave a comment

BlogLogistics