ScaleScaleScaleScale

Tips / Nginx


How To Block User Agents Using Nginx

Nginx offers many ways to block unwanted traffic. You can use allow/deny rules or set a password protected directory. You can also get rid of bad bots and avoid attacks from HTTP user agents with GET or POST requests. Following are some examples to protect websites from common illegitimate connections.

What is the user agent?

The user agent (UA) is simply a text string that helps servers and systems to identify the browser and operating system used by the client.

When you are browsing a website, your browser includes a user agent field in its HTTP header. The contents of the user agent string may vary from browser to browser.

Basically, the user agent identification is a way for the browser to say “This is Google Chrome running on Linux,” or “This is Internet Explorer running on Windows.”

Why do you need this user agent information?

User agents are a good way to block common attacks against software. Here, you will configure Nginx to block unwanted traffic using some very specific user agents, like curl or wget.

Edit the nginx.conf file:

nano -w /etc/nginx/nginx.conf

Inside of the HTTP{} section, add this:

if ($http_user_agent ~* (Wget) ) {
return 403;
}

In this example, the wget tool is blocked and returns a 403 Forbidden response.

How can you block multiple user agents?

if ($http_user_agent ~* (Windows 95|Windows 98|wget|curl|libwww-perl) ) {
return 403;
}

You can see here that the old operating system browsers (Windows 95 and 98) are blocked, along with wget, curl and libwww-perl (common ways to attack servers remotely). There are many other bots and tools to block; this is just a basic example.

If you want to know all of the existing user agent strings, check the user agent string website; it has all of the user agent strings that you will ever need.

Conclusion

You can block almost anything, if you know the user agent. This can be found on the web server logs or with any statistic software.

Popular search terms:

  • nginx block user agent
  • nginx vary user-agent deny
  • nginx user agent
  • $http_user_agent nginx
profile

Esteban Borges

Linux Geek, Webperf Addict, Nginx Fan. CTO @Infranetworking

  • Leo

    It’s really bad to use if like this. Use map instead and you will be happy

    • Troy Wolf

      I don’t know that using the $http_user_agent match is bad, but Leo’s comment spurred me to find this, which I’m using now: http://www.queryadmin.com/1214/block-user-agents-referrers-nginx-map/

      It also taught me about the special nginx 444 status code — it tells nginx not to send any response to the client but is logged. I like that better than a 403 response.