How to Log Request Headers with Nginx and NJS

Sometimes, when bad bots attack your site, it is convenient to look at the request headers to see if there is a signature that allows you to block them all easily.

For example, sometimes the bots send an empty Cookie: header, or append :443 to the hostname in the Host header.

Unfortunately, there is no variable available that you can use in the log_format directive to log all request headers.

However, if you have the NJS module, you can easily overcome this.

First, you need to ensure that the NJS module is loaded. Check for a line similar to this one in your nginx configuration file:

load_module "/usr/lib/nginx/modules/ngx_http_js_module.so";

The path to the module may differ; please make sure to use the actual path.

After that, you will need to create a JavaScript file (say, utils.js) and put it into the directory with nginx.conf:

function stringify_headers(r) {
    return JSON.stringify(r.headersIn);
}

export default { stringify_headers: stringify_headers };

Then, you will need to import the module. Place this directive into the http block of your nginx.conf:

js_import utils.js;

Next, you need to create a variable that will hold the headers and use it in your custom log format. These lines should also go into the http block of your nginx.conf:

js_set $headers_json utils.stringify_headers;
log_format custom '$remote_addr\t$time_local\t$request\t$status\t$headers_json';

utils in js_set must match the filename (without extension) of the file with the stringify_headers() function.

Finally, use this log format in the access_log directive of your virtual host:

access_log /var/log/nginx/with_headers.log custom;

Do not forget to reload nginx after making these changes (systemctl reload nginx or whatever command is appropriate for your environment).

You can make the log more readable with a command like this:

cat with_headers.log | sed 's!\\x22!"!g; s!\\x5C!\\!g' | awk -F'\t' '{print $5}' | jq

Conditional Logging

Most likely, you don’t want to log request headers for every request: in case of a DoS attack, this will generate lots of I/O and will result in very large log files. It may be desirable to log only some requests.

Luckily, nginx supports conditional logging.

For example, if your site is behind Cloudflare and you want to log requests coming from Brasil (using the CF_IPCountry header), you can do something like this:

map $http_cf_ipcountry $loggable {
    default 0;
}

server {
// ...

    access_log /var/log/nginx/with_headers.log custom if=$loggable; 

// ...
}