Parsing HTTP POST data with NGINX OpenResty LUA | Devops Junction

In this article let us learn how to parse through the HTTP post payload in Nginx Openresty with LUA language

OpenResty is an nginx distribution which includes the LuaJIT (Just in Time) interpreter for supporting Lua scripts. this LUA script helps you extend your regular NGINX with custom functionalities and features

Open resty supports a lot of Great third-party plugins and components, you can find the list here

The great advantage of OpenResty over NGINX is the support of LUA and it enables you to program and extend the NGINX

In this article, let us see an example on how to use Open resty LUA scripting to parse the HTTP POST request body and transform the request.

Adding Header Dynamically based on the Payload or Request Data

Refer to the following nginx.conf file of openresty where we have the LUA code block.

If the request method is POST the LUA starts the Logging and prints the request body ( you can optionally disable it as it might cause IO latency when the payload is enormous in size)

Our Objective is to look at whether there is an external email ID  ending with yahoo, aol or Gmail, outlook etc

Here is the complete flow diagram of the design

We parse the POST body and the args to check whether it has any public email domains such as yahoo, ymail, outlook etc.

If it's found, we are adding a header X-is-email before forwarding the request to the backend using proxy_pass

 

OpenResty NGINX Conf with Lua script - to Parse HTTP POST body data

here is the complete source code

server {
    resolver local=on valid=1s;
    set $upstream_endpoint api.gritfy.io;

    listen 80;

    location ~ (^.*$)  {
      proxy_set_header X-Forwarded-Host $host;
      proxy_set_header X-Forwarded-Server $host;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Url-Scheme $scheme;
      proxy_set_header X-Forwarded-Proto $scheme;
      proxy_set_header Host $http_host;
      proxy_redirect off;

      client_max_body_size 100m;
      client_body_buffer_size 100m;

      set $logme 0;

      #access_log /var/log/nginx/lua.log log_body if=$logme;

      if ($request_method = POST ) {
        access_by_lua '
        local isemail="FALSE"
        ngx.req.read_body()
        local args, err = ngx.req.get_post_args()
         if not args then
             ngx.say("failed to get post args: ", err)
             return
         end
         for key, val in pairs(args) do
             – ngx.say("KEY: ", key, ", TYPE: ", type(val), ", VALUE: ", val)
             if type(val) == "table" then
                – for k, v in pairs(val) do ngx.say("k: ", k, ", v: ", v) end
                for k, v in pairs(val) do
                  if type(v) == "string" then
                    local match = ngx.re.match(v, "(%40|@)(yahoo|aol|ymail|gmail|live|outlook)")
                    if match then
                        isemail="TRUE"
                        break
                    end
                  else
                    ngx.var.logme=1
                  end
                end
             else
                – ngx.say(key, ": plain: ", val)
                 local match = ngx.re.match(val, "(%40|@)(yahoo|aol|ymail|gmail|live|outlook)")
                  if match then
                      isemail="TRUE"
                      break
                  end
             end
         end
         ngx.req.set_header("X-is-email", isemail)
         – ngx.say("X-is-yahoo : ",isyahoo )
        ';

        proxy_pass http://$upstream_endpoint/api/v1$1;
    }
  }
}

 

This is just one use case and an example of what OpenResty NGINX and LUA scripts can do.

you can further extend this or change it to suit your needs

Hope this helps

Cheers

Sarav AK

---