nginx日志改为json格式

nginx 日志改为json格式 场景描述效果变更旧样式新样式

---

场景描述

正常使用nginx时，使用默认的日志输出格式，对于后续日志接入其他第三方日志收集、清洗环节，因分隔符问题可能不是很友好。

xxxx - - [19/Feb/2024:11:16:48 +0800] "GET /time_feed HTTP/1.1" 200 42 "http://sre-support.xxxx-inc.top/back" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36" xxxx - - [19/Feb/2024:11:16:49 +0800] "GET /time_feed HTTP/1.1" 200 42 "http://sre-support.xxxx-inc.top/back" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36" xxxx - - [19/Feb/2024:11:16:50 +0800] "GET /time_feed HTTP/1.1" 200 42 "http://sre-support.xxxx-inc.top/back" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36" xxxx - - [19/Feb/2024:11:16:51 +0800] "GET /time_feed HTTP/1.1" 200 42 "http://sre-support.xxxx-inc.top/back" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36" xxxx - - [19/Feb/2024:11:16:52 +0800] "GET /time_feed HTTP/1.1" 200 42 "http://sre-support.xxxx-inc.top/back" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36" xxxx - - [19/Feb/2024:11:16:53 +0800] "GET /time_feed HTTP/1.1" 200 42 "http://sre-support.xxxx-inc.top/back" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36"

nginx同时支持将日志输出为json格式，修改较小，下面为效果。

效果 {"@timestamp":"2024-02-19T11:23:09+08:00","server_addr":"10.0.0.251","remote_addr":"xxxx","host":"sre-support.xxxx-inc.top","uri":"/time_feed","body_bytes_sent":42,"upstream_response_time":0.000,"request":"GET /time_feed HTTP/1.1","request_length":349,"request_time":0.001,"status":"200","http_referer":"http://sre-support.xxxx-inc.top/max_data","http_x_forwarded_for":"","http_user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36"} {"@timestamp":"2024-02-19T11:23:10+08:00","server_addr":"10.0.0.251","remote_addr":"xxxx","host":"sre-support.xxxx-inc.top","uri":"/time_feed","body_bytes_sent":42,"upstream_response_time":0.001,"request":"GET /time_feed HTTP/1.1","request_length":349,"request_time":0.001,"status":"200","http_referer":"http://sre-support.xxxx-inc.top/max_data","http_x_forwarded_for":"","http_user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36"} {"@timestamp":"2024-02-19T11:23:11+08:00","server_addr":"10.0.0.251","remote_addr":"xxxx","host":"sre-support.xxxx-inc.top","uri":"/time_feed","body_bytes_sent":42,"upstream_response_time":0.000,"request":"GET /time_feed HTTP/1.1","request_length":349,"request_time":0.001,"status":"200","http_referer":"http://sre-support.xxxx-inc.top/max_data","http_x_forwarded_for":"","http_user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36"} {"@timestamp":"2024-02-19T11:23:12+08:00","server_addr":"10.0.0.251","remote_addr":"xxxx","host":"sre-support.xxxx-inc.top","uri":"/time_feed","body_bytes_sent":42,"upstream_response_time":0.000,"request":"GET /time_feed HTTP/1.1","request_length":349,"request_time":0.001,"status":"200","http_referer":"http://sre-support.xxxx-inc.top/max_data","http_x_forwarded_for":"","http_user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36"} {"@timestamp":"2024-02-19T11:23:13+08:00","server_addr":"10.0.0.251","remote_addr":"xxxx","host":"sre-support.xxxx-inc.top","uri":"/time_feed","body_bytes_sent":42,"upstream_response_time":0.000,"request":"GET /time_feed HTTP/1.1","request_length":349,"request_time":0.001,"status":"200","http_referer":"http://sre-support.xxxx-inc.top/max_data","http_x_forwarded_for":"","http_user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36"} 变更 旧样式

nginx一般默认会打开日志格式化格式，我这里为默认的格式，可能有的同学使用的是自定义格式，但是都是用“ - ” 分割的。

样式为：

新样式

因本身nginx支持json格式日志输出，下面为我使用的示例，我这里增加了获取源站响应时间upstream_response_timer，所以下面先增加了upstream_response_timer取值方式。 下面的log_format展示了当前获取的字段，各位可根据自己需求进行自定义。

http { map $upstream_response_time $upstream_response_timer { default $upstream_response_time; "" 0; } log_format log_json escape=json '{"@timestamp":"$time_iso8601",' '"server_addr":"$server_addr",' '"remote_addr":"$remote_addr",' '"host":"$host",' '"uri":"$uri",' '"body_bytes_sent":$body_bytes_sent,' '"upstream_response_time":$upstream_response_timer,' '"request":"$request",' '"request_length":$request_length,' '"request_time":$request_time,' '"status":"$status",' '"http_referer":"$http_referer",' '"http_x_forwarded_for":"$http_x_forwarded_for",' '"http_user_agent":"$http_user_agent"' '}'; access_log /var/log/nginx/access.log log_json; error_log /var/log/nginx/error.log error; ........ }

样式为：