Customize HAProxy Log format(Mode TCP / HTTP)

Customizing the HAProxy Logs

There are plenty of reasons why you might want to customize the fields captured by the HAProxy logs. For example:

• the default log format is giving you more information that you need
• you’re missing an important piece of information with the default log format
• you need to structure the fields in a way that an external tool can read them
• you rely on a standard log format and HAProxy must also comply

Luckily, it’s easy to add a template that says which fields to log and in which order. You just need to use the log-format directive. Let’s dive into its structure.

Adding a New HAProxy Log Format

The log-format directive goes into your defaults, frontend or listen section.

It specifies a string that contains variables referring to the fields that you’d like to capture.

 

defaults

    log-format ""

 

When you place this into your defaults section, it affects all of the proxy sections (frontend and listen) that follow. If you need a different format for a particular proxy, adding another log-format to that section will override the default.

In the next section, you’ll learn about the variables that are available.

Variables

A large part of your custom log format will likely be made up of variables that are pre-defined by HAProxy. Others you will define yourself.

In the broadest terms, all variables follows the rules below when added to a format string:

• It is preceded by a percent character: %
• It can take flags in braces {}
• If multiple flags, then they are separated by commas within the braces
• Flags may be added or removed by prefixing them with a + or a – sign
• Spaces, other than those between variables, must be escaped by a backslash

Currently, there are three flags:

• Q: quotes a string
• X: hexadecimal representation
• E: escapes characters (“, \, ]) with backslashes

Here are a few examples:

 

log-format "%{+Q}r"

# Outputs: "GET / HTTP/1.1"

 

log-format "%{+X}Ts"

# Outputs: 5C5342A0

 

log-format "%{+E}HQ"

# Outputs: ?myparam=[some_brackets\]

 

log-format "%{+E,+Q}HQ"

# Outputs: "?myparam=[some_brackets\]"

 

 

The pre-defined variables (also available in the documentation) are:

Variable	Meaning
%B	bytes read
%CC	captured request cookie
%CS	captured response cookie
%H	hostname
%HM	HTTP method (e.g. POST)
%HP	HTTP request URI without query string
%HQ	HTTP request URI query string
%HU	HTTP request URI
%HV	HTTP version (e.g. HTTP/1.1)
%ID	unique ID
%ST	status code
%T	GMT datetime
%Ta	active time of requests (from %TR to end)
%Tc	time to establish TCP connection to server
%Td	Tt – (Tq + Tw + Tc + Tr)
%Tl	local datetime
%Th	connection handshake time (SSL, PROXY protocol)
%Ti	idle time before the HTTP request
%Tq	time to get the client request
%TR	time to receive the full request from first byte
%Tr	response time
%Ts	timestamp
%Tt	total session duration time
%U	bytes uploaded
%ac	process concurrent connections
%b	backend name
%bc	backend concurrent connections
%bi	backend source IP (HAProxy connects with)
%bp	backend source port (HAProxy connects with)
%bq	backend queue
%ci	client IP
%cp	client port
%f	frontend name
%fc	frontend concurrent connections
%fi	frontend IP
%fp	frontend port
%ft	frontend name transport (‘~’ suffix for SSL)
%lc	frontend log counter
%hr	captured request headers
%hrl	captured request headers (CLF style)
%hs	captured response headers
%hsl	captured response headers (CLF style)
%ms	accept date (milliseconds, left-padded with 0)
%pid	HAProxy process ID
%r	HTTP request
%rc	number of retries
%rt	request counter
%s	server name
%sc	server concurrent connections
%si	server IP
%sp	server port
%sq	server queue
%sslc	SSL cipher used (e.g. AES-SHA)
%sslv	SSL version (e.g. TSLv1)
%t	datetime (with millisecond resolution)
%tr	datetime of start of HTTP request
%trg	GMT datetime of start of HTTP request
%trl	local datetime of start of HTTP request
%ts	termination state
%tsc	termination state with cookie status

 

The default log formats use these variables in the following ways:

TCP log format:

log-format "%ci:%cp [%t] %ft %b/%s %Tw/%Tc/%Tt %B %ts %ac/%fc/%bc/%sc/%rc %sq/%bq"

log-format "%bi:%bp %ci:%cp [%tr] %ft %b/%s %TR/%Tw/%Tc/%Tr/%Ta %ST %B %CC %CS %tsc %ac/%fc/%bc/%sc/%rc %sq/%bq %hr %hs %{+Q}r"

 

HTTP log format:

log-format "%ci:%cp [%tr] %ft %b/%s %TR/%Tw/%Tc/%Tr/%Ta %ST %B %CC %CS %tsc %ac/%fc/%bc/%sc/%rc %sq/%bq %hr %hs %{+Q}r"

 

CLF log format:

log-format "%{+Q}o %{-Q}ci - - [%trg] %r %ST %B \"\" \"\" %cp %ms %ft %b %s %TR %Tw %Tc %Tr %Ta %tsc %ac %fc %bc %sc %rc %sq %bq %CC %CS %hrl %hsl"

 

 

You can create your own format by using any of the pre-defined variables in the order that you wish. The following logs the client’s IP address and port, the server’s IP address and port, the path, and the response status:

 

 

 

 

참고

https://www.haproxy.com/blog/haproxy-log-customization/

https://cbonte.github.io/haproxy-dconv/1.7/configuration.html#8

https://www.sumologickorea.com/blog/haproxy-log-format/