The Logtail Apache mode parsing plug-in structures log content based on the definitions in an Apache log configuration file. The plug-in parses the content into multiple key-value pairs.
Introduction to Apache logs
Apache servers output logs based on the log format, log file path, and name specified in the Apache log configuration file. For example, the configuration CustomLog "/var/log/apache2/access_log" combined
specifies that the combined format is used to print logs and that the log file path is /var/log/apache2/access_log
.
Apache log formats:
Combined format
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
Common format
LogFormat "%h %l %u %t \"%r\" %>s %b"
Custom format
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %D %f %k %p %q %R %T %I %O" customized
The following table describes the fields. For more information, see mod_log_config.
Format
Field name
Description
%a
client_addr
The IP address of the client.
%A
local_addr
The local IP address.
%b
response_size_bytes
The size of the response in bytes. If no bytes are sent, a hyphen (-) is displayed.
%B
response_bytes
The size of the response in bytes. If no bytes are sent, 0 is displayed.
%D
request_time_msec
The time required to process the request, in microseconds.
%f
filename
The filename.
%h
remote_addr
The name of the remote host.
%H
request_protocol_supple
The request protocol.
%I
bytes_received
The number of bytes that the server receives. You must enable the mod_logio module.
%k
keep_alive
The number of requests processed on this connection.
%l
remote_ident
The identification information provided by the remote host.
%m
request_method_supple
The request method.
%O
bytes_sent
The number of bytes that the server sends. You must enable the mod_logio module.
%p
remote_port
The server port number.
%P
child_process
The ID of the child process.
%q
request_query
The query string. If no query string exists, an empty string is returned.
%r
request
The request content, which includes the method, address, and HTTP protocol.
%R
response_handler
The type of handler on the server.
%s
status
The initial HTTP status of the response.
%>s
status
The final HTTP status of the response.
%t
time_local
The server time.
%T
request_time_sec
The time required to process the request, in seconds.
%u
remote_user
The username of the client.
%U
request_uri_supple
The path of the requested URI, without the query string.
%v
server_name
The server name.
%V
server_name_canonical
The server name set by the UseCanonicalName instruction.
"%{User-Agent}i"
http_user_agent
The client information.
"%{Referer}i"
http_referer
The source page.
Entry point
If you want to use a Logtail plug-in to process logs, you can add a Logtail plug-in configuration when you create or modify a Logtail configuration. For more information, see Overview.
Configuration description
Parameter | Description |
Log Format | Select the log format that is defined in the Apache log configuration file. The options are common, combined, and custom. |
Apache Configuration Field | The log configuration section in the Apache configuration file. This section usually starts with LogFormat.
|
Source Field | The source field that stores the log content before it is parsed. The default value is content. |
Regular Expression | The regular expression that is used to extract Apache logs. Simple Log Service automatically generates the regular expression based on the content in the Apache Configuration Field. |
Extracted Fields | The log fields (keys) are automatically generated based on the content in the Apache Configuration Field. |
Keep Source Field On Parsing Failure | Select Keep Source Field On Parsing Failure to preserve the source field if parsing fails. |
Keep Source Field On Parsing Success | Select Keep Source Field On Parsing Success to retain the source field after parsing. |
Renamed Source Field | If you select Keep Source Field On Parsing Failure or Keep Source Field On Parsing Success, you can rename the source field that stores the original log content. |
References
Configure a Logtail pipeline using API operations:
GetLogtailPipelineConfig - Get a Logtail pipeline configuration
ListLogtailPipelineConfig - List Logtail pipeline configurations
CreateLogtailPipelineConfig - Create a Logtail pipeline configuration
DeleteLogtailPipelineConfig - Delete a Logtail pipeline configuration
UpdateLogtailPipelineConfig - Update a Logtail pipeline configuration
Configure a processor plugin in the console:
Collect container logs from a Kubernetes cluster using a CRD (stdout/file)