Splunk fields extraction for container logs

For the container logs, forwarded by the collector, it is possible to specify field extractions rules, specific for image names, container names, or combination of them.

All container logs have source format, which includes container ID, container name, container image name, and stream.

/docker/{docker_container_id}/{docker_container_name}/{docker_container_image}.{docker_stream}

Using this knowledge you can create field extraction rules for specific image or container, also including glob patterns, using wildcards.

As an example, you can specify field extraction for nginx container in props.conf using a wildcard character for the container ID, container name, and docker stream. This field extraction applies to all containers created from the nginx docker image.

[source::/docker/*/*/nginx:*]
EXTRACT-nginx-ingress-controller-http = ^(?P&lt;remote_addr&gt;[^ ]+)\s+\-\s+\[(?P&lt;proxy_add_x_forwarded_for&gt;[^\]]+)\]\s+\-\s+(?P&lt;remote_user&gt;[^ ]+)\s+\[(?P&lt;time_local&gt;[^\]]+)[^"\n]*"(?P&lt;request&gt;[^"]+)"\s+(?P&lt;status&gt;\d+)\s+(?P&lt;body_bytes_sent&gt;\d+)\s+"(?P&lt;http_referer&gt;[^"]+)"\s+"(?P&lt;http_user_agent&gt;[^"]+)"\s+(?P&lt;request_length&gt;\d+)\s+(?P&lt;request_time&gt;[^ ]+)\s+\[(?P&lt;proxy_upstream_name&gt;[^\]]+)]\s+(?P&lt;upstream_addr&gt;[^\s]+)\s+(?P&lt;upstream_response_length&gt;\d+)\s+(?P&lt;upstream_response_time&gt;[^\s]+)\s+(?P&lt;upstream_status&gt;\d+)$</code></pre>

You can also find how to use Splunk Fields Extractor to extract fields from container logs in our blog.

Links

• Installation
  • Start monitoring your docker environments in under 10 minutes.
  • Automatically forward host, container and application logs.
  • Test our solution with the embedded 30 days evaluation license.
• Collector Configuration
  • Collector configuration reference.
  • Build custom image on top collector image with embedded configuration.
• Container Annotations
  • Forwarding application logs.
  • Multi-line container logs.
  • Fields extraction for application and container logs (including timestamp extractions).
  • Hiding sensitive data, stripping terminal escape codes and colors.
• Configuring Splunk Indexes
  • Using not default HTTP Event Collector index.
  • Configure the Splunk application to use not searchable by default indexes.
• Splunk fields extraction for container logs
  • Configure search-time fields extractions for container logs.
  • Container logs source pattern.
• Configurations for Splunk HTTP Event Collector
  • Configure multiple HTTP Event Collector endpoints for Load Balancing and Fail-overs.
  • Secure HTTP Event Collector endpoint.
  • Configure the Proxy for HTTP Event Collector endpoint.
• Collecting metrics from Prometheus format
  • Configure collector to forward metrics from the services in Prometheus format.
• Monitoring multiple clusters
  • Learn how you can monitor multiple clusters.
  • Learn how to set up ACL in Splunk.
• Streaming Docker Objects from API Engine
  • Learn how you can poll docker containers and images and forward them to Splunk.
• License Server
  • Learn how you can configure remote License URL for Collectord.
• Alerts
• Troubleshooting
• Release History
• Upgrade instructions
• Security
• FAQ and the common questions
• License agreement
• Pricing
• Contact