Monitoring Docker, OpenShift and Kubernetes - Version 5.7 - Journald input

Version 5.7 of our applications and Collectord includes bug fixes and a new input that allows you to forward logs directly from Journald.

Journald input

For OpenShift clusters, we recommended using rsyslog to forward messages from journald to /var/log/message. Now you can uninstall rsyslog if you don’t need it anymore and forward messages directly from journald.

You can find the reference for the journald input in configurations for Docker, Kubernetes, and OpenShift:

As follows:

yaml

 1[input.journald]
 2
 3# disable host level logs
 4disabled = false
 5
 6# root location of log files
 7path = /rootfs/var/log/journal/
 8
 9# when reach end of journald, how often to pull
10pollingInterval = 250ms
11
12# if you don't want to forward journald from the beginning,
13# set the oldest event in relative value, like -14h or -30m or -30s (h/m/s supported)
14startFromRel =
15
16# override type
17type = kubernetes_host_logs
18
19# specify Splunk index
20index =
21
22# sample output (-1 does not sample, 20 - only 20% of the logs should be forwarded)
23samplingPercent = -1
24
25# sampling key (should be regexp with the named match pattern `key`)
26samplingKey =
27
28# set output (splunk or devnull, default is [general]defaultOutput)
29output =

In the case of Kubernetes and OpenShift clusters, include it in your ConfigMap in file 002-daemonset.conf. If you are upgrading from the previous version of the application, we recommend specifying

ini

1startFromRel = -1h

This will tell Collectord to start reading journald from only one hour behind. Considering that you have already forwarded all the host logs from /var/log/messages, this will minimize the amount of forwarded journald logs from the first start and cause fewer duplications in Splunk.

Links

You can find more information about other minor updates by following links below.

Release notes

Upgrade instructions

Installation instructions

About Outcold Solutions

Outcold Solutions builds applications for Splunk Enterprise and Splunk Cloud. Our certified monitoring solutions, powered by Collectord, bring logs, metrics and events from Kubernetes, OpenShift and Docker clusters, Linux hosts and Windows containers into Splunk, with the dashboards and alerts that help developers watch their applications and operators keep their clusters healthy. Our search apps query Kubernetes and AWS live from the search bar, with nothing to ingest, and OS AI Agent puts the model you choose to work inside Splunk, with each user's own permissions. Since 2017 we have been helping businesses keep what they need to answer complex questions about their infrastructure in one place.