Monitoring Windows Containers Configuration
Collector configuration
Join Rules
By default collector joins all messages with previous if they start with spaces. Below you can find how to specify a custom rule on the example of java application.
If this is a sample of the application logs.
[2017-09-04T06:28:05,664][WARN ][MyComponent] java.security.AccessControlException: access denied at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) ~[?:1.8.0_131] at java.security.AccessController.checkPermission(AccessController.java:884) ~[?:1.8.0_131] [2017-09-04T06:28:05,664][WARN ][MyComponent] another message
You can specify the join rules, where you configure that you want to match all containers with the name that contains my_app
in their name, and pattern for the new message should match regex ^\[\d{4}-
.
[pipe.join::my_app] matchRegex.docker_container_name = .+my_app.+ patternRegex = ^\[\d{4}-
collector.conf Reference
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 | # collector configuration file # # Run collector with flag -conf and specify location of the configuration file. # # You can override all the values using environment variables with the format like # COLLECTOR__<section>__<key>=<value> # As an example you can set dataPath in [general] section as # COLLECTOR__general__dataPath=C:\\some\\path\\data.db # This parameter can be configured using -env-override, set it to empty string to disable this feature [general] # location for the database # is used to store position of the files and internal state ; dataPath = .\data\ # log level (trace, debug, info, warn, error, fatal) ; logLevel = info # http server gives access to two endpoints # /healthz # /metrics ; httpServerBinding = :8080 # telemetry report endpoint, set it to empty string to disable telemetry ; telemetryEndpoint = https://license.outcold.solutions/telemetry/ # license check endpoint ; licenseEndpoint = https://license.outcold.solutions/license/ # license server through proxy ; licenseServerProxyUrl = # license ; license = # docker daemon hostname is used by default as hostname # use this configuration to override ; hostname = [general.docker] # path to docker root folder ; dockerRootFolder = C:\ProgramData\Docker\ # Log files [input.files] # disable container logs monitoring ; disabled = false # root location of docker files ; path = C:\ProgramData\Docker\containers\ # glob matching pattern for log files ; glob = *\*-json.log* # files are read using polling schema, when reach the EOF how often to check if files got updated ; pollingInterval = 250ms # how often to look for the new files under logs path ; walkingInterval = 5s # include verbose fields in events (file offset) ; verboseFields = false # override type ; type = wincontainers_logs # read hardlinks instead of original files # on Windows that allows not to block the original directory, when # files are getting deleted by daemon ; hardlinks = true ; hardlinksPath = C:\ProgramData\Docker\containers\ [input.files::docker_service] # disable host level logs ; disabled = false # root location of docker files path = C:\ProgramData\Docker\ # regex matching pattern match = ^service.txt$ # limit search only on one level recursive = false # files are read using polling schema, when reach the EOF how often to check if files got updated pollingInterval = 250ms # how often o look for the new files under logs path walkingInterval = 5s # include verbose fields in events (file offset) verboseFields = false # override type type = wincontainers_host_logs # field extraction extraction = ^\[(?P<timestamp>[^\]]+)\]\[(?P<docker_service_component>[^\]]+)\]\[(?P<docker_service_level>[^\]]+)\]\s(.*)$ # extractionMessageField = # timestamp field timestampField = timestamp # format for timestamp # the layout defines the format by showing how the reference time, defined to be `Mon Jan 2 15:04:05 -0700 MST 2006` timestampFormat = 15:04:05.000 # Adjust date, if month/day aren't set in format timestampSetMonth = true timestampSetDay = true # timestamp location (if not defined by format) timestampLocation = Local # Splunk output [output.splunk] # Splunk HTTP Event Collector url ; url = # Splunk HTTP Event Collector Token ; token = # Allow invalid SSL server certificate ; insecure = false # Path to CA cerificate ; caPath = # CA Name to verify ; caName = # Events are batched with the maximum size set by batchSize and staying in pipeline for not longer # than set by frequency ; frequency = 5s ; batchSize = 768K # Splunk through proxy ; proxyUrl = # Pipe to join events (container logs only) [pipe.join] # disable joining event ; disabled = false # Maximum interval of messages in pipeline ; maxInterval = 100ms # Maximum time to wait for the messages in pipeline ; maxWait = 1s # Maximum message size ; maxSize = 100K # Default pattern to indicate new message (should start not from space) ; patternRegex = ^[^\s] # Define special event join patterns for matched events # Section consist of [pipe.join::<name>] # [pipe.join::my_app] ## Set match pattern for the fields #; matchRegex.docker_container_image = my_app #; matchRegex.docker_stream = stdout ## All events start from '[<digits>' #; patternRegex = ^\[\d+ # Stats for Windows Containers [input.wincontainer_stats] ; disabled = false # how often to collect stats ; statsInterval = 30s # override type ; type = wincontainers_stats # Not supported on Windows [input.docker_events] ; disabled = true # Not supported on Windows [input.system_stats] ; disabled = true # Not supported on Windows [input.proc_stats] ; disabled = true |