Outcold Solutions LLC

Monitoring OpenShift in Splunk: integration with Web Console

October 26, 2018

[UPDATE (2018-11-02)] Switched to version v1.1.0, that uses nodejs:8 by default, to make this app compatible with OpenShift Console version below 3.11.

[UPDATE (2019-08-07)] Switched to version v1.2.0, which allows to override application name from default monitoringopenshift.

[UPDATE (2021-02-10)] For OpenShift 4.x look at Integrating OpenShift Web Console 4.x with Monitoring OpenShift application in Splunk

Today we are open sourcing a Node.JS application openshift-webconsole-integration. This application allows you to integrate OpenShift web console with Splunk. It embeds two links in the OpenShift Web Console. The first link gives you the ability to navigate to the Pod or Workload dashboard, where you can review the performance of the containers, review network activity and see the logs. The second link navigates you directly to search where you can start working with the logs from this specific workload or pod.

Two examples, how the navigation works. In this example we can navigate directly to the ReplicationController monitoring dashboard and see all the metrics, network activity and the logs from all the Pods schedule with this ReplicationController.

Monitoring OpenShift - navigration from the OpenShift Web Console to Monitoring Dashboard

In the second example, we navigate directly to the Splunk search page, where we already have a predefined filter, that shows us logs only from this replication controller.

Monitoring OpenShift - navigration from the OpenShift Web Console to search logs

Installation

You can find detailed installation instructions at README.md.

We assume that you already installed Monitoring OpenShift application and collectorforopenshift.

Switch to the collectorforopenshift project, if you want this application to run in the same project, like all other our workloads.

oc project collectorforopenshift

For the next step, you need a Splunk Web URL, similar to http://splunk.local.outcold.solutions:8000. Change in the following command this URL to the Splunk Web URL, where you have Monitoring OpenShift application installed.

oc new-app -f https://raw.githubusercontent.com/outcoldsolutions/openshift-webconsole-integration/v1.2.0/openshift/templates/outcoldsolutions-webconsole-integration.yaml \
    --param=SPLUNK_WEB_URL=http://splunk.local.outcold.solutions:8000 \
    --param=SOURCE_REPOSITORY_REF=v1.2.0

This template will install the application directly from the https://github.com/outcoldsolutions/openshift-webconsole-integration with version (tag) v1.2.0. For production environments we recommend you to fork this repository and move sources of this application to your own location. With the parameters you can override the location of the openshift-webconsole-integration application. As an example

oc new-app -f https://git.local.outcold.solutions/outcoldsolutions/openshift-webconsole-integration/v1.2.0/openshift/templates/outcoldsolutions-webconsole-integration.yaml \
    --param=SPLUNK_WEB_URL=http://splunk.local.outcold.solutions:8000 \
    --param=SOURCE_REPOSITORY_REF=v1.2.0 \
    --param=SOURCE_REPOSITORY_URL=https://git.local.outcold.solutions/outcoldsolutions/openshift-webconsole-integration

This template automatically creates the route and exposes script.js. You can find the direct link to this script with

echo "https://$(oc get route outcoldsolutions-webconsole-integration -o=jsonpath='{.spec.host}')/script.js"

As an example, it can look similar to

https://outcoldsolutions-webconsole-integration-collectorforopenshift.apps.local.outcold.solutions/script.js

Open this script in the browser, to verify that you can access it. It might take a few moments for application to be built and deployed. You can check the status with oc status. Verify that you see an expected Splunk URL in the links.

script.js in browser

Now we can integrate this script in the Web Console. You can use CLI commands by following the guide Loading Extension Scripts and Stylesheets or you can do that from the Web Console as well.

OpenShift Web Console Update ConfigMap

Open the Development Console in your browser and refresh the page, verify that you can find a script.js we just created

OpenShift Web Console with Development Console

If you don't see the script, force OpenShift to recreate Console with the new configuration by deleting Web Console Pods

oc delete pods -n openshift-web-console -l app=openshift-web-console

After that you should see links at every Logs view in OpenShift web console.

OpenShift Web Console Example

Hope that this simple integration will boost your productivity!

Feel free to fork, contribute and open issues at https://github.com/outcoldsolutions/openshift-webconsole-integration.

openshift, splunk, integration, logs, metrics

About Outcold Solutions

Outcold Solutions provides solutions for monitoring Kubernetes, OpenShift and Docker clusters in Splunk Enterprise and Splunk Cloud. We offer certified Splunk applications, which give you insights across all containers environments. We are helping businesses reduce complexity related to logging and monitoring by providing easy-to-use and deploy solutions for Linux and Windows containers. We deliver applications, which help developers monitor their applications and operators to keep their clusters healthy. With the power of Splunk Enterprise and Splunk Cloud, we offer one solution to help you keep all the metrics and logs in one place, allowing you to quickly address complex questions on container performance.