Machine Learning in practice

When using Machine Learning, the more data the better. It is a smart way to extract value from data, collecting a lot of data helps to increase accuracy. Hence, Big Data and Machine Learning go hand in hand.


Splunk offers the possibility of actually using Machine Learning in an easy way. For example, for an alert for deviant response times, you no longer have to rely on statistical models that have difficulty with irregular patterns. Machine Learning enables you to create clusters of regular and unusual response times. These clustering algorithms can also be linked to other relevant metrics, for example it can be normal to have an increased response time when it’s busy on a website.


Splunk Machine Learning Toolkit

When using a big data platform, there are different ways to have access to Machine Learning. Splunk offers the most extensive functionality through the free add-on “Machine Learning Toolkit“. The web-based environment provides access to a wide range of algorithms, without the need to write a single line of code. The included algorithms are directly applicable to a wide range of use cases. There are a number of algorithms for detecting deviations from the standard; Is the number of web shop visitors normal on this Sunday morning?

In addition, there are also a number of algorithms that help to look into the future. This helps a donut seller to plan his inventory. Or an IT manager with the timely ordering of new hardware or determining his budget for the purchase of cloud services in the next quarter.

To gain insight into underlying structures or dependencies in data, a number of clustering algorithms is used. This is how an insurance company, for example, has noticed that houses built before 1940 with thatched roofs and a chimney in cold winters have a disproportionately large risk of fire. As this specific combination of characteristics is now known, a discount for sweeping the chimney can be offered to this specific group of customers. This will result in less fires, which is beneficial to both the customer and the insurance company.


ITSI (IT Service Intelligence)

Premium app – Splunk ITSI  lets you look at an IT landscape or organization through services. A service is an abstract concept and can include many different things; think about technical things such as web servers, databases or storage, but also business services such as support desk or web shop sales. Each service has an indicator which measures the status of the service. The status of the KPIs indicates whether the service is functioning properly or not. For example, a KPI for a database is the number of queries per second and the time it takes to respond.

It can be difficult to classify the value of a KPI. A minimum number of active users in a web shop on Sunday morning is not a problem, but if we see the same number of active users on a normally busy Thursday night there is almost certainly a problem. In order to overcome this problem, Machine Learning is an integral part of ITSI. It helps to classify the value of KPIs by looking at the normal behaviour in the previous time period. Deviations are then classified as a potential problem. By using Machine Learning, the status of a service can even be predicted up to 30 minutes ahead. We look at the link between KPIs and service indicator in the past, after which it is applied to the values ​​of KPIs at this moment. Based on the collected data, the organization can make data-driven decisions for preventive action or conduct research. More importantly before there actually is a problem that affects customers or the business.


In practice

Splunk’s web environment provides access to many different algorithms that can use all the data available in the platform. These algorithms and trained models can be used immediately in operations. An alert can be given, if the real values ​​deviate too much from the prediction. Or a dashboard may light up red if a potentially fraudulent transaction has been detected. This allows us to say that we are at the beginning of a new era.

10 years ago, Splunk was one of the first to easily extract value from log files and machine data. Today Splunk makes Machine Learning accessible to everyone.


July 2018