Machine data operational intelligence company Splunk has used its .conf 2017 event in Washington DC this week to explain how machine data is the lifeblood of the IoT.
Machine data is data produced inside machines or devices, inside applications and inside the interconnecting software services and layers that form the lifeblood of the IoT. It’s the data inside your application (and in a Word document, that could simply be text), but it is also the data that your application uses to function… so in that sense, it also includes server response-time information and business process logs.
Machine data is also application ‘call’ data – in other words, information relating to the behavior of the application itself as it makes calls to databases, service connectors, application programming interfaces (APIs) and all other elemental parts of the network within which the app itself exists. It can also be web ‘clickstream’ data and website activity logs.
Read more: Splunk the IoT, digging through the machine data fabric
Data trails
When an IoT device runs a piece of embedded software, or ‘firmware’, it will very typically perform some or all these actions and thus will leave a trail of machine data in its wake. In this way, you can start to understand why Splunk talks about machine data being the moving, flowing, dynamic lifeblood of the IoT itself.
For the record, machine data also includes application and database configuration commands, information on message queues, change event logs and the output of diagnostic commands.
So prevalent is machine data across all computers, all networks, all instances of programmatic logic and all IoT devices that Splunk also refers to it as ‘digital exhaust’ – and from digital exhaust, we can build a ‘digital fingerprint’ to allow us to map out the behavior of all data moving through any given system using machine learning techniques.
Read more: The IoT needs a new kind of database
What is Splunk?
Machine data so defined then, we also need to ask… what is Splunk?
The company is so-named as a reference to spelunking, the term used in the United States and Canada for caving or potholing. The connection (and inference) is that Splunk provides a web-style interface for searching, exploring and generally shining a light on the interesting issues and anomalies that can be observed in machine data. Splunk also works to monitor and analyze machine data, but the search and and exploration element is paramount, hence the name.
Splunk (the product) captures, indexes, and correlates real-time data in a searchable repository from which it can generate graphs, reports, alerts, dashboards and visualization.
Splunk CEO Doug Merritt has explained that his team’s ‘first vision’ for the toolset and platform was driven by a desire to create a Google Search function for the enterprise.
“Splunk exists to make big (machine) data searchable, sortable, consumable and usable,” said Merritt.
Read more: Sharpened edge needed for IoT edge databases
Splunk .conf2017 on IoT
Splunk used the .conf 2017 conference itself to explain how these technologies come together to serve the data analytics needs thrown up by the IoT itself.
In terms of specific sessions, Markus Boenisch of BMW and Georg Schroder of German energy market software company Robotron hosted a session to share how they are using Splunk for machine learning and natural language processing (NLP) to assist production line workers with catching and fixing production quality issues.
Dereck Merck from Rhode Island Hospital hosted a session to share uses for Splunk in a hospital context, including radiation exposure monitoring with DICOM (Digital Imaging and Communications in Medicine) data, radiology workflow optimization and workload prediction with HL7 data.
In a light-hearted session entitled ‘Splunkin’ my Harley’, Geoffrey Martins, a senior technical instructor and consultant with Splunk, showed that anybody can use Splunk to analyze any IoT data, by explaining how he collects and analyzes data from his own Harley Davidson motorcycle.
“Staples uses Splunk Enterprise for real-time analysis of critical business transitions – from order management, to invoicing, to warehousing – to ultimately enhance our customer experience and stay ahead of online competitors,” said Faisal Masud, chief technology officer, Staples. “Splunk analytics and metrics are helping us optimize every aspect of what we do, including quickly identifying and correcting irregular transactions so customers receive the best possible service. The Splunk Enterprise platform is a critical piece of our business operations foundation.”
Machine data runs in IoT machines and so forms the lifeblood that pumps into the synapses of machine learning that make the IoT smart.
Coming soon: Our IoT Build events, taking place in London in November 2017 and San Francisco in March 2018 are a great opportunity for attendees to explore the platforms, architectures, applications and connectivity that comprise the IoT ecosystem.
