Tuesday, March 26, 2013

Detecting patterns in wireless logs

Machine data can contain various types of unstructured information ranging from events to configuration and statistical information. Trouble shooting wireless devices involves running several CLI(Command Line Input) commands and analyzing the output for patterns. An access point may be rebooting, a process may be consuming unusual amounts of memory, a controller may have some incorrect settings, ports may be shutdown etc. The number of variables a support person in a product company, selling wireless devices, needs to track from the logs is so large and dispersed across various CLI outputs that manually going through this is very time consuming. Our wireless customers use Glassbeam to parse these files intelligently so business rules can be built on top of the parsed data to identify common patterns. A collection of such patterns are then made into a summary report dramatically reducing the time to problem resolution. All this is possible only because there is technology to take highly random structured or multi-structured machine data and make sense out of it in an automated way.

This is an example of a highly complex Big Data( High Variety) problem involving parsing, ETL and data modeling done with an advanced language called SPL( Semiotic Parsing Language).


Tuesday, March 19, 2013

Know your products and your customers

Sometimes its the small data hidden inside the Big Data that ends up making a huge impact on the business. Often we find our customers using Glassbeam machine data analytics to find all sorts of information about their customers that they could never have known if not for their machine data logs.

When a problem occurs in a product out in the field its important to know who else( which other customer) could be affected by the issue. Product managers and customer services people can then proactively notify them about it. When you have multiple products or many different versions/models of a product this becomes complicated. Your field sales does not have the time to keep track of the most recent changes to every device you have shipped to every customer. The only source of truth is your machine data.

Mining this machine data using Glassbeam, our customers have been able to identify the high priority accounts that have a similar problem and proactively communicate to their customers about issues and resolutions. This leads to lower costs and higher customer satisfaction.

In other situations our customers have found product usage information that was not possible to detect without logs. For example, one of our projects involved working with a company that sells fire detection and other industrial equipment. This particular equipment included a lens that tracked possible contamination or dirt levels in an environment. But the lens itself can get dirty overtime and needs to be replaced. In order to determine the level of dirt in the lens a sensor would send back data periodically indicating the dirt level. While analyzing the sensor logs, we found that the dirt level did not change overtime in some locations. This was puzzling since the lens does get dirty after months of normal use. The coating must have been so good or something else was going on. Our customer further investigated this and found that in such cases the installer had forgotten to remove the lens cap!!!

Machine data is informative and insightful if mined the right way. While there is a lot of talk around Big Data, its the small data hidden inside the Big Data such as machine data that often provide the Ahahs!



Sunday, March 10, 2013

How big data is changing the world of business

"Big data can help capture customer preferences and put that information to work in designing new products. In this area, online companies are taking the lead." - WSJ

There are other big brands, with connected products, that are putting Big data to use in designing better products and providing better value to their customers. We are just beginning to make sense of all the data out there. Exciting.

Friday, March 1, 2013

Glassbeam @ Strata

We had a great launch at Strata. Many people stopped by and had wonderful things to say about our new product - Glassbeam Search a unified search and log management solution. We got several good pieces of advice and feedback as well and many thanks to all of you.

As we continue building this out we find most companies needing the following

  1. Structured and unstructured analysis - meaning search *and* BI on their data.
  2. Uses beyond operational search - Some companies use traditional log management solutions today or have built their own but when they saw all of Glassbeam's capabilities their eyes open wide with possible applications of the product through out the enterprise and not just IT logs. 
  3. An ETL/ELT based approach to extracting value from unstructured data in many cases( as opposed to index and then deal with it). 
  4. Business focused solutions in addition to syadmin/IT focused solutions. For example how can we create dashboards that show my product managers everything about my installed base, versions. 
  5. Monetizing data- Several companies were interested in Glassbeam's approach to helping monetize data by building self service dashboards on customer's capacity and performance, site level views etc. 
We are excited to be working on some compelling use cases around complex data and analysis with some big name brands and most recently a large $10 billion global healthcare company. So stay tuned for more.