Tuesday, November 20, 2012
ABCs of log file analytics
Saturday, October 6, 2012
Big Data and DSL ( Domain Specific Language )
In our space of log management a DSL serves many purposes. Log files, especially multi-structured log files, contain very rich information – not only at a system level but also at a business and feature level. This information is “logged” not in one file but is spread across many files of many file types. Providing a simple search solves specific problems that IT is interested in but the usefulness of a simple search on log files stops at that. The higher value of business intelligence from log files requires a DSL. Lets see what the benefits of a DSL are.
- Describes arbitrary text layout for automated parsing
- Describes the context or meaning of the text in order to express more than that which is explicitly stated
- Automatically creates an efficient structured schema
- Defines semantics for search and browse of log files
- Defines application specific tags – example “trend-able” attributes, Status and configuration etc.
Having this rich definition allows for a wide range of applications that can be built out of log files. For example knowing which attributes are status, configuration and trends allows an app to treat them differently and use them appropriately inside the application.
SPL™ is a DSL for machine data that enables companies like Aruba and IBM to mine their logs and enable a wide set of people inside their enterprise ranging from support and services to sales and product management to leverage this data.
More in upcoming posts on how SPL™ enables rapid development of Big Data applications for enterprise and IT.
Tuesday, September 25, 2012
Machine data benefits sales and service
How can sales people benefit from log data?
Most systems( think storage boxes or wireless devices or medical equipment or aircraft engines) produce bundles of data - some logs have error and support related information. Some have system configuration, information on which knobs have been turned on and usage information( how long was the call and how many participants were on a call for example). Glassbeam can consume all that disparate multi-structured data and make sense of it all through its SPL(Semiotic Parsing Language).
So based on machine log information, Glassbeam can highlight which customers would run out of licenses by feature based on their past 3 months of usage, plot when they will run out and flag the account as needing follow up.
A customer may be running out of their service entitlements ( if you think your CRM system has upto date customer information think again and check your log files). Glassbeam can dashboard the customers who will be eligible for contract renewals and provide a 3 month snapshot of the customer to enable a renewal sales opportunity.
Technical sales people can quickly look at a customer's configuration before they go on a sales call. They may find that a customer has changed a configuration recently that has resulted in slowness and be prepared to address their customer concerns with actual data during their visit.
There are many more and I will showcase such examples in the following posts but Big Data from machine logs are a gold mine for businesses. Smart companies are already exploiting this to their competitive advantage.
Sunday, September 16, 2012
What is the value of Big Data?
Our customers are line of business executives in large companies and they understand the need to extract answers and solutions from their big data projects.
Wednesday, September 5, 2012
Business Impact of Machine Data Analytics in a Fortune 100 Account
Saturday, August 25, 2012
Loading log data for analytics
Our overall approach is using our patent pending SPL™ for defining the semantics & semiotics of the unstructured data and creating a platform and set of applications that can then create a normalized structure and a pre-defined set of applications to visualize that data.
As an example lets say you want to see all customers on a particular version of your product with a specific known defect and you want the trends of how often this defect as affected other attributes of the product such as performance over time and you want to create rules to alert any time such events happen. This involves parsing machine log data, combining it with a knowledge base and integrating with a CRM application. Glassbeam dramatically reduces the time to deliver this business value by providing pre-built platform components as well as a pre-defined yet configurable set of applications to report on such data.
Wednesday, August 15, 2012
Machine data and customer intelligence
In the business-to-business world there are hundreds and thousands of devices, which periodically send back, log information providing information on usage, errors, various configuration parameters etc. This information is a gold mine but it’s hard to extract the gold since the log files are esoteric and not fully structured. It takes a lot of time and effort to understand the meaning and context of the data in the files. Glassbeam has been working with large enterprises over the past three years perfecting a solution to extract gold from dirt using its patent pending SPL™
.
Now, companies like IBM and Aruba have instant insights into their users and usage of their products. For example a high tech manufacturer selling complex systems, can now know real-time if a customer is close to their license limit and proactively engage the customer for upsells. There is a real time time instant dashboard, available to all employees and execs, showing the status of all machines and devices sending data, their versions, licenses, usage patterns and enabled features. Before the sales person goes to the account, she is armed with all possible information about the account, mined from machine logs using Glassbeam and delivered in easy to use dashboards. All discussions are now based on facts rather than assumptions.
Sunday, July 22, 2012
Leveraging machine data to reduce support costs
Most progressive support organizations are now moving to leverage Big data to become proactive. They want to put behind the days when support teams were always behind the curve, with the customer knowing about problems much before support knows about it. Further its takes days or weeks for support teams to understand what is going on based on logs uploaded. Tools to analyze logs and determine possible issues have helped but they are mostly single user tools which help highlight simple keywords and when a log file consists of bundles of files with multiple sections and formats a simple search does not help.
According to a recent survey among the support groups of 3 of the world’s top selling storage vendors, it takes an average of 12 hours to identify and resolve any issue. Of this, up to 30% of the time is spent in determining root cause - finding, organizing, and making sense of the glut of diagnostic data coming back from the product.
At Glassbeam we have been working on some very interesting solutions that dramatically improve support productivity. What if you could parse multiple sections of a log file or a set of files, whatever format they in, and apply business rules on data within the file to determine if there were known issues. As an example – say your customer uploads a log file into you salesforce.com instance( or whatever CRM you use) while creating a case. What if the log file can automatically be parsed, a quick summary of current configuration shown, rules from your knowledgebase applied to the file to determine possible issues and even recommend solutions based on previously known cases? That's a potential savings of 11 hours and 55 minutes per case!
At Glassbeam we are applying big data to solve thorny problems for the enterprise and its executives.
Sunday, July 15, 2012
Looking at the softer side of execution
Hire the best...
We should hire only the BEST and no one else. This is very true especially as we are at a critical growth phase in our evolution. “A” players tend to hire A+ players because they care about their reputation and know that is the only way to excel and achieve higher goals. “B” players end up hiring similar caliber or “C” players, and “C” players go down the rank to build a team of “D” players because they avoid tackling tough problems. By being careful and deliberate on hiring the best and the best only, we avoid a deck of cards that does not take anyone anywhere.
Never give up...
Never give up, retool if necessary, but never ever give up. Start--ups are like swimming in an ocean where tides come and go. By having the courage of constantly trying and not giving up, we tend to dramatically increase our probabilities of success. Markets change, strategies change, products change, but if one is clear on the core values of why we are in this together, then each downturn is an opportunity to excel at the next upturn of events.
You cannot do everything...
It is very easy to want all and not be able to focus on one or two important things that can drive product success in the market. Therefore, we owe it to ourselves to constantly keep reminding each other on making clear choices and tough decisions. This is truer for Glassbeam since we deal with data end to end as an end user application. We have to make a conscious effort to focus only on a few things that we do the best and leave others for our customers and partners to derive value on their own.
Tuesday, July 10, 2012
Moneyball for sales and service
Glassbeam gives you the tools to make such decisions in your business, based on unfiltered machine data.
Once products are sold to customers, the product manufacturer has very limited visibility into how a customer uses the product. The customer feedback is always filtered through the words of the field person or the call center agent or the customer themselves. And not all data is captured. This is not the way the internet companies work. Web based product companies are constantly collecting data on consumer usage and mining the data to make their web pages better, to provide a better experience to their customers or to help realize additional value to simply provide a better performance by knowing and acting on issues proactively. Physical product manufacturers are reactive because the tools to collect and mine log data are not geared towards business users.
Any product executive needs to be on top of how customers are configuring and using their product and find issues before their customer does. As one of our customers put it, “It is embarrassing when my customers know more about what is happening to my product before I do”. Many product manufacturers log or collect tons of data but don't have tools to make sense of the data being collected.
With Glassbeam, sales and service people can arm themselves with detailed customer intelligence before an account call – based on actual machine reported data. Everything can be logged and should be logged – machine usage, performance metrics, specific configuration and customer specific settings, various events across all the components that make up the storage box. Glassbeam’s customers are realizing the power of data by capturing machine chatter and mining the data for to make better decisions on support models, account management, product roadmaps and licensing.
Monday, February 6, 2012
Skills needed for big data analysis?
While these skills are important for an organization that wishes to do everything in-house, there are companies like us that are obviating the need for acquiring this talent. Especially if you are a manufacturer of computer-centric technology products you don't need to look beyond our SaaS-based offering to easily collect and analyze data.
What's really important is to adopt a practice, nay culture, of building products designed to collect and send back operational data. That's the battle half-won. Leave the rest to us.
And think twice before posting that Excel guru req!
Tuesday, January 31, 2012
McKinsey weighs in on Big Data
Some key takeaways:
-- Industries with the most potential for reaping benefits from big data include Finance, Insurance, Transportation and Warehousing, and Health Care.
-- Big data is spawning new business models - in some cases companies that genuinely embrace big data are turning into big data consultants for other firms
-- The majority of the economic surplus from big data is being garnered by consumers - in the form of reduced prices, better information etc
-- Big Data is putting up big numbers: A beverage manufacturer improved forecasting accuracy by 5%; retailers reducing the number of items it stocks by 17% and so on.
-- And perhaps most importantly for this economy, Big Data Analytics is expected to generate 140K-190K additional specialist jobs, as well as create the need for an additional 1.5 million managers!
Way to go!
Tuesday, January 24, 2012
So, what is SPL?
SPL is an intuitive language that describes the structure and semantics of a class of documents. For semi-structured data, as is typical with our customers, it means we can utilize the structure inherent in the data and reflecting it in a high-performance and highly normalized data warehouse.
The SPL description of a class of documents and its semantics is passed through an interpreter to generate the database Data Definition Language (DDL) for staging, database DDL for the final warehouse, DDL and Data Markup Language(DML) for metadata to generate the UIs, and internal representations to perform a parse, a transactional or bulk load, and ETL transformations.
Please contact us if you need more technical details on our technology.
Sunday, January 15, 2012
Our customer-onboarding process
While a detailed description of each phase is beyond the scope of a blog post, here's an outline:
1. Discovery and Documentation - document amongst other things what aspects of a log file need to be processed.
2. Analysis - essentially study log files and the feasibility of parsing relevant sections with SPLi
3. Design - here's where we identify and codify the relationships between various objects.
4. Development - develop cool SPL code to parse our client's data.
5. Testing
6. Deployment - including setting up the infrastructure and populating it with existing/historic data.
7. Provisioning - ensuring user access to the system.
Keen to try it out? Start off with a Jumpstart
Thursday, January 5, 2012
Glassbeam and support teams
One of the greatest benefits of using Glassbeam was the usage of threshold events and automatically opening cases in CRM systems (hitherto done by L1 support personnel). This resulted in substantial cost savings.
Further, the client was able to support a growing install base without increasing headcount in a linear fashion. Thereby, accruing more revenues without corresponding increase in cost - all adding up to revenue and margin increases!
Bottom lime was a 20% reduction in costs. And, a more satisfied and productive support team.