Flight School for Application Performance Aces

by November 28, 2017

The Evolution of Modern Application Monitoring


In the early days of aviation pilots flew by the “seat of their pants”. With little in the way of instrumentation, they relied on sight and judgement. Fine when the skies were clear and conditions good, but not so great for flying in clouds, fog and things got, well, soupy.

Stuck in the haze with no points of reference, pilots were in a space where nothing behaved normally. If they sensed the plane was descending, they pulled back on the yoke to gain altitude, only to find the plane diving steeper. Even when they believed the plane way was level, indicators suggested a sharp turn.

Not surprisingly, the best course of action was often to bail out before the plane hit the ground.

The Need for Comprehensive Instrumentation

Gut feel and instinct isn’t great for flying planes in adverse conditions. This has something to do with the way pilots process information using visual and vestibular systems to figure out where they in space.  As it turns out, fluid movements within the inner ear canal can play all sorts of tricks. What’s actually level flight might actually be processed as a steep turn, and any intuitive action to correct the situation just makes matters worse. And, without views of the horizon, pilots became so spatially disoriented that they lost complete control.

With the birth of “instrument flight” this problem was averted. It meant supplementing the basic navigational devices most aircraft carried, with a cohesive set of instrumentation pilots could use to double-check what their very fallible senses were telling them. This included, artificial horizons combined with turn and bank indicators.

But providing instruments alone wasn’t a total solution. Pilots still had to be become skilled in using them –to trust them. It’s why today’s pilots can only fly in the soup when they’ve mastered the instruments.

Up in the Soup – Piloting Modern Applications

So what does any of this have to do with modern application monitoring? Well, there are quite a few parallels.

In the not too distant past, monitoring was like the early days of aviation. Like the pilots of yesteryear, we mostly enjoyed clear visibility and failure conditions across on premise applications were predictable. But today’s cloud-based applications are far less visible and distributed systems behave in ways that challenge established monitoring beliefs. For example, a CPU utilization increase for a cloud app might now be a good thing and any associated alerting becomes unnecessary and costly overhead.

So just as aviation involved an instrumentation leap-of-faith, the same must happen in application monitoring. The systems we design and build must be capable of sensing, analyzing and responding to complexities mere mortals can never comprehend.

But this involves so much more than acquiring another tool or building fancy dashboards. Like modern avionics systems, modern application monitoring must be capable of correlating information across various components and delivering what singular dials and widgets can never provide – context. Easy to say of course, but tricky to achieve.

The Problem with More Dials and Dashboards

Too often as engineers we put faith in own dashboarding systems. We build and maintain excellent systems for monitoring and alerting individual of individual data points. CPU metrics for sysadmins, memory leaks and traces for app developers and so on. The only problem is that any action is driven without context. So, while a network alarm seems troublesome to a network specialist, does it warrant immediate attention if customer experience isn’t impacted? We don’t know because we lack context, without which we make the wrong decisions. Like ‘bailing out’ from a release when we don’t need to, or blissfully ignoring slow-acting performance twists and turns that cumulatively bring a service to its knees.

In aviation pilots are trained to quickly assess and correlate information across multiple readings on their instrumentation panel; never putting complete faith in one readout. But this isn’t so easy in IT operations. Important data might be located in multiple repositories, held in different formats, even owned by other teams.

What we need are modern application monitoring systems that deliver the equivalent of what modern pilots provide – only fully automated and at massive scale. Systems that automatically collect, scan, process and correlate data, be that logs, metrics, transactions – whatever, across everything contributing to customer experience and better business outcomes – applications, infrastructure and networks.

Beyond Monitoring – Context is King

At CA Technologies, we recognize the importance of providing both contextual awareness and guidance. Take for example CA Application Performance Management (APM), which now provides the capability to visualize and correlate infrastructure metrics with logical application topology. This means that when customers conduct web performance monitoring across modern containerized applications, they have immediate access to host and container metrics – in context of application performance – all from one intuitive interface.

And with the rich insights context delivers, we can drive better actions; it becomes our automated guidance system. That’s why many of our solutions provide automated workflows to not only analyze data and pinpoint likely root-cause, but intelligently support corrective remediation or performance optimization. One such example is the assisted triage functionality we’ve introduced with CA APM.

We can no longer manage application performance using gut feel and instinct – by the seat of our pants. We need solutions that deliver the full performance picture, whatever the complex technology soup, fog and cloud we find ourselves working in.