Every couple months or so John Willis (aka @botchagalupe) and I meet up over at the Marietta Diner to catch up on what we’ve each been working on. Often these multi-hour discussions lead us into the depths of how emerging Internet and cloud based company’s IT organizations evolve and operate in comparison to the more traditional medium-to-large enterprises (DevOps, ConfigMgmt, Monitoring, Tooling, Agile, Lean). This most often includes discussion on the latest F/OSS tool or app and how these new companies make use of these to meet the needs of what the traditional large enterprise companies rely on the “Big4” type vendors to provide.
I’ve been spending quite a bit of time thinking about what the new new thing may be in the area of IT operations, IT management and IT monitoring areas with specific interest in what the future holds for BSM, APM, Consolidated Event Management, Social in ITOps, etc. There are so many new things out there that appear to be working incredibly well that there most certainly appears to be opportunity to apply them to age old IT ops, IT management and IT monitoring found in so many well established companies. As someone who’s worked in those environments or been exposed to them in my current role at IBM I see so many areas ripe for innovation and improvement. I’ve probably got a pet peeve list a mile long that I would love to attack with new and innovative technologies that could easily shake up and break up the monotony of the legacy approaches.
One area I’ve recently become very interested in is the very hot big data, analytics and search areas specifically how they could be used with common IT Operations areas and data sources such as log, machine data, events, metrics and KPIs. When I think about these areas and the potential this type of technology has – I see a capability that could radically change the established foundation within the typical legacy IT operations environment and how they visualize, consume, interact with and make decisions during typical IT operations, IT management and IT monitoring activities and processes. There’s a lot of innovation happening in this area and I’ll try to touch on my thoughts on that topic soon as this will become a new theme area for my personal blog.
I’ve recently become a huge fan of the Logstash, Kibana and Elasticsearch projects. With this trio of technology and a few other BigData components I see a world of possibilities within the IT Ops space to bring disparate data sources (application logs, syslogs, events, traps, etc.) together within a simple, scalable architecture to provide a familiar, intuitive and easy to use solution for extracting valuable, actionable information from the mountains of data collected in a typical small/medium/large enterprise environment. The platform created by these F/OSS projects could move many closer to this “holy grail” of getting value from those data sources which many companies invest significant money, resources and effort in IT management, business intelligence or similar data archive/warehousing, dashboard/reporting platforms and BSM/APM/EventMgmt platforms.
In the next few posts I’ll share some of the experiments I’ve completed recently around an initial theme of “Big Events” and how search technologies offers a fresh approach to the decades old IT event management approaches as I explore these technologies and practical applications that they offer today.