Category — Business Activity Monitoring

I’d like to share my vision and definition for what I feel best describes Business Transaction Management (BTM). I’ve started a new page to track this emerging area, value proposition, marketplace, vendors and technology approaches. I’m planning a series of BTM oriented guest authors and podcasts as well as deeper dives into BTM as it relates to my emerging BSM Value Proposition model for 2009. Feel free to join in the conversation!

First off, let’s talk about just what a transaction is. Here’s my current working definition.

Transactions are one of, if not the most, critical components of how IT supports the business in meeting business goals and objectives. They are the simple and complex entities that get work done. They are the “movers and shakers” within the IT environment that are actively responding to client or business requests via the business services and applications that IT delivers. They move key data and information between infrastructure components that make up complex end-to-end business services and applications that exist in nearly every company today.

There are many common types of transactions and transactional architectures deployed by IT to support getting things done within companies today. These include the most common n-Tier (web, appsvr, db) transactions, the much hyped Service Oriented Architecture (SOA) approach, the tried and true mainstay of Mainframe (CICS, IMS) based transactions and I’m sure many others could be added to this list in the EAI, Complex Application, Web, J2EE, Mainframe or Network Service Provider areas.

I also like to bundle into this area something that’s likely not often thought of when we discuss types of transactions or transactional architectures and that’s the area of transactional based Workload Management (Batch Job/CRON) and Managed File Transfer (FTP/SFTP/SCP) type solutions. These technology areas have a very transactional based nature and are often responsible for getting some of the most high value units of work done within a typical enterprise. They should not be forgotten from any modern day or leading edge BTM solution.

What is Business Transaction Management (BTM)?

My views for BTM span many areas and focus on the following types of BTM activities. The BTM marketplace has vendors focused on one or more of these areas. Hopefully we can establish a uniform way to classify vendors and their technology, products and solutions into these or other categories.

Transaction Discovery and Inventory: Techniques for discovering transactions whether active, passive, configured, scheduled, or in-flight, for the purposes of establishing relationships between dependent systems, applications, databases, components, etc. based upon this transaction discovery as well as establishing an inventory or repository of transaction information. This would be used to extend a configuration management system, database (CMDB), asset management system or other. Ideally, we’d use this capability to establish the finite business relationships each transaction has, how it supports a business service, application or other unit of work, the impact if the transaction fails, blocks, queues, is slow, etc.

Transaction Tracing: Heavily focused on in-flight transactions, flows and paths, transaction tuning and optimization, transaction troubleshooting, complex relationships with other systems applications databases. This section includes transaction latency tracking, timing and profiling.

Transaction Monitoring: Focused on monitoring every aspect of the transaction, performance, availability, capacity, SLA, timings, etc. This section includes latency monitoring, an area of critical importance in many financial and trading environments.

Transaction Intelligence and Analytics: The application of sophisticated analytics to transactions to learn, monitor and managed transactions with applied intelligence. This is often seen in the financial and telecommunications industry as anti-money laundering solutions, credit card or other fraud or abuse of services/networks or similar solutions aimed at watching for patterns of activity or behavior as seen in transactional activity. The Business Activity Monitoring (BAM) and Business Process Management (BPM) areas are also very focused on using transaction intelligence and analytics to drive the business, make better decisions and respond to changing conditions in the business environment.

Transaction Management: General term, focused on all of the above areas. I consider a transaction management solution as one that provides capabilities across all of these areas rather than just one or two.

Are there and other areas?

Why does Business Transaction Management (BTM) matter?

BTM is a key component of my “BSM Value Proposition” model and a maturing BSM strategy for any company. I’m tempted to say that if you’re not incorporating BTM into your BSM strategy than you’re likely to never get real value from your BSM solution.

I think BTM matters because the transactions are the things that get stuff done in business. It doesn’t matter if you have vendor x, y or z software, application, database, etc. they could all be black boxes. It’s the transactions that flow in and out of those things, those finite units of work that are the backbone of every business.

The future will be filled with clouds or generic compute capability full of loosely coupled services, interfaces, and transaction pipelines to get work done. Things we know today as web servers, application servers, database servers, etc. will all be commodity black boxes in the future and we’ll program against API’s or other loosely coupled interfaces to accomplish work or move data and information between the IT environment components.

Understanding the performance, availability, capacity, reliability, and impact of these transactions on IT and the business is critical just as it has been for the traditional IT infrastructure that enables them. This level of understanding and impact must not be limited that of the typical IT organization, application or integration architect, developer or administrator. Transactions and their impact must be easily understood by the folks on the front line, within IT management and the line of business. Every transaction’s contribution to the bottom line or impact on a higher or lower level system, interface or business goal or objective must be thoroughly understood.

Folks, it’s simple. Transactions that don’t complete, fail, slow down, queue up, hit a bottleneck, etc. will impact the business in significant ways. Batch jobs or other file transfers that don’t complete can cost companies in many industries large fines and penalties, lost revenue, botched orders, and ultimately upset customers or business partners. Managing end-to-end transaction flow and user experience could easily be said to be much more important than anything else you do in the network/systems/application management & monitoring area.

How important are transactions in your environment? What are you doing to increase your ability to think, operate and respond differently by deploying transaction management within your organization?

What’s the state of operationalizing Service Oriented Architecture (SOA) monitoring and management tools? Are the typical network, systems, enterprise operations/management centers (NOC/SOC/EOC/EMC) up to speed on how to manage, monitor, triage, troubleshoot and in general understand how SOA is being used in companies that are adopting it? Should the operations center care that they have an event from something related to SOA infrastructure and respond differently than they would for a non-SOA event? Have SOA events, incidents, problems, process and workflow been thoroughly implemented in such a way that “it just works” like traditional enterprise monitoring and management? Or, are these fancy SOA monitoring and management solutions really reserved for those applications experts responsible for complex application support and development?

If a client continues to struggle with fundamental e2e service monitoring and management, transaction monitoring and management or even batch job monitoring and management, what will their chances of success be for SOA monitoring and management? Could SOA and associated “service or transaction oriented monitoring” be a catalyst to shore up these other areas? Should one be tackled/improved before starting on another? At a minimum, instituting a “service oriented” organizational structure and mentality is certainly something I’d recommend for anyone adopting broad based SOA principles.

Eric Roch offers some solid advice on SOA Monitoring and Management which highlights that there’s more need for doing the fundamentals of systems, application and service management and monitoring really well as a foundation for SOA Monitoring and Management.

Others (and my preferred focus area) feel that monitoring SOA should really be more closely related to monitoring what this SOA initiative and deployment’s all about - the business. Business Service Management (BSM), Business Activity Monitoring (BAM) and Business Process Management (BPM) all play a key role in helping understand how IT infrastructure, systems, applications, etc. support and impact the business’s goals and objectives. The fairly new buzzword Business Transaction Management (BTM) spearheaded by Correlsense and OpTier really speaks to the desired need here.

I feel that it’s got to be a focus on both of these areas, but with a strong preference to the “B” buzzword set since most IT organizations are likely using the “improve/standardize/reuse/efficiency/time-to-market” spin to aide in business support and justification for their SOA initiatives. That said, you’d BETTER focus on the things that the business cares about and show them tangible evidence that your SOA initiative is making things better for them. This is very possible by adopting a BSM, BAM, BPM, BTM (or a preferred combination) strategy that focuses on providing the right level of business visibility into the SOA environment and more importantly the e2e business services, applications, transactions, processes and activities. It ultimately all ties back to the service level agreements delivered to the business anyway right?

What’s on the market these days for SOA Monitoring and Management? Should you get your monitoring and management tooling from your core SOA platform vendor or should you take a third party, “best of breed” approach? Are there true “vendor neutral” solutions out there? Are clients implementing SOA architectures based on multiple vendor’s technology, solutions and products?

• Progress/Actional
• Amberpoint
• Avicode
• CA Wily
• Correlsense
• HP
• IBM Tivoli ITCAM for SOA
• Manged Methods JaxView Suite
• MuleSource Galaxy and HQ
• Nastel Autopilot
• OpTier
• Oracle/BEA
• Progress (Actional, Sonic, Mindreef)
• SOA Software
• Symphoniq
• Tidal Software
• Truviso

Some additional content on some of these vendor solutions is available here.

Who might the “market leader” be of these SOA specific solutions? What makes them a leader? What capabilities, features, functions would be considered “best of breed”, differentiator, must have, core, desired, nice to have, etc.?

What’s “really” needed for SOA monitoring and management?

1. Web Services
2. ESB
3. Transaction Performance
4. Transaction Availability
5. Transaction State/Status
6. SOA Registry
7. SOA Security
8. Service Discovery and Relationship/Dependency Mapping
9. Transaction Discovery and Mapping

Anything else missing here? What here needs to be specialized in its own product versus just extending the investments clients have already made?

Please do share your thoughts here. There are folks lurking who really need help in figuring this stuff out and/or improving products and capabilities on the market today!

In an repost of an article from a couple years back, Robin Bloor provides some updated color on the state of BPM and SoA. It’s apparent that some of the other “B” buzz words have the same challenges that exist on the BSM front.

Things to ponder…

• How can these projects with such touted value to the business or IT be successfully implemented?
• Where are vendors falling short in helping “solve the organizational problems” that often cause these “B” projects to fail?
• Is throwing technology/product at the problem the best place to start?
• What should a next generation organizational structure look like? Can IT and Business organize around end-to-end business service/process delivery and support?
• How can organizations be incented, encouraged, mandated to have an end-to-end business service/process focus?
• Where success stories for BPM, SOA, BSM, BAM, etc. exist, how have these technologies been operationalized, organizations changed, workflow/process/procedure modified to reap the benefits?
• Is it foolish to think that any of these organizational challenges can ever be solved or at least minimized?
• Do we have generational issues here that will change as Baby Boomers retire and Gen X/Y/Z move up the ranks in IT and Business?

Give the post a read, I found these two very applicable to all of the things I’m seeing with BPM, BSM, BAM, etc.

Question 5: What are the most difficult steps within a BPM project – and what makes a SOA project tedious?

“B” is our middle name. We have “B” scattered throughout everything that we do. At times we fight over who owns the “B” word. I’m in search of a unified “B” story and solution. IMO, if we had this, it’d be tough to compete with us in any of the “B” acronyms.

The “B” Business TLA’s: Business Service Management (BSM), Business Process Management (BPM), Business Performance Management (BPM), Business Activity Monitoring (BAM), Business Transaction Management (BTM), Business Intelligence (BI) and I’m sure there are others.

What’s it going to take to have a unified “B” story and solution? Sure, we’ve probably got mentions in individual roadmaps and presentations of how we’ll integrate with this, share data what that, use Cognos here or there, send events from one tool to the other, etc. but what about a real “B” solution? IMO, these approaches just prolong client value and significantly delay any real innovation in core products.

What’s the cost of “forking” and creating a new solution entirely? One that focuses on becoming best of class in all of the “B” areas (ok, at least do all of them pretty darn good)? One that can be implemented and managed by one team free from (well, probably not) the organizational politics that’d exist if it was a “solution by integration” solution. One that has the best possible chance of truly aligning business and IT. Ok, this is probably cost prohibitive, but its GOT TO BE THE END GOAL!

This is where the politics come in unfortunately…where would you start? Which “B” is the most important “B”? Is it Business Service Management - my preference is here of course. Our friends in other organizations would see it other ways for sure. We must find the right way to develop the “B” story and solutions in ways that are most beneficial to the client. We must include content in each others products that “treads” on each others turf. We must have joint releases that build towards the unified “B” story and solution. When we release a new business process management suite (BPMS) we must include dashboards, models and integrations that provide value OUT OF THE BOX inside our BSM product. This must be backed up with the business and services consultants who have consultative based skills to guide our clients through the process because this isn’t about the product as much as it is about working through the organizational problems and politics.

A unified “B” story and solution may sound like a pipe dream, but it’s what clients really want to strive towards and our competitors are making giant strides in this direction. What would your ideal “B” solution look like? If you were king for a day …

In my last post, I discussed the issues facing IT Operations today that make real time analytics-based solutions a “must have”. In this post, I’d like to address more specifically the problems they can solve. Since I want to give enough detail on the problems and how real time analytics can help, I’ll spread this over a couple of posts.

“Too many alerts that don’t help me solve problems”

One of the biggest issues facing Operations teams is that when performance problems occur with mission critical applications, they have to expend a massive amount of manual effort to identify and repair them. They sift through the endless stream of alerts from their siloed monitoring solutions and try to humanly correlate them based on tribal knowledge. Much of this effort is because static threshold-based monitoring solutions give them alert storms for perfectly normal behavior and mask abnormalities that are the earliest precursors to problems.

Real time analytics-based solutions solve this by obviating the need for static thresholds. Instead, these solutions learn the normal behavior of every metric being collected. Armed with this understanding of normal, these solutions alert only to the abnormal behaviors that are the true precursors to problems. There is no longer a need to sift through alert storms and try to determine which alerts are relevant to the current problem and which are not. Sophisticated dynamic thresholding algorithms are used to learn the normal behavior down to the most granular level possible using clustering calculations. Algorithmic sophistication is critical as different metrics behave very differently and cannot be modeled from a single algorithm or assumed distribution. These solutions must also have mechanisms to handle seasonal events that may differ greatly from normal behavior, but do not indicate a real problem. Without this level of sophistication, large amounts of false positives result as was seen in early forays into dynamic thresholding that assumed normally distributed data (which IT metric data rarely is). My company’s solution, Integrien Alive, provides mechanisms to import large amounts of historical data very quickly and use it to calculate normal behavior immediately to remove the need for a learning period. Alive also performs topology-based rollup of alerts to provide a smaller total number of alerts with better context. The bottom line is that dynamic threshold-based alerting eliminates a ton of manual effort in problem solving.

“We don’t understand what leads to problems so we are always reactive”

Because the IT Operations team is relying on human correlation of alerts after a problem occurs, they are always in a reactive state. In some cases they may have tribal knowledge of the patterns of behavior of certain problems that gives them a heads up, however even in these cases it is most often too late to do anything before the problem occurs. Some Operations teams attempt to capture their tribal knowledge in correlation rules. These rules may help for awhile, however as the business or infrastructure changes they soon become obsolete, resulting in more manual effort to manage them. The problem is the sheer number of devices and metrics being managed in today’s infrastructures. There is no way a human can correlate hundreds of thousands (even millions) of metrics from tens of thousands of devices. This is another problem that real time analytics solutions are built to solve.

Armed with the knowledge of normal, these solutions can correlate previous alert and metric behaviors and predict future abnormal behaviors based on currently observed abnormalities. For example, the solution can alert to a problem in the application server tier based on the amount of devices in that tier that are performing abnormally. The Level 3 application expert receiving this alert is also provided with predictions of future abnormal behavior. In this case, the alert indicates that a key database performance indicator will be breached in 15 minutes or less with 86% probability, bringing down the database. The Level 3 application expert forwards this information to the DBA, who then makes a quick configuration change that avoids the database crash. This type of automated correlation allows a proactive approach to problem solving that isn’t possible with manual methods.

My company’s Integrien Alive solution takes correlation one step further, allowing users to set performance key indicators at the business, user experience, or IT infrastructure level. When these key indicators are breached, Alive captures a model of the building pattern of abnormalities that led to the problem, up to an hour before it occurred. These problem models (called Problem Fingerprints^TM) focus troubleshooting efforts and reduce Mean Time To Identify (MTTI) and Mean Time To Repair (MTTR) the first time the problem occurs by indicating exactly which tiers of the application (and what specific metrics) are performing abnormally. Once these models have been captured, Alive can scan real-time metric data for a return of that pattern. If a problem pattern defined in one of the Problem Fingerprints is matched with high enough probability, Alive sends a predictive alert informing the Operations team of the looming problem, the probability the problem will occur, when it is likely to occur, what to look for and how it was solved previously.

As we’ve seen in this discussion, real time analytics solutions are all about:

• large reductions in the manual effort associated with static threshold-based alerting
• increased focus for troubleshooting efforts to reduce MTTI/MTTR
• predictive alerting to allow proactive performance management

In my next post we’ll discuss additional problems solved by real time analytics. We’ll also delve into BSM and how these solutions are an essential catalyst to achieving it.

Is there such a product or capability in a product that could do enterprise wide scheduling and calendaring? What I’m interested in is not related to people, meeting, conference room scheduling, but more along the lines of enterprise planning, BI, etc. where one could input into such a solution that ‘between the hours of 12pm and 2pm’ are the most critical hours of the day for this business service, application, transaction. Or something that could be the repository for killer business impact information based on BI type data feeds, etc. Or something that bleeds into service catalog/IT cost & usage that could track costs/impacts/revenues for use of business and IT services during a certain period, from a certain location, etc.?

Not sure if this is an SLA type tool as SLA’s and OLA’s may not always necessarily be involved. Something more along the lines of an uber-business-intelligence knowledgebase that’s business service, application, process, transaction, flow, etc. aware.

Just curious…

I worked up a pretty slick demo for a client this week who expressed an interest in Business Activity Monitoring (BAM) as part of their Service Oriented Architecture (SOA), Enterprise Service Bus (ESB) and Websphere MQ/Message Broker/ESB evaluations. I’ve had many thoughts on this topic from my past, but the work on this demo and discussions with the client affirmed that leveraging the Tivoli Netcool/RAD platform and enabling technolgies/products as an entry point into Business Service Management (BSM) or Business Activity Monitoring (BAM) is viable. Maybe it’s just the way my brain works, but bringing a Service Oriented Architecture and associated services to life with the Tivoli Netcool/RAD Business Service Management (BSM) / Business Activity Monitoring (BAM) solution was really quite easy.

I continue to be amazed by the depth of assets and resources that we have here at IBM. The things that I’m seeing come from our Service Oriented Architecture (SOA) efforts are incredible enablers for creation of the Business Service Management (BSM) and Business Activity Monitoring (BAM) practice. The BSM, BAM, BPM, B-I-N-G-O acronym soup of technologies, products and concepts will benefit greatly should the concepts of service orientation take hold in the mainstream and we can make it easy for clients to grasp these concepts, give them straight forward and easy ways to change from the legacy ways of enterprise and service provider architecture and design, and support them with solutions that help them manage, monitor and leverage these investments for the betterment of their business goals and objectives.

Lots of exciting times in the future for sure!

Let’s face it, here in North America most IT organizations are structured around functional silos of expertise. It’s not uncommon to see the Network Group, Windows Server Group, Unix Server Group, Mainframe Group, Application Group, Operations Group (NOC/OCC/ECC, etc), Tech Support/Help Desk, etc. These functional silos support everything and anything the IT organization delivers for the business. Organizing this way seems to be the norm here in the United States as it allows for certain economies of scale to be achieved in the functional silo areas. I’ve seen weekly/monthly reports highlighting measures such as “server admins per 100 servers” or “network engineers per 100 interfaces” as the key contribution from within the functional silo. The “we can do more with less” battle cry was born of the days of controlling costs and headcount. This process of organizing to achieve economies of scale is generally fairly broad at the lower levels of the IT infrastructure and tapers as you progress up the stack. There are fewer experts in the application areas, fewer database, web and application server administrators, etc. About the only time you see someone who claims ownership for a key business or IT service may be found in a “Product Manager” type role serving as a bridge (roadblock) between IT and the business.

Let’s look at this from the operations perspective. The typical Network Operations Center (NOC), Operations Center, Enterprise Command Center (ECC) or Help Desk are the “ones” who get “stuck” with some level of responsibility for how the business, clients, customers, etc. use the sum of all the parts developed and delivered by those functional IT silos. They get the calls that the service is unavailable, that transactions are failing, that the business is being impacted by poorly performing IT services, applications, etc. They resemble the responsible party, but generally aren’t given the responsibility, access, authority or tools to be anything more than ticket and triage experts who know how to follow the run book to escalate the tickets before the timer for the SLA is violated. Fair enough, this capability is needed.

But who really owns the service? Who owns the business experience? Who owns and is responsible for poorly performing services that impact business goals and objectives? Who doesn’t get their bonus paid out in full? Who gets a poor performance review? Who will stand up and take responsibility for the end-to-end service instead of pointing a finger towards another functional silo? Who will work with the business directly, in partnership, to manage the service as a whole, top to bottom, across all technologies, integration points, applications, and functional IT silos?

You’re all very well aware of how these organizations work. There is constant focus and incentives on silo specific tasks, projects and activities over end-to-end service delivery goals and objectives. Finger pointing, territorial mud slinging, gnashing of teeth, etc. are the water cooler conversations after the traditional “all hands on deck troubleshooting crisis conference bridge” session is complete. Do the goals and objectives of Business Service Management (BSM) have a chance becoming reality in these environments? I don’t think so. If these groups can’t check their emotions and egos at the door and realize that they exist for the business to meet business goals and objectives, Business Service Management will never mature to a high level in the Business Service Management maturity model. Yes, Business Service Management projects often start within these organizations. Yes, Business Service Management projects may be viewed as successful in these environments. But moving up the maturity model to a point where quantifiable value and direct business improvement can be measured will be a significant challenge in these organizations.

I don’t think this end-to-end service owner is the VP or Director of IT, MIS, or other common IT management roles or functions. They certainly play a role, but I have a feeling they’re just too distracted with the other pressures of running IT organizations today. I don’t think this is the Operations Center or Help Desk. These folks are often in the trenches, fighting fires, and dreaming of getting in front of problems rather than reacting to them. They do a great job managing the lifecycle of outages and incidents in the IT environment, escalating, communicating and manning the phones and this is a needed function as well. I’m not sure if this needs to be a new organization that breaks down those functional silos into functional service silos. It may make sense in smaller organizations to consider this, but you do need to have some economies of scale to manage larger environments. I see the early stages of Service Managers here in the USA but these tend to be too closely aligned with ITIL and IT process improvement efforts. They don’t appear to have enough responsibility to do anything other than define SLA/OLA/UPC’s related to service delivery and support.

Does a service management/ownership matrix overlay organization make sense? A matrix overlay that has named senior level ownership responsibility for each service where this senior level owner may be from the IT side or the business side of the organization. One that joins together named functional SME’s across the service delivery, service support silos AND from the business side to form a cross-functional, cross-organizational team that can step up to the plate in terms of service delivery and support ownership. One that has complete ownership of the service, associated transactions, processes, activities and flows that deliver IT and business services. Their goals and objectives would be aligned to service delivery and support that is in complete alignment with business goals and objectives.

Expanding the tenants of Business Service Management and IT - Business Alignment across an organizational structure similar to this would be a huge step forward in realizing the true and measurable value of Business Service Management. Aligning technology components with the business services and applications they deliver and support is an initial step, but aligning people, process, goals and objectives with the same is HUGE. I’d be very interested to hear your thoughts and comments on these ideas. What works, what doesn’t work, are organizations thinking like this? Am I way off base thinking like this???

Additional food for thought:

• New roles of the service aligned organization: Business Service Management (BSM) Analyst, Business Service Management (BSM) Architect, Business Service Architect, Service Modeling Architect, Transaction Management Engineer, Process Management Engineer
• Traditional IT organization management evolves into the emerging Enterprise Architecture function
• Shared IT Services, Virtualization, Grid and Utility Computing could be a key component to underpin this service aligned organization
• How to sell / deploy BSM into the “traditional” IT organization
• How to sell / deploy BSM into an emerging and evolving IT organization