WYNTK on TBSM: TBSM Design Pattern – Architectural Model (COTS and Custom Composite Applications)

The purpose of this TBSM Design Pattern is to create models representative of the key relationships and dependencies within COTS and Custom Composite applications. The implementation of this design pattern should provide a foundation for visibility into the applications by operations support teams, SMEs, developers and IT management. The level of detail expected in models of this type is generally abstracted from executives and the line of business in favor of higher level model design patterns.

One of the “holy grails” of Business Service Management is to have an understanding of the cause and effect relationships between the IT environment, the business services and applications provided and their actual use and/or performance to meet the businesses goals and objectives. Clients are beginning to deploy synthetic and real user monitoring solutions to help provide this insight. IT and the LoB are reporting weekly how well they’re performing and meeting business goals and objectives. This design pattern lays a foundation for being able to relate fine grained IT components to business service and application user experience and performance and the understanding of how one may impact the other. Without this foundation, we will continue to force operations and support teams to seek “the needle in the haystack” when the business reports problems within key business services and applications.

The first goal of modeling within this design pattern is to understand the architectural and implementation characteristics that applications, databases and custom composite applications can take. This is the nuts and bolts; the model that represents deployment onto the servers, the networks and into the datacenters. It’s how load balancers, virtualization, redundancy, clustering and high availability come into play in supporting applications, databases and custom composite applications. For example, an Oracle deployment may be in an Active-Passive, Active-Active (Cluster) or Oracle Grid architecture, deployed on one or more servers within one or more datacenters. It’s one thing to model within this area, but most clients struggle with knowing the state or status of the architecture. Clients must develop an approach for understanding what architecture components are active, online, in standby, load sharing, etc. through improved instrumentation.

The second and most important goal is to continue to improve the level of visibility and understanding of how these components operate, perform and ultimately their role in supporting business services and applications. How is the business service impacted when an application or database has a problem? What happens when one member of the database cluster fails? This doesn’t come from your vendor but from your SMEs, support groups, developers, and the business. The path to this point may be achieved with a simple model or require a fairly sophisticated model depending on your environment. Follow on TBSM design patterns will focus on interacting within this area.

The first challenge a client often has is that their monitoring tools group have not matured their fundamental management and monitoring to a point of having enough visibility into these areas. Clients continue to battle with keeping up with the basics of hardware and operating system changes, new versions, etc. Most clients have invested in some fundamental hardware and operating system monitoring capability or are making use of custom scripts, system logging or similar. They may have set up some basic logfile or process/daemon monitoring, but not yet invested into capabilities that enable deep visibility into applications. More often than not, it’s the SME groups (SysAdmins, Application Support or DBAs) that have the best visibility into these areas. Do not accept “they have their tools and we have our tools” as the answer. There is tremendous value in integrating SME tooling, scripts, etc. into the collective management and monitoring environment, even if it’s only for the purposes of driving these models.

More and more, COTS applications and databases include some capability for internal instrumentation and visibility into state, status, availability and performance characteristics. This is usually enabled through some additional configuration option, module, via an administrative console or a complete stand alone application (Oracle Enterprise Manager). Third party vendors and open source solutions may exist for management and monitoring of these applications or databases. Think agents for things like Exchange, SAP, DB2 or Oracle (IBM Tivoli, Quest Software, etc.) here or specialty applications that tend to fall into the advanced diagnostics and performance tuning areas (Quest Spotlight on Oracle).

Custom composite applications are going to expand the usage of COTS technology to include custom software, application servers, integration techniques (SOA, EAI, etc.), and other distributed and mainframe systems. The same approaches described above should be followed, with more emphasis on working with the various development and support SMEs.

Creating an accurate model is the easy part, bringing it to life requires data points, metrics or other information that can be used to determine state, status, performance, availability and impact. Modeling requires working with the various SME groups to fully understand what has been deployed, its operating characteristics and how it supports the business services and applications.

Once a good understanding has been achieved, the monitoring tools group will need to perform a gap analysis on what they have and don’t have to be able to represent the model in a production TBSM deployment. Do yourselves a favor here and partner up with the SME’s. Work through the politics and find a way to integrate their information and insight into the core management, monitoring and event collection solutions. Identify the gaps you have in your core tools and visibility, identify a plan to fill those gaps so your BSM solutions can provide the expected value.

I generally see a couple different approaches for building out these Architectural Models and they tend to play into the two scenarios above. Most often due to the weaknesses in fundamental instrumentation and monitoring, clients will simply create a template for the server and a template for the COTS application. Each template will generally include the expected “up/down” type status rules looking for incoming events. The COTS application template may have include a few more rules to cover more specifics about the application, but more often than not they are broad based status rules looking for process, daemon and log file messages.

There’s nothing wrong with this approach at all, but it’s a matter of getting to the fine grained visibility that’s difficult with this approach. Do you need to know that a specific transaction, process or table space out of dozens or hundreds is the cause and that it’s only impacting a portion of your business service or is knowing that you have an application or database problem good enough? My operations background says that it’s always best to help folks get to the specific problem as quickly as possible and to not generalize things.

The preferred approach is to break things out into more specific components of the architecture deployed. This approach DEPENDS on instrumentation and visibility into all of these areas. This approach DEPENDS on your ability to generate unique events, metrics, KPIs, etc. that can be directly associated to these areas. This approach gives you the most flexibility to tie discreet components into the specific business service and application areas that thy may most directly support, enable or impact.

Focus on simple templates for the COTS application/database or custom composite application PLUS simple templates for the supporting infrastructure server(s), application/database core components and marry them together based on the behaviors of the application as it relates to the underlying infrastructure components. This is the key here. This is the only way to manage each uniquely and as a whole based on how the overall architecture was designed and implemented. If you can get this right, you will be able to achieve the powerful business service and application models needed for Business Service Management plus the ability to put the right information into the hands of the operations and support staff so they can identify and resolve the problems faster.

Upcoming TBSM Design Patterns will focus on the service, functional/sub-service, and process/transactional design patterns.

**NOTE to Readers: Would examples of these within TBSM or Visio be helpful?**