Category — dashboard

While I’m sure that TBSM’s use of ILOG wasn’t in the business case for IBM’s announced purchase of ILOG today, I am getting my hopes up that this is the Christmas present that anyone using TBSM is looking for.

TBSM makes extensive use of ILOG under the covers for the core of the canvas and visualization. We barely scratch the surface in capabilities (see demos below) in how its used, but it is the foundation for all of the data driven widgets used in the canvas, custom dashboards, etc.

Let’s face it, we have a long way to improve in our dashboarding and visualization capabilities. The acquisition and product convergence focus has been a significant impact on new innovation in this area within TBSM. With the upcoming release of TBSM v4.2 soon to be behind us, I hope that we can finally make investments where they are desperately needed within the data visualization, user experience, widgets, real time charting/graphing and general sex appeal areas for creating the dashboards expected of products in this price point.

TBSM 4.2 will give us a new platform to build upon, but it WILL NOT MAKE SENSE to develop other components (widgets, charts/graphs, analytics, rules, visaulziation, workflow, etc.) in the buzz word of the day (AJAX, Dojo, Python, RoR, PHP, TCR/BIRT) when we may have this ILOG portfolio at our disposal. (my opinion only, I ack the biz side of these decisions)

I strongly encourage you to voice your thoughts and requirements for improvement of TBSM’s capabilities in these areas via appropriate channels.

ILOG has a log of powerful capabilities to offer should we choose to take advantage of them. Better yet, equipping you to take advantage of them with self service designers, SDKs, APIs, etc.

For example:

ILOG Visualization Portfolio
ILOG Diagrammer (TBSM uses this)
ILOG JViews Demos
ILOG Elixer
ILOG Elixer Demos AWESOME! Look at Gauges & Dials Demo!

I hope this is the Christmas present we’ve all been waiting on for TBSM this time next year!

FireScope’s countdown excitement was revealed today (a little bit late ) and their next steps in Business Service Management innovation are now visible. It appears that there will be a 28 day launch cycle of numerous new products and tools.

FireScope’s Business Edition (BE) concept consists of a couple key components aimed as initial entry points for smaller enterprises. The fuller featured Enterprise Editions (EE) are the natural migration paths or starting point for larger enterprises. Pricing modelers are available right on their website.

Some of the “coming soon” tools looks like it puts them in company with PacketTrap and SolarWinds. FireScope BSM:BE and CMDB:BE puts them in company with Nimsoft and Managed Objects.

BSM BE

Optimized for the unique needs of small to medium sized businesses and starting at only $2,450, FireScope BSM:BE features an engaging and user-friendly interface and flexible auto discovery provides rapid time to competency, with multiple setup wizards and inline help designed to make implementation accessible for any level of user.

CMDB BE

FireScope CMDB:Business-Edition delivers the same enterprise-grade functionality as traditional CMDB’s, minus the exhorbitant price tag and pain inducing deployment. For less than $10,000, IT Organizations gain real-time visualization and documentation of their infrastructure along with a complete view of the interrelationships of the software and systems impacting those operations, without the need to hire an army of consultants and mind-numbing planning sessions.

Hmm, the free trial download isn’t available to me yet. Anyone else?

I think Ryan’s monthly wrap up posts are really nice and I’m going to start these as well.

Hot Posts for January

In Where is Quest Software’s BSM Play? I ask what’s taking Quest Software so long to roll out a new BSM story based on their Magnum Technologies acquisition. Lots of good comments here from Quest Software clients who have had both success and challenges with Foglight. Quest Software’s Foglight Product Manager chimes in for some commentary! Don’t be a stranger Brad!

In Will Compuware 2.0 Include a Clear BSM Story and Viable Solution? I question if Compuware can really reinvent itself and compete in the BSM space. Still no response here from Compuware. Where’s the 2.0?

In So You Want to do BSM? and several articles that EMA has put out in the trade rags this month highlighting the results of their BSM/SLM Market Forecast for 2008-2012 they have painted a bright future for Business Service Management. They also are emphasizing the challenges with the traditional ways of implementing it, something that I’m in complete agreement with and have been dealing with for years. Still looking for a final version of this market forcast report. More here and here.

DevCampTivoli Thought Provoking Series

In Is Your Tivoli Monitoring, Netcool/OMNIbus or TBSM Organization Structure a Barrier to BSM Success? to propose new approaches for IT organization structure to focus on end-to-end service management ownership.

In My ITM 6.x BSM Profile should include a BSM Descriptor File I propose an approach for every managed system to provide key information needed for BSM.

In All I want for the New Year is a BSM Profile for ITM 6.x I propose a concept for specific and purpose built instrumentation of managed systems using a BSM Profile.

WYNTK on TBSM Series

In WYNTK on TBSM Design Patterns: Architectural Model for COTS and Composite Applications I introduce TBSM design patterns for modeling COTS and Composite Applications.

Industry Highlights

BarCampESM was a success. Check out the OMC site for all the follow on activity and blip.tv for all the sessions. My presentation and video are available here. Take 30 minutes and watch the video. Let me know what you think? Am I way out in left field here? Too passionate?

Digital Fuel issued a couple PR’s announcing some impressive wins within very large telecommunications companies. Me thinks they’re ripe for an acquisition.

I’ve started to watch a bunch of new vendors this month. These all fit into various niche areas of the management and monitoring space and are very critical to a maturing Business Service Management deployment. True value oriented and powerful BSM can’t be done without capabilities offered by vendors such as these.

Integrien looks like the newest player in this “monitoring analytics” area. Netuitive and the former ProactiveNet (BMC) also play in this area. Integrien released version 6.0 of their product and apparently has addressed some of their scalability challenges. I was very impressed with what I saw in terms of their presentation layer, but haven’t seen much other than that. Steve Henning (VP Products, Integrien and ex-IBM Tivoli Security guy) joins the conversation and shares some insight as do many others who are very familiar with all three vendors in this space. I’ve invited Steve to guest author on the blog so maybe we can get some more insight into this much needed space!

I’ve always been pretty close to the user experience, user performance, synthetic/real-user monitoring segment. Three new vendors crossed my wires this month to join the others I’m pretty keen on (Keynote, Gomez, Tealeaf, Coradiant, IBM, HP, Quest, Compuware). Mature BSM deployments absolutely depend on the perspectives that vendors like this provide. It’s absolutely required for successful BSM and is the “glue” that joins the end-to-end service delivery chain together as the end user sees it.

Check out Knoa Software, Symphoniq and Aternity. If you have any information or experience with these vendors, I’d love to hear about it!

Almost as important to knowing how critical business services and applications are performing from the end user perspective, trying to really understand in instrument the ultra-critical transactions flowing across end-to-end services and applications is a sign of a very mature BSM deployment. In Two to Watch in Transaction Management and Monitoring Space” I call out Correlix and Correlsense as two that should be considered. I’m also looking for anyone with personal experience or information on these vendors and their technology. Correlsense’s CTO/Founder is an ex-IBMer and has a great start to a blog with teeth!

On the To-Do List for February

• The Next Generation of Business Service Management
• Hey! You got your monitoring in my RIA!
• New IBM Tivoli developerWorks collaboration sites
• More WYNTK on TBSM Design Patterns
• More in the DevCampTivoli Thought Provoking Series

What do you want to hear about?

Our TADDM product has a pretty nifty capability to help it along in its discovery process. You have an option to create files called Application Descriptors that are simple XML files that describe what business applications are deployed onto the server, what components make up the application and how these various components are organized, grouped or have relationship to the business application. Examples of TADDM Application Descriptors are available here.

What if we took this extremely simple concept and turned it into something for the ITM 6.x BSM Profile? What if we had a BSM Descriptor File? It may contain many different sub-components that help me to express the unique characteristics of what this server and installed software do to support business services and applications.

The BSM Descriptor File may contain:

• Business Service Descriptors: Information on the business service(s) this component supports/enables
• Business Application Descriptors: Information on the business application(s) this component supports/enables
• Transaction, Process or Activity Descriptors: Information on key transactions, processes, daemons, batch jobs, etc. that this component supports/enables
• Impact Descriptors: Information on how this component may impact the business goals and objectives, revenue, customer experience, metrics, KPIs, etc.
• Compliance Descriptors: Information on compliance controls that this component must adhere to.
• Risk Descriptors: Information on business risks that may be associated with this component
• Security Descriptors: Information on security policies applied to this component
• Business Schedule or Calendar Descriptors: Information on when there may be important times during the day, week, month that this component may need to be managed differently (end of month batch jobs, financial runs, maintenance windows)
• Operations Support Descriptors: Information about the on call group, escalation paths, etc.

Part of the XML tagging within the BSM Descriptor File should include annotation on how these unique components are mapped into events generated from individual ITM 6.x monitoring agents and their BSM Profile. With this information flowing freely into the event stream, making use of the powerful capabilities within Netcool/OMNIbus and TBSM 4.x become very easy. These BSM Descriptor concept maps very nicely to the TBSM Design Patterns that I’m also currently blogging about.

In an effort to collaborate on how to create such a BSM Descriptor and the ITM 6.x BSM Profile, DevCampTivoli has been created. The theme for this event is “Collaborative Development of End-to-End BSM Solutions”. The desired outcome is to come up various approaches for developing a BSM Descriptor File and BSM Profile for ITM 6.x, necessary configurations within the Tivoli EIF probe, Netcool/OMNIbus and TBSM 4.x that can be easily customized and implemented at any client. Whatever the DevCampTivoli produces will be freely available to anyone to take, modify and use to improve their BSM deployments.

Take a few minutes to visit DevCampTivoli. This event will be the May 17-18, 2008 which is the weekend before the annual IBM Tivoli User Conference Pulse 2008 in Orlando, FL. The thought and hope is that SME’s and practitioners in ITM, Netcool/OMNIbus and TBSM will already be coming to Pulse 2008 and will be able to come in a couple days earlier to participate.

More to follow…

I’m always skeptical by what I see in a demo until I can dig into what’s under the covers, but what I saw in the Integrien Alive demo impressed me. It looks like what could be a solid foundation for Business Service Management (BSM) in the future with focus by Integrien in key areas such as dashboard visualization, modeling and alignment to business services and applications.

It looks like Integrien competes firmly with Netuitive and the former ProactiveNet (now BMC), maybe Firescope and Managed Objects to some degree.

Effective, trusted and value oriented Business Service Management absolutely depends on an accurate data stream whether it be events, metrics, KPIs, etc. Taking the default out of the box configurations and thresholds with your monitoring tools and poor monitoring and event management lifecycles has led to the development of solution such as Integrien’s to “take back control” and give you back a trusted insight into IT infrastructure.

I’d love to see or hear more about Integrien technology. Anyone have any first hand experience? IMO, we have a gap in the IBM Tivoli portfolio in this technology and capability area.

One of the foundations of Business Service Management (BSM) is to see things from the business perspective. To get there, one of the best ways to do that is to instrument for BSM at the source. The majority of all server agent deployments for HW/OS monitoring and COTS application agents (DB, AppServer, etc.) are deployed in a pure out of the box (OOB) manner. Most clients will take what their vendor offers up within these OOB configurations as their own “best practices” and leave it at that. Some may take the time to modify thresholds in an effort to control the signal to noise ratio in the event console. This practice generally leads to a “needle in a haystack” approach for event management and a bad reputation for the tool in use for monitoring.

There will always be a need for the “best practice” OOB monitoring offered by vendors in their solutions. The primary audience is the NOC, EOC and support organizations. They speak this language and have been programmed to know what to do when the “95% CPU Utilization” event comes in. The BSM Profile concept is to help organizations move beyond this to focus how these lower level things may impact the business services, applications and activities that help the business meet its goals and objectives.

I want to see things change. I want to see the ability to configure monitoring with a purpose that’s above and beyond the OOB configurations. The establishment of a BSM Profile for managing the server HW/OS and deployed applications enables visibility into what that server or application really exists for - supporting the business.

The ideal BSM Profile for ITM 6.x would include the following:

1. Custom instrumentation into very specific business service, application and transactions ALIGNED to the LoB, services, applications and key business activities that they enable (and impact).
2. Custom free form text fields that enable creation of a specific and unique message relative to the above.
3. Mapping into custom event fields/slots of the ideal BSM Event Format.
4. Generate purpose built events, KPI/KPM or other data for the purposes of driving a service model or dashboard

In an effort to collaborate on how to create such a BSM Profile, DevCampTivoli has been created. The theme for this event is “Collaborative Development of End-to-End BSM Solutions”. The desired outcome is to come up various approaches for developing a BSM Profile for ITM 6.x, necessary configurations within the Tivoli EIF probe, Netcool/OMNIbus and TBSM 4.x that can be easily customized and implemented at any client. Whatever the DevCampTivoli produces will be freely available to anyone to take, modify and use to improve their BSM deployments.

Take a few minutes to visit DevCampTivoli. This event will be the May 17-18, 2008 which is the weekend before the annual IBM Tivoli User Conference Pulse 2008 in Orlando, FL. The thought and hope is that SME’s and practitioners in ITM, Netcool/OMNIbus and TBSM will already be coming to Pulse 2008 and will be able to come in a couple days earlier to participate.

More to follow…

The purpose of this TBSM Design Pattern is to create models representative of the key relationships and dependencies within COTS and Custom Composite applications. The implementation of this design pattern should provide a foundation for visibility into the applications by operations support teams, SMEs, developers and IT management. The level of detail expected in models of this type is generally abstracted from executives and the line of business in favor of higher level model design patterns.

One of the “holy grails” of Business Service Management is to have an understanding of the cause and effect relationships between the IT environment, the business services and applications provided and their actual use and/or performance to meet the businesses goals and objectives. Clients are beginning to deploy synthetic and real user monitoring solutions to help provide this insight. IT and the LoB are reporting weekly how well they’re performing and meeting business goals and objectives. This design pattern lays a foundation for being able to relate fine grained IT components to business service and application user experience and performance and the understanding of how one may impact the other. Without this foundation, we will continue to force operations and support teams to seek “the needle in the haystack” when the business reports problems within key business services and applications.

The first goal of modeling within this design pattern is to understand the architectural and implementation characteristics that applications, databases and custom composite applications can take. This is the nuts and bolts; the model that represents deployment onto the servers, the networks and into the datacenters. It’s how load balancers, virtualization, redundancy, clustering and high availability come into play in supporting applications, databases and custom composite applications. For example, an Oracle deployment may be in an Active-Passive, Active-Active (Cluster) or Oracle Grid architecture, deployed on one or more servers within one or more datacenters. It’s one thing to model within this area, but most clients struggle with knowing the state or status of the architecture. Clients must develop an approach for understanding what architecture components are active, online, in standby, load sharing, etc. through improved instrumentation.

The second and most important goal is to continue to improve the level of visibility and understanding of how these components operate, perform and ultimately their role in supporting business services and applications. How is the business service impacted when an application or database has a problem? What happens when one member of the database cluster fails? This doesn’t come from your vendor but from your SMEs, support groups, developers, and the business. The path to this point may be achieved with a simple model or require a fairly sophisticated model depending on your environment. Follow on TBSM design patterns will focus on interacting within this area.

The first challenge a client often has is that their monitoring tools group have not matured their fundamental management and monitoring to a point of having enough visibility into these areas. Clients continue to battle with keeping up with the basics of hardware and operating system changes, new versions, etc. Most clients have invested in some fundamental hardware and operating system monitoring capability or are making use of custom scripts, system logging or similar. They may have set up some basic logfile or process/daemon monitoring, but not yet invested into capabilities that enable deep visibility into applications. More often than not, it’s the SME groups (SysAdmins, Application Support or DBAs) that have the best visibility into these areas. Do not accept “they have their tools and we have our tools” as the answer. There is tremendous value in integrating SME tooling, scripts, etc. into the collective management and monitoring environment, even if it’s only for the purposes of driving these models.

More and more, COTS applications and databases include some capability for internal instrumentation and visibility into state, status, availability and performance characteristics. This is usually enabled through some additional configuration option, module, via an administrative console or a complete stand alone application (Oracle Enterprise Manager). Third party vendors and open source solutions may exist for management and monitoring of these applications or databases. Think agents for things like Exchange, SAP, DB2 or Oracle (IBM Tivoli, Quest Software, etc.) here or specialty applications that tend to fall into the advanced diagnostics and performance tuning areas (Quest Spotlight on Oracle).

Custom composite applications are going to expand the usage of COTS technology to include custom software, application servers, integration techniques (SOA, EAI, etc.), and other distributed and mainframe systems. The same approaches described above should be followed, with more emphasis on working with the various development and support SMEs.

Creating an accurate model is the easy part, bringing it to life requires data points, metrics or other information that can be used to determine state, status, performance, availability and impact. Modeling requires working with the various SME groups to fully understand what has been deployed, its operating characteristics and how it supports the business services and applications.

Once a good understanding has been achieved, the monitoring tools group will need to perform a gap analysis on what they have and don’t have to be able to represent the model in a production TBSM deployment. Do yourselves a favor here and partner up with the SME’s. Work through the politics and find a way to integrate their information and insight into the core management, monitoring and event collection solutions. Identify the gaps you have in your core tools and visibility, identify a plan to fill those gaps so your BSM solutions can provide the expected value.

I generally see a couple different approaches for building out these Architectural Models and they tend to play into the two scenarios above. Most often due to the weaknesses in fundamental instrumentation and monitoring, clients will simply create a template for the server and a template for the COTS application. Each template will generally include the expected “up/down” type status rules looking for incoming events. The COTS application template may have include a few more rules to cover more specifics about the application, but more often than not they are broad based status rules looking for process, daemon and log file messages.

There’s nothing wrong with this approach at all, but it’s a matter of getting to the fine grained visibility that’s difficult with this approach. Do you need to know that a specific transaction, process or table space out of dozens or hundreds is the cause and that it’s only impacting a portion of your business service or is knowing that you have an application or database problem good enough? My operations background says that it’s always best to help folks get to the specific problem as quickly as possible and to not generalize things.

The preferred approach is to break things out into more specific components of the architecture deployed. This approach DEPENDS on instrumentation and visibility into all of these areas. This approach DEPENDS on your ability to generate unique events, metrics, KPIs, etc. that can be directly associated to these areas. This approach gives you the most flexibility to tie discreet components into the specific business service and application areas that thy may most directly support, enable or impact.

Focus on simple templates for the COTS application/database or custom composite application PLUS simple templates for the supporting infrastructure server(s), application/database core components and marry them together based on the behaviors of the application as it relates to the underlying infrastructure components. This is the key here. This is the only way to manage each uniquely and as a whole based on how the overall architecture was designed and implemented. If you can get this right, you will be able to achieve the powerful business service and application models needed for Business Service Management plus the ability to put the right information into the hands of the operations and support staff so they can identify and resolve the problems faster.

Upcoming TBSM Design Patterns will focus on the service, functional/sub-service, and process/transactional design patterns.

**NOTE to Readers: Would examples of these within TBSM or Visio be helpful?**

Business Service Management (BSM) requires some level of visibility and insight into the core networks, systems, applications, transactions and processes happening across the IT environment. This visibility and insight requires some contextual understanding of how those things support and enable the key business services, applications, transactions, processes and activities that are critical to the business meeting their goals and objectives. The more emphasis on this contextual understanding we can establish directly from the source systems and applications, the easier and more efficient that operations, event and business service management can be in upstream solutions.

My findings and overall assumption is that most fundamental Tivoli monitoring is implemented in such a way that it’s only enabling the SME groups (SysAdmins, EOC/NOC, etc.) to identify, triage and resolve low level problems. I posed a series of questions to the only two Tivoli Monitoring gurus that I know about to try and gauge what could be done to better equip Tivoli Monitoring clients to implement fundamental system HW/OS and application/database monitoring so that it enables a client to implement true BSM solutions upstream. My intent from this dialog was to start a new series of blog postings called “The Top 10 Things and ITM Client Can/Should Do to Enable BSM and How to Implement Them”.

John “The Uber ITM Guru” Willis bit and we had breakfast to discuss. John’s got a lot of great ideas from what can and should be done from the ITM perspective. He mentioned a few of his clients that really get it and what they’ve done in the past to get there. We talked about the realities of client deployments today, politics, keeping up with constant changes and releases in the products and IT environment. Apparently the game really changed from ITM 5.x to ITM 6.x and things really need to be thought of in a different way making use of the Universal Agent. John’s answers continued to amaze me because of the level of effort it sounded like to do something as simple as this.

I kept coming back to a couple simple scenarios:

• How can I get something as simple as the the operating system name/version, server location, datacenter rack/row embedded in every event coming from an ITM agent?
• How can I get the business service/application that this server/application/database supports embedded in every event coming from an ITM agent?

We’d been discussing collaboration in the community via wiki’s, blogs, mailing lists, etc. for some time now. We landed on the idea of a scenarios based collaboration event focused on how one could solve real world problems using Tivoli products within the Business Service Management space. Something straight from the experts and practitioners out there. Something that shows what can/should be done from end-to-end using ITM 6.2 (and its dependencies) and TBSM 4.1.1 (and its dependencies) to create real world BSM solutions that any Tivoli client could implement.

Introducing DevCamp Tivoli. Our thoughts are that we’d meet before the annual Tivoli Technical User Conference (TTUC) called IBM Pulse next year. The conference next year is planned for May 18-22 so we’re targeting having this DevCampTivoli on Saturday May 17th. We’re betting on SME’s and Practitioners being able to fly in early for the conference they may already be attending and being able to participate in this event. Whatever the outcome of the DevCampTivoli is, we’d like to present that during a BoF session during the conference and on the OPAL site for everyone’s benefit. Listen to my first podcast ever on this topic with John here. Read over John’s blog posting announcing the event here. Visit the DevCampTivoli website and sign up!

More to come on this as we noodle through the concepts. Visit the site and sign up if you’d like to help out. We’re certainly interested in your input towards scenarios and development approaches within ITM 6.2, Tivoli EIF Probe, Netcool/OMNIbus and TBSM 4.1.1.