Category — metrics

This is a good read from CIO magazine. The last few paragraphs sum it up nicely. Keep these same concepts in mind as you begin your business service management (BSM) journey.

–snip–

“IT executives aren’t good at describing IT work in business terms,” says Forrester’s Orlov. “They describe it in terms of technologies they are supporting. IT spending should be described in terms of growing revenue, lowering cost and improving the time it takes to do something. If all you talk about is uptime, you are a cost center, not a strategic partner.”

“In the end, the real metric is not IT, it is business performance,” says Dow’s Kepler. “What is the output per employee? How efficient are we as a business per employee?

“If you’re looking for a metric to justify IT spending, that’s not the right mind-set,” he adds. “The right mind-set is to understand how processes, systems and people tie together to get business results.”

While I was at our Raleigh Executive Briefing Center (EBC) this week, I saw my first glimpse into a recent acquisition of the company CIMS Lab. The technology is now rolled into a product called ITUAM and aims at really enabling the concept of charge backs to come to life. Here is the IBM page for ITUAM.

I think this picture sums the concept up quite nicely.

In my last job, management spent countless hours trying to manually do this. We built a time tracking tool (that just made everyone happy) to try and account for time spent doing various tasks, projects, maintenance, etc. which was then correlated with asset information from the IT perspective (in another home grown tool), which was correlated with Finance asset information from Great Plains, which was all mashed togther with their secret sauce in some BASS (big a#$ spreadsheet).

If this ITUAM product really simplifies an organizations goal to understand costs associated with a business process, activity, service, application, datacenter, type of hardware, software, network, people, etc. or move towards a charge back environment then it’s sure to be another winning product in the Tivoli portfolio. From what I saw from our dynamic speaker (”Mr. Serverguy”), the approach and technology makes sense to me assuming you can get access to all of the datasources you’d need to round out the costing model you choose to implement.

One key point that stuck with me from my various ITSM/ITIL training was that even if you aren’t moving real money there is real power in a zero dollar invoice delivered to your internal customer, line of business, etc. This can start to have a real mental impact on how they view the IT services they’re provided and help them associate a dollar value to what they may be taking for granted.

Understanding the costs to deliver a service, process, function can become an important metric in many other ITSM areas. I see this technology underpinning many other areas we talk about and providing a powerful tool during budget planning and contract renegotiation time with vendors, suppliers, etc.

The folks over at RSSBus have taken a few more covers off of thier RSSBus product. I’ve played around with the RSSBus Desktop Server some now and continue to believe in the potential it has in many of the areas I write about in this blog, especially enabling the “average person” to publish events for consumption by business rules, event and visualization solutions. I’m very excited about the ability to suck metrics, kpi/kpm, etc. out of all those BASS out there!

Check out RSSBus here and download their public beta here. Read the whitepaper, it’s really good.

They’re keeping a blog here where you can follow the product’s progress. I’ve had private email exhanges with their CEO Gent Hito about their plans for the product. They plan to keep parts of it free and are considering open source for parts. Reach out and encourage them to consider this!

I’ve taken you through the trenches of the organization and IT environment to find and capture what’s important to your audience. If you need to catch up, don your safari hat, some boots and check out this page. The next part in this series is one of my favorite areas and probably the most important. This is the part where you’ll show off the fruits of your labor, where the rubber meets the road in terms of how valuable your work and solutions will be for the business and your audiences. This is also the part that everything you do can come into question, be challenged, or simply blown off as garbage, eye candy or a waste of time and money.

Visualization of data and information is an art in itself. There have been many books written on the subject. See the Dashboards page for a list of references. Our goal here is simple. Take everything you’ve done to this point and present the message in the most meaningful, efficient and effective way possible for your audience’s consumption. Your challenge is to figure out what works best for your audience and to ensure that the message can be consumed and have the desired effects of prompting action, decision making, etc.

Iteration is key in getting the visualization right. Allow for a considerable amount of time in your project plans for work in this area. I strongly recommend mockups and prototyping in Microsoft PowerPoint, Excel, Visio or your favorite graphics program such as Adobe Illustrator, Photoshop or Fireworks. A tip here is to look at the dashboard references or vendor products/presentations and cut/copy/paste the widgets (dials, gauges, charts, etc.) into your mockups and prototypes. It’ll help to be as close to what your capabilities are as you review with your various audiences. Keep on this task until you get buy in and a sense that this will work for them. Seek the 80% rule here.

I encourage you to ask your vendor for examples of successful dashboard deployments. See if you can speak to referencable customers and then really dive in with them about how they’ve visualized messages within their environments. A new blog is available that has been collecting examples of dashboards and visualizations called Dashboard Spy. I encourage you to take a look at what has been captured here for ideas.

There are references and links on the Dashboards page that will help you with all the right and wrong uses of gauges, stoplights, sliders, stoplights, charts, etc. I won’t go into those because I don’t necessarily have an opinion one way or another on what they are saying. I do know that every audience will be different. There will likely not be a one size fits all representation of your message. You may be able to get it to look similar, but I promise there will be someone who prefers a gauge or dial instead of stoplight or chart.

Once you’ve got the mocked up visualization of your message complete, it’s time to start implementing it within your solution. I’ll defer from speaking in detail on how to do this, but we’ve previously talked about how to generate events on what’s important and the message. Your solution should offer an easy way to extract this information from an event, database, or any other datasource for alignment and mapping into the visualization components that you will be using. It should be capable of processing these events, metrics, etc. in large quantities in real-time from a large number of distinct sources. You should be able to apply analytical logic, rules, calculations and statistical evaluations, timers, counters, etc. to any single piece of data or any group or collection of data. I’d be happy to recommend a very flexible solution for accomplishing this within your environment off-line.

Release your visualization into a controlled production environment and let it run over the course of the normal cycles associated with your message, what’s important and your data sources. Make sure you’ve also built up enough of the contextual references that may be needed. If you feel the visualization is at a point where it closely resembles your mockups using your solution and real data, it’s time to review and level set with your various audiences.

Get their feedback. Show them how it will work in production. Ask them if they “get” the message. Can they tell you what needs to be done or the state of the business? Will it work? Do they believe in it? Stand behind it? Iterate here until you get to this point. Go back and review everything you’ve done up to this point - discovery interviews, audience needs, what’s important, the message and make sure you’ve got everything covered.

When you’re 100% there, release into your production environment and place under your normal change, configuration and document controls. Establish a monthly or quarterly (at a minimum) review cycle to sit back down with the various audiences and review the solution with them. Talk with them. See how they use it. Capture metrics associated with any improvements, value, savings, etc. that can be attributed to the solution. Get them to vouch for these accomplishments. Don’t walk away from this review meeting without knowing what’s working, not working or needs to be changed or updated. There is nothing worse than a solution that’s not used or is ignored because it’s out of date or providing no value.

Here are some general guidelines I’ve picked up over the years will help ensure your success. Check the Dashboards page for more ideas and starting points. There are a lot of really good things out there from the BI, BPM and analytics folks!

Consumability

• The message (what’s important) should be communicated in seven (7) seconds or less (one page/screen of information)
• Choose 3-5 key messages, themes or topics to communicate for each audience or each level (see “Determining Your Audience” and “Determining Your Message“)
• Keep things aggregated, correlated and presented in summarized views that prompt action
  (Is the ship on the right path? Do I need to take some action to steer around the iceberg? , How quickly do I need to take action?)
• Try to convey a sense of movement or flow in a uniform manner for visualizations that represent activity, processes, workflow, transactions, etc. Keep them top to bottom or left to right as much as possible.
• Try to draw your audience’s eyes to the most important parts of the message. Don’t let these get lost on the page.
• Think Web2.0 - white space, rounded corners, smoothness, etc.
• Don’t use wild color schemes. Avoid eye candy, all black backgrounds, etc. for more executive and non-technical audiences.

Freshness

• Ideally one hour updates or more frequently
• No more than one week’s data points on a dashboard (just enough to have some context on what’s happening)
• Goal is to manage where the business is going “real-time” (using GPS) versus where we were yesterday (looking in rear view mirror)

Provide Context Where Relevant

• Historical view/info providing as needed context for making decisions
• Aggregate daily, weekly snapshot
• Provide comparables - Hour of day compared to hour of day, Day of week compared to day of week, Week in month compared to week in month
• Link out to or reference other sources that may provide context - avoid replicating data

Ease of Use

• Should support drill down from any click-point or metric and maintain context through every click through
• A common display panel is desirable for maintaining context. Clicking on a metric or indicator causes the results to be displayed in the common window.
• Double clicking or right clicking would cause drill down
• Hover displays are also useful for this approach, but not for key metric or indicator display (you want those immediately visible)
• -“Breadcrumbs” should be used to help understand where in the navigation drilldown someone is and how they can get back to upper layers

Organizational Politics

• Anticipate questions that may be asked
• Have your ducks in a row - what’s important and why, your metrics catalog, source quality, etc.
• Avoid overlap and “competition” with data warehouse, business intelligence/analysis or enterprise reporting groups
• If results make an organization, department, group or person look bad, seek them out in advance to review and prepare them as needed

Stay tuned for my next planned topic in the “You’ve Got Events, Now What? series where I’ll focus on “Managing in Real-Time”.

Catch up with the “You’ve Got Events, Now What?” series here.

As I’m working on my next part in the You’ve Got Events, Now What? series, I’ve created a reference page on Dashboards and added some more useful links on the Metrics page.

I encourage you to check out this great blog on dashboards - http://dashboardspy.wordpress.com

I’d love to hear about other references you’ve got!

Funny how this blogging stuff works. The minute you post something, soon after I usually find something similar or something that enhances or detracts from what I was writing about. Fortunately, this one may greatly enhance my post!

I talked about having an arsenal full of instrumentation, data and information collecting tools in yesterday’s posting YGE, NW? Part IV: Mapping Events to What’s Important and Your Message. I mentioned using the normal NMS/EMS/OSS/BSS tools, logfiles, scripts, database triggers and stored procedures, etc. to help collect metrics and KPI/KPM and turn them into events for processing upstream.

I came across another potentially useful approach that may make this instrumentation and collection process significantly easier in the future by using Really Simple Synidcation (RSS) to create a service bus (not ESB). Their goals is to accomplish what has been reserved in the past large companies with large IT staff and large IT budgets - easy integration and sharing of data between applications, services, etc.

The company, RSSBus, is in pre-release mode still and has a white paper available discussing their approach aimed at greatly simplifying integration, access and sharing of information.

I think this has great potential for enabling “the rest of us” to instrument the business and use that important data and information to create rich dashboards and portals and maybe even powerful BSM/BAM/BPM implementations. Imagine subscribing to dashboard feeds, business activity monitoring feeds, etc. Something like Pageflakes could become the enterprise BSM dashboard portal fed by numerous business, technology, people, process and operations feeds. Could this be the start to Web2.0 solutions in these areas?

Some highlights from the whitepaper:

“With RSSBus, our goal is to offer a simple, easy alternative for the small organization with little to no IT assets, little to no professional development tools, and no professional programmers to use them.”

“What we are building is something different, a service platform for the rest of us, the nonacronym-speaking crowd. If you have bits of pieces of data that you would like to quickly exchange with and/or connect to other systems, if simplicity and ease of use is your most important consideration, please read on.”

“With RSSBus, our goal is to build general purpose software that connects or has the ability to easily connect to every system, data, or information source of any significance. Our core focus is to enable connectivity as simply and as easily as possible, and we believe our experience building networking software components and connectivity toolkits for the past decade, and the software assets we have created in the process, give us a unique advantage.”

I’m keeping these guys on my radar to see how their ideas and products develop. No indications as to availability, costs (open source?), etc. yet.

Now that you’ve identified the sources of what’s important within your environment and crafted that data and information into messages that prompt action and decision making, it’s time to think about getting this data and information into a manageable format for processing and visualization.

I’ve discussed what events are and shared some initial thoughts on building events for BSM here and included references to complex event processing (CEP), event driven architecture (EDA) and event stream processing (ESP) here. I still plan on diving in deep to the topic of building events and the idea of the Common Base Event and Common Event Format. I also want to introduce the Event Data Dictionary / Event Catalog which will be useful for capturing information about what events exist in your environment and why. Every event that’s generated should be done so for a purpose. There’s nothing that will turn your NOC or IT support group against you quicker than if you’re collecting data and generating events just for the sake of doing so because they may be available via and SNMP MIB or agent. They don’t need any more “noise” to deal with during the day.

There may be many ways to incorporate this data and information into the messages you’re planning to communicate. The approaches and their ease of use are going to be entirely up to the tools, applications and solutions you’re using. You may be able to establish direct connections with the datasource, perform screen scrapes, import spreadsheets, or even perform queries against the source. The general concept of this series of articles has been around the assumption that you have the ability within your environment to generate events. Generating events usually comes through some form of instrumentation, collection and evaluation against a threshold, state, rule, etc.

What I want to talk about here is instrumenting those sources of important information, data and metrics within your environment you’ve identified as you completed your Metrics Catalog. Some of these sources may be outside the comfort zone or capabilities of the average IT Operations group normally used to operating with SNMP, server and application monitoring agents.

Since you’ve identified the source of the important information or data, how frequently it gets updated, and how to access it you’re half way there. The next task is to identify the person(s) or group(s) responsible for that information source. This may be the owner, administrator or support group for the application, tool, file, spreadsheet, database, server, etc. that produces, evaluates, communicates or makes available that information or data. The task here is to establish the business need with the owner to instrument that source so that the important data or information is provided in a way that can be easily processed upstream.

Once you’ve established the business need, you can have a discussion about the best way to instrument the information source and generate those events. Discuss the various tools in your event generating arsenal with the owner and their technical staff. Cover the normal EMS/NMS/OSS/BSS solutions and their capabilities for collecting information and generating events. Discuss more generic approaches such as log files (application, system, etc.), scripts, XML/SOAP/WebServices, etc. Scripts can be written to parse logs or collect other information from applications, GUIs, command lines, etc. and pass those off to an event generation function. If you’ve been able to consolidate information into a database or corporate data warehouse, consider leveraging database triggers and stored procedures to collect, format and generate an event. There are certainly more sophisticated methods available here if your organization leverages an EAI or ESB technology. Just keep in mind that the goal is to keep it simple, efficient and effective. You don’t want to be blamed for causing a performance slowdown or outage to that important business application!

You’ll want to map the events you’re generating into the appropriate format of your internal systems that will process them. Be sure to capture the relationship between these event types and their purpose for communicating an important metric, KPI/KPM, etc. At a minimum, one of the fields in the event format should be the Metric ID from the Metrics Catalog. This will be critical in linking the events to their purpose. The more thought and planning you put into how you build these events the better. Consider the use of an enumeration schema to capture information. This can be parsed and evaluated later by other solutions such as dashboard, BSM, BAM, BPM, rules or workflow solutions. An example may be populating a field in an event like this: “A1-2-3″ which may represent Metric Source = A1 (CRM System), Metric ID = 2 (Customer Count) and Metric Update = 3 (Daily). The sky’s the limit here but do consider the impact these may have on your internal event processing solutions or those that will need to parse and evaluate the enumeration schema you create.

Spend some time testing and evaluating the effectiveness of the new instrumentation you’ve done. Follow up with the owners you identified and the business to make sure that the data, information, metrics, etc. you’re now collecting passes their “sniff tests”. They’ll have a fairly good understanding of what’s good or bad - they always seem to have a sixth sense about this. If you get the sense that this information isn’t accurate, useful or otherwise have them excited, immediately start to evaluate why and do whatever you can to remedy it. You absolutely do not want to be presenting bad information later!

Now that we’ve got these important bits of data, information, metrics, etc. being collected and processed by our internal systems and tools automatically, it’s time to think about visualizing our message effectively for our various audiences. Stay tuned for that topic in “You’ve Got Events, Now What? Part V: Visualizing the Message.

Catch up with the “You’ve Got Events, Now What?” series here.