Douglas “Dougie” Stevenson has initiated a Sourceforge project focused on developing a high performance, ultra scalable polling, correlation and event management engine. If you don’t know Dougie, he’s got a tremendous history and knowledge in this space. He’s built some powerful tools like this before and I’m sure he’ll do wonders with this new project. He’s the ultimate geek, coder, hacker, bits and bytes twiddler that is driven by taking the concepts and ideas he’s laid out below to places nobody has imagined they could go before.
I encourage you to check out the project page and get in touch with Dougie if you’d like to participate. It’d be great to see a modular approach taken here where the efforts of the many other great OSS projects can all come together to form that ecosphere I’ve mentioned before in the blog. (something for inventory/discovery like ZipTie, visualizations/dashboards/wiki/mashups/knowledgebase/enrichment like ??, rules/workflow/bpel/bpms like Intalio, integration/esb/glue with Mule, etc.)
-snip-
In getting this project off the ground, I’m putting together the requirements, features, and use cases for the different functions of DICE.
I’m also soliciting input, thoughts, and ideas of what could be put into a World class Correlation Engine. So, if you want to contribute, I’m all Ears!
Within the basic requirements, the needs include:
- Scaling to handle greater than 1 million events a minute.
- Be able to dynamically add and subtract handler components on the fly.
- Be able to accomodate a single Control port to the entire distributed application such that any component can be controlled, queried, and monitored via this control port.
- Be able to take raw data feeds from Syslog, various log files in differing file formats, SNMP Trap data, SOAP Services, and assign handlers and process these as a function of workflow.
- Be able to log and track event records throughout the process.
- Be able to enable administrative users to build and modify filtering, parsing, and processing rules as part of a web based build process.
- Be able to add, remove, or schedule changes related to built and tested filters, parsers, and processing functions without causing application downtime.
- Be able to display Objects and states via a Tabular type display.
- Be able to provide state information to an iconic, canvas based map sort of display. This map display ought to be exportable to Visio.
- Be able to enact state based polling via SNMP, TCP connections, and SSH.
- Be able to provide for an intelligent MIB Compiler and analysis function. MIBs should be loadable in any order and be able to be verified and tested against devices in the infrastructure. In effect, the MIB compiler function needs to be able to document differences between a published MIB Structure and real attributes from a given devices’s agent.
- Workflow and states mechanisms should be documented within the product in BPEL format as well as Excel Spreadsheet formats.
Comments on this entry are closed.