Event detection and event stream processing is a requirement in most, perhaps all, environmental observing systems. The real-time detection and response to complex patterns provides the situational awareness necessary for many applications, from data QA/QC to system state-of-health monitoring to adaptive control and manual intervention. We integrated Esper an open-source Event Stream Processing (ESP) software with the OSDT middleware to provide a principled and scalable solution to address this requirement. More information about Esper can be found at: esper.codehaus.org

Many DataTurbine applications have the need for event-triggered data collection and processing.
Researchers need to monitor fast moving data streams, quickly seek out patterns of significance, and respond with actionable results. For example, detecting and responding in real-time to events such as animal death, fire detection, and, in general, adaptive control based on observed patterns in the data signals. Often there is a requirement to move the control/decision making/intelligence closer to the sensors/dataloggers, since it is often impractical to stream continuously due to bandwidth or power limitations. To address the requirement of deploying DataTurbine in an embedded environment in conjunction with event detection software, we developed a prototype that directly interfaces DataTubine with Esper.

At the same time, there is a need for detecting and responding to events in a data center that typically pulls in tens of thousands of sensor streams. Data center also includes integrating data acquisition system with databases, visualization softwares, and modeling applications. Integrating these diverse applications in a scalable and principled manner is a challenge. To that end, an Enerprise Service Bus (ESB) provides an modular and standards-based integration environment that spans well beyond the reach of tradition hub-and-spoke architecture and provides a clean separation of application logic and integration services and enables agile SOA-based service development, deployment, and testing. GlassFish Enterprise Service Bus (ESB) is a lightweight and agile ESB platform with in-built functionality for clustering, load balancing and high availability for mission critical enterprise deployments. GlassFish is open source with an option for commercial support from Sun. More information about GlassFish can be found at: (http://docs.sun.com/app/docs/prod/gf.entsvc.bus). To address the requirements in a data center environment, we have integrated DataTurbine with GlassFish ESB. This involved interfacing a DataTurbine sink with a BPEL workflow Web Service that was deployed on GlassFish ESB. The workflow included Esper for real-time event stream processing.

Both the prototypes are being tested on real-time sensor data from the GLEON NTL site and CREON MCR LTER site. The integration was relatively straightforward and we are pleased with the results. We have plans to test the system for NSF-funded MoveBank community. Many thanks to our GLEON and CREON community on their help with testbed set up and sharing their science requirements!