RSS
RTI Event Processing FAQ
What is Complex Event Processing?
What are the benefits of using CEP?
What is included with RTI's CEP Solution?
How do I write CEP applications?
How does RTI's CEP solution address high-performance requirements?
Why are CEP applications easier to develop and debug than those written in a procedural language?
What is Complex Event Processing?
Complex Event Processing, or CEP, is the computational complement to RTI's real-time messaging and caching solution. CEP processes continuously, with low latency and high speed, streaming real-time data.
A messaging and caching service, like RTI's Data Distribution Service, distributes, in real-time, data in the form of messages. Often these messages represent events, such as a stock trade (e.g. 1,000 shares of Google at $424.80 at 13:08h EST). But many interesting events don't correspond to a single message. For instance, the event “Fifth SSH probe from IP address aaa.bbb.ccc.ddd in the last 10 seconds” is not a single event, but a derived event. It must be synthesized from potentially hundreds of messages using complex rules. A notion of time is needed (last 10 seconds), a port number (22) must be looked up in a database to return the port name (SSH), and an IP address must be pattern matched across multiple messages.
Complex event processing software makes it possible to run tens or hundreds of thousands of complex calculations on streams of events to generate interesting events like the SSH probe example given above. Much more complicated processing like dynamically changing queries, pattern matching state machines or hierarchical events are also possible.
Multi-Sensor Data Fusion
US Navy ships need to provide timely classification and identification of approaching targets to determine if they are friend or foe. Raw data comes in from a variety of sensors and processors located in different parts of the ship. This data must be normalized and the sensor's different views of the world (radar, sonar, infrared, etc.) compiled into one integrated view that can be used to compare a target's actual behavior against the expected behavior of that class of objects.
RTI's messaging and caching solution can provide the data distribution service that takes the differently formatted sensor data and deliver it to a CEP server for processing.
Financial Services
CEP can simplify the development, deployment and maintenance of high performance trading systems. The low latency, high throughput and rich functionality of CEP can be used:
- in algorithmic trading models and to determine arbitrage opportunities
- to aggregate order book quotes and to determine the best bid/offer
- to cleanse or filter data received from a variety of sources before committing to storage
- to calculate derived values for pre-trade analysis and risk management
When combined with a messaging and caching solution, CEP can perform complex calculations on messages derived from many sources and produce results within milliseconds.
Intrusion Detection
Operators of large computer networks are naturally very concerned with preventing unauthorized access to their networks. But the latest techniques for gaining unauthorized access to a computer involve hiding the origin of an attack by using multiple compromised machines that cooperate in an attack on a single computer. Since looking at the network traffic from just one machine doesn't reveal a pattern, one must correlate the data from multiple sites to determine if an attack is taking place.
CEP can perform in real-time the pattern matching needed to determine that unauthorized access is taking place. CEP is essential because the patterns being searched for are spread out over many streams and over time. A real-time data distribution system is needed because large volumes of data are generated by nodes reporting on their network traffic. RTI's messaging and caching architecture has no single point of failure, scales extremely well, and has high throughput and low latency.
What are the benefits of using CEP?
RTI's complex event processing solution offers a complete development environment specialized for designing and deploying event processing applications, along with a runtime environment that can be paired with RTI's messaging and caching solution to create high performance real-time, distributed messaging and event processing applications. Using RTI's CEP reduces the time and effort needed to develop event processing programs, increases the sophistication and capabilities of those programs, and produces higher quality results.
RTI's complex event processing solution, called RTI Event Processing, is powered by Coral8, an RTI partner and a leading supplier of CEP. It is fully and seamlessly integrated with RTI's messaging and caching solution, the RTI Data Distribution Service.
What is included with RTI's CEP Solution?
RTI's CEP solution includes both development and runtime components. For developers, RTI Event Processing consists of Studio, a graphical environment for developing, testing and deploying RTI CEP components and modules. Studio offers full CCL (see below) compilation, debugging and monitoring capabilities.
RTI's CEP uses the SQL-based Continuous Computation Language (CCL) as its programming language to specify the calculations to be performed on events. In contrast to SQL which runs a series of queries on (relatively) static data, CCL runs (relatively) static queries against streams of data.
SQL has been extended to support windows and aggregation primitives, correlation (joins), various kinds of pattern matching, and database subqueries. A SDK allows developers to supply user-defined functions which can then be included in CCL.
For runtime, RTI's CEP includes a query processor and a wide variety of standard adapters. Packaged (I/O) adapters enable rapid integration with a variety of event data streams and sources like relational databases, message queuing middleware, email, and RTI's data distribution and caching solution. Protocol adapters add support for socket, SOAP, SNMP and standard languages like C/C++, Java, .NET, perl and python.
How do I write CEP applications?
CEP applications are developed using Studio, the graphical CEP development environment that comes with RTI's CEP solution. With Studio, developers can create, compile, debug and test CCL programs. They can specify the data streams they want to operate on and the adapters they need to convert data to and from the stream's format into a format usable by the CEP processing engine. Developers can then deploy their application and can monitor the performance and resource consumption of their processing.
How does RTI's CEP solution address high-performance requirements?
The Coral8 query processor, which powers RTI CEP solution, implements a server-based lightweight streaming architecture for highly parallel execution of continuous queries on high-speed data. It optimizes its performance by smart caching and asynchronous reads & writes. In addition, its highly tuned optimizer and scheduler takes into account many static and dynamic conditions in making decisions.
Using simple filter queries, RTI's CEP has been benchmarked at over
a million events per second, and with complex queries, at thousands
of events per second.
Why are CEP applications easier to develop and debug than those written in a procedural language?
CEP applications are easier to develop, debug and deploy than applications written in a procedural language like C or C++ because developers deal with the computational aspects of event stream processing at a conceptually higher level than programs written in a procedural language such as C or C++. RTI Event Processing uses a declarative, SQL-based language called Continuous Computation Language (CCL). CCL's higher level of abstraction hides non-essential bookkeeping details that are exposed in a procedural language like C or C++. Just as it no longer makes sense to develop programs in assembly language because compiled languages are so much more powerful, it doesn't make sense to use a lower level language like C or C++ when a language like CCL is available.
How do I get more information?
For more information, or to evaluate RTI's Complex Event Processing solution, please contact your RTI account manager, or email info@rti.com, or call us at 408-990-7400.