RSS
Whitepapers
A Data-Driven Approach to UAV Architecture (PDF)
Designing and building an unmanned autonomous vehicle (UAV) is one of the most difficult problems in engineering; and it is particularly challenging from a software systems perspective. By optimizing software performance, scalability, high availability and reliability, security, interoperability, and affordability, system designers can create a UAV that is adaptable to new mission parameters while remaining robust across the product lifecycle.
Towards a publish/subscribe control architecture
for precision assembly with the
Data Distribution Service (PDF)
This paper presents a comprehensive overview of the Data Distribution Service standard (DDS) and describes its benefits for developing robust precision assembly applications. DDS is a platform-independent standard released by the Object Management Group (OMG) for data-centric publish-subscribe systems. It allows decoupled applications to transfer information, regardless of what architecture, programming language or operating system they use. The standard is particularly designed for real-time systems that need to control timing and memory resources, have low latency and high robustness requirements. As such, DDS has the potential to provide the communication infrastructure for next generation precision assembly systems where a large number of independently controlled components need to communicate. To illustrate the benefits of DDS for precision assembly an example application is presented.
Applying Publish-Subscribe to Communications-on-the-Move Node Control (PDF)
J. Darby Mitchell, Marc L. Siegel, M. Curran N. Schiefelbein, and Armen P. Babikyan
Modern military satellite communications terminals have typically been
built as multiprocessor systems. Because of increasing pressure for reuse and
modularity, current programs have been encouraged to consider the use of
component middleware. While Common Object Request Broker Architecture is
the most mature middleware standard available, its invocation semantics present
considerable challenges for the development of such systems. Through reasoning
about quality attributes, we found that a real-time publish-subscribe middleware
reduces coupling, improves composability, and reduces the risk of architectural
mismatch, deadlock, and integration problems compared to an invocation based
system. In building a communications-on-the-move (COTM) node, we
found that this type of middleware, which exemplifies an implicit-invocation
architectural style, promotes ease of system evolution and an incremental
integration approach.
How Does Your Real-time Data Look? (PDF)
Are all real-time distributed applications supposed to be designed the same way? Is the design for a UAV-based
application the same as that of a command-and-control application?
In the case of a UAV-based application, a high volume of data gets created, only some of which is of interest to the base station. To preserve the radio link's bandwidth, only the relevant information is transmitted. The application will use the data for post-mission analysis, so it also has persistence and data-mining needs. In contrast, a real-time command-and-control application needs low-latency and high-reliability, but has little need to persist or cleanse the data in real-time. No, all real-time distributed applications are not designed the same way. While we categorize both these applications as real-time with similar data-transmission characteristics, their architectures and designs vary significantly because the information that they manage and process varies significantly.
This paper characterizes the lifecycle of data in real-time applications — from creation to consumption. The paper covers questions that architects should ask about data management — creation, transmission, validation, enrichment, and consumption; questions that will determine the foundation of their project.
RTI in Financial Services: Performance,
Consistency, Reliability (PDF)
Increases in data volumes over the past few years have stretched current financial information backbones, resetting the technology challenges in a way that demands a new approach.
For example, Automated
Trading Desk (atdesk.com), now part of Citigroup Inc., chose RTI to distribute real-time data from direct-exchange and ECN feeds to price-prediction engines and automated trading applications. PIMCO (pimco.com) has selected RTI as part of a new initiative to enforce and monitor regulatory and client-imposed pre-trade investment restrictions.
RTI’s solution is a software-only messaging backbone based on a true peer-to-peer architecture that fundamentally challenges earlier generations of daemon-based architectures, and recent peer-to-peer approaches. Leveraging 16+ years of research and development, RTI’s key technical value is unparalleled intelligent messaging that performs predictably as message sizes increase and consistently as overall message throughput grows under extreme market conditions. This deterministic behavior is unmatched.
With no intermediate daemons, brokers
or other components to add latency,
market data and market order/execution
updates are benchmarking at over
3 million messages per second.
RTI customers have reported mean
latencies as low as 43 microseconds
in Gigabit Ethernet environments.
Current tests support symbol spaces
of over 1,000,000 "subjects." Overall
throughput is essentially linear
as publishers are added, with maximum
throughput limited only by available
bandwidth.
Meeting Real-Time Requirements in Integrated Defense Systems (PDF)
Modern military operations rely on integration between many disparate systems for functions including command and control, weapons and self-defense. Developing and integrating these applications is particularly challenging because of their strict real-time performance constraints. Often, their latency and throughput requirements exceed the capabilities of traditional enterprise messaging and integration middleware.
This paper examines defense systems’ real-time
requirements and shows how they
can be met with commercial middleware
specifically designed to meet their
stringent performance requirements.
Implementing Net-Centric Tactical Warfare Systems (PDF)
The tactical battlefield has long been characterized by the use of many different data collection and analysis systems that present information on small and discrete areas of the conflict to separate command and control stations. The operators of these stations attempt to use that data to estimate enemy intentions and actions, and counter with manual direction of the equipment and personnel in a simulacrum of coordinated response.
The result is a disjointed and often extremely dynamic environment of forces operating across the battlefield; a variety of aircraft with different weaponry, performance, and flight characteristics, fixed and mobile artillery, shipboard combat and weapon systems, and dismounted soldiers all with unique pieces of data when aggregated represent the complete strategic picture of the battlefield operations. When these disparate systems are integrated, it is often with a particular mix and mission in mind.
A much discussed way to dramatically improve the speed-of-command on the battlefield is to create a net-centric battlefield operation. Each individual element of a tactical system performs its narrow mission, but shares data as needed with others in a way that provides a more complete and accurate representation of the battlefield environment and the role of that system in the environment.
The purpose of net-centric warfare is to translate an information advantage into a battlefield advantage through the comprehensive networking and dynamic data-sharing between geographically dispersed forces. The shared situational awareness enables better strategic coordination of forces and enhances speed-of-command, which dramatically increases mission effectiveness.
The Data-Centric Future (PDF)
Truly profound technologies become part of everyday life. Motors, plastics, computers, and now networking have made this transition in the last 100 years. These technologies are embedded in billions of devices; they have melted into the assumed background of the modern world.
Another stage is emerging in this progression: pervasive, real-time data. Like the Internet; except that this pervasive information infrastructure will connect devices, not people. Just as the "web" connected people and fundamentally changed how we all interact, pervasive data will connect devices and change how they interact.
Today's network technology makes it easy to connect nodes, but not so easy to find and access the information resident in networks of connected nodes. This is changing; we will soon gain the ability to pool information from many distributed sources, and access it at rates meaningful to physical processes. Many label this new capability the "network centric" architecture. However a more appropriate term being used is "data centric" because the change, fundamentally, is driven by the dynamic availability of information, not the static connectivity of the network itself. Whatever the name, this concept will drive the development of vast, distributed, information-critical applications.
The Data-Centric Future, Part II: Performance (PDF)
The Rise of Data-Centric Programming
The network is profoundly changing the nature of system design. The "web" is just a first step; the Internet today focuses on connecting people at human interaction speeds. Future networks will connect vast arrays of cooperating machines at rates meaningful to physical processes. These connections make truly distributed applications possible. Distributed applications will drive the future in many areas, from military information systems to financial trading to transportation.
Data-Oriented Architecture (PDF)
The growing popularity of cheap and widespread data collection “edge” devices
and the easy access to communication networks (both wired and wireless) is weaving in more devices and systems into the fabric of our daily lives. As computation and storage costs continue to drop faster than network costs, the trend is to move data and computation locally, using data distribution technology to move data between the nodes as and when needed. As a result, the quantity of data, the scale of its distribution and the complexity integration is growing at a rapid pace.
The demands on the next generation of distributed systems and systems-of systems include being able to support dynamically changing environments and configurations, being constantly available, and being instantly responsive, while integrating data across many platforms and disparate systems.
How does one systematically approach the design of such systems and systems-of- systems? What are the common unifying traits that can be exploited by architects to build systems that can integrate with other independently systems, and yet preserve the flexibility to evolve incrementally? How does one build systems that can be self aware and self-healing, and dynamically adapt to changes in their environment? Can this be done on the scale of the Internet, and yet be optimized for the best performance that can be supported by the underlying hardware and platform infrastructure? Can this be done without magnifying the ongoing operational and administrative costs?
These and related topics are the subject of this paper.
A Fully Standards-Based Approach to Logging High-Throughput
Distributed Real-Time Data: Leveraging the DDS and SQL Standards (PDF)
The ability to capture run-time activity for future analysis and playback is vital to being able to debug,
integrate, test, and support complex distributed systems. Unfortunately, implementing a logging capability
is often very difficult and expensive because data must be captured from disparate sources, likely consists
of many different native data structures, and often has a higher aggregate throughput (burst or steady-state)
than can be supported by traditional commercial databases or even hard disks in some cases. As a result,
distributed application developers traditionally have had to implement their own custom logging
capabilities and tools to go with them.
By integrating widely-used standards and high-performance database technology, a fully standards-based
and off-the-shelf approach to logging high-throughput distributed real-time data is now possible. This
solution uses the Object Management Group (OMG) Data Distribution Service (DDS) standard to distribute
real-time data to one or more instances of a Structured Query Language (SQL) database that provides
persistent storage. When used with a memory-optimized SQL database, optionally as a cache to a
traditional disk-centric database, throughput rates can be supported and sustained that are an order of
magnitude higher than most legacy databases support.
Embedded to Enterprise Application
Bridging (e2E) Utilizing DDS & RDBMS Technologies (PDF)
System architects and engineering teams are designing increasingly complex
embedded systems in order to satisfy their customers’ stringent
functionality and performance requirements. It is not uncommon for a
system’s architecture to require deterministic real-time behavior
while moving large quantities of information over non-deterministic
network transports. Many of these systems must now face the challenge
of interfacing or ”bridging” to the Global Information Grid
(GIG) in a seamless fashion in order to: a) communicate information
with the larger electronic community while b) not negatively impacting
high performance mission-critical functionality.
As the Object Management Group’s (OMG) Data Distribution Service
(DDS) specification
continues to gain market traction, particularly within the Department
of Defense (DoD), the ability to seamlessly bridge DDS based high-performance
tactical systems with Structured Query Language (SQL) and Relational
Database Management System (RDBMS) based enterprise applications provides
the potential for a viable architectural strategy that specifically
addresses the embedded to Enterprise (e2E) communication challenge.
This paper describes how, by leveraging both DDS and SQL (DDSQL), RTI
offers a powerful solution for DDS to RDBMS and RDBMS to DDS bridging.
A Comparison and
Mapping of Data Distribution Service (DDS) and Java Message Service
(JMS) (PDF)
Data-centric design is emerging as a key tenet for building advanced
data-critical distributed embedded and enterprise systems. DDS and JMS
are popular middleware API standards that are easy to use, and offer
the benefits of using a publish-subscribe communication model resulting
in loosely coupled scalable distributed applications. However, their
differences have significant impact on a data-centric design.
DDS and JMS are based on fundamentally different paradigms with respect
to data modeling, dataflow routing, discovery, and data typing; yet
they offer a similar and easy to use experience to the application programmer.
They differ significantly in their support for data filtering and transformation,
connectivity monitoring, redundancy and replication, and delivery effort.
Each also offers some distinct capabilities; and they both offer some
equivalent capabilities. We provide a detailed functional comparison
of the two standards, and discuss their implications on datacentric
design.
We also discuss the practical considerations and differences in using
the two standards. These include middleware architecture, platform support,
interoperability, transports, security, administration, performance,
scalability, real-time application specific support, and enterprise
application specific support.
DDS and JMS APIs may be used together in an application. The can leverage
each other via JMS-DDS bridging, JMS/DDS bindings, or by using DDS for
JMS discovery. We discuss these approaches and their suitability for
different data-centric integration scenarios.
DDS and JMS merit careful consideration for data-centric design and
integration. Using one or both can considerably simplify data-centric
development, and help maintain the focus on application issues, rather
than becoming hijacked by communication and data delivery concerns.
A Comparison and Mapping of Data Distribution Service and High-Level Architecture (HLA) (PDF)
In this paper we provide a comparative overview of the data distribution service with respect to high-level architecture.
We describe the equivalent terminology and concepts, and highlight the key similarities and differences in the areas of
declaration management, object management, data distribution management, ownership management, federation
management, and time management.
We explore the architectural mapping between HLA and DDS. We develop an outline for translating from one model
to the other, and examine the needed supporting transformations and assumptions. We conclude with remarks and
observations on building applications that can utilize both HLA and DDS technologies.
Is DDS for You? (PDF)
Today’s embedded software applications are increasingly distributed;
they communicate data between many computing nodes in a networked system.
Several network middleware designs have arisen to meet the resulting communications
need, including client-server, message passing, and publish-subscribe
architectures.
The new Object Management Group (OMG) Data Distribution
Service (DDS) standard is the first comprehensive specification available
for “publish-subscribe” data-centric designs. This paper helps
system architects understand when DDS is the best fit to an application.
It first provides an overview of DDS’s functionality and compares
it to the other available technologies and standards. Then, it examines
driving factors, potential use cases, and existing applications. It concludes
with general design guidance to help answer the question of when DDS is
the best networking solution.
Can Ethernet
be Real-Time? (PDF)
Historically, Ethernet has been perceived as an unsuitable medium for
real-time data distribution. The fear was that the fundamental access
algorithm, CSMA/CD, and the popular transport protocol, TCP/IP, could
not provide sufficiently consistent latency for deterministic applications.
This paper takes an analytical look at the possibility of using IP and
Ethernet for high-performance real-time network applications. We will
examine the
performance of Ethernet network hardware for real-time designs by studying
the statistics of the non-deterministic collision-arbitration algorithm
and present a formula that you can use to evaluate its effect on your
design. Finally, we will look at the software layers and propose a communications
model that enables deterministic distributed application operation.