RTI Recording Service: How to Record, Analyze and Replay Real-Time Data

In Episode 53 of The Connext Podcast:

• [0:24] What is Recording Service?
• [2:14] Recording Service highlights
• [4:42] Most popular use cases
• [9:08] How does Recording Service integrate with other RTI Tools?
• [10:14] What's the easiest way to start using Recording Service?

Related Content:

• [Free Connext DDS Trial] Try a fully-functional version of Connext DDS for 30 days.
• [RTI Recording Service] Recording, analysis and replay of real-time high-throughput data, events and messages.
• [RTI Routing Service] Scale and integrate DDS systems across multiple platforms, systems and networks.
• [RTI Labs] A free program that provides you with early access to new technology, including software projects, downloads and tools. 

Podcast Transcription: 

Steven Onzo:

Hello everybody, and welcome to another episode of The Connext Podcast. Today I'm glad to be joined by Juanlu Jimenez. He's a Senior Software Engineer here at RTI and our expert on RTI Recording Service. Juanlu, welcome to the podcast. How are you?

Juanlu Jimenez:

Hi, Steven. Thanks, I'm fine. Thank you for having me.

Steven Onzo:

We're glad to have you. To add some context for our listeners, Juanlu is live from Granada, Spain right now, while I'm here in the California, Bay Area. I was glad we were able to make this work with the time difference. As I mentioned before, we're here to discuss Recording Service, so let's start at square one. What is Recording Service?

Juanlu Jimenez:

Recording Service sounds like one tool, but it's actually a set of tools. It's a recorder, which you can understand this by thinking of a black box. It's an application that can subscribe to your systems topics and then record them to permanent storage. So similar to what the black box in planes do. There are also two additional tools, because there's the replay service, which allows users to reproduce the data flow when it was recorded back into the DDS system.

And then there's a converter. Converter allows you to extract the data, so you're not locked into a database format. You can get your data to analyze it later. From a functional point of view, it is a black box, because it records every event that happens in your system. It is highly configurable so topics can be filtered in and filtered out. You can select many configuration options. Although, we wanted to keep it simple. It also allows you to perform tasks in the user systems. For example, post merging numbers, mission analysis, testing and simulation. That is what it was designed for.

Steven Onzo:

Can you discuss some of its highlights and how DDS developers will benefit from them?

Juanlu Jimenez:

Recording Service has had many versions. It's a very old tool now, although it has been re-architected recently. Starting with version 6.0, it is a very new tool. We wanted a whole new framework because we wanted to increase the performance. Also, we wanted to increase the stability from a maintenance point of view since it was built over the well known tool we call RTI Routing Service. Users may know about it already. This architecture is stable in performance and has been tested in many, many systems. So what we did is built Recording Service on top of it, we thought it was a very good idea. Now, on top of that, we added something that was not available in the old version, which is plug-ability. The old Recording Service is nice, but it can only recall to SQLite. Now, with the new version, apart from the things I've mentioned, we added the ability for users to plug in their own permanent storage needs and their own technologies. It provides an API that customers can use and adapts to their needs. This works for recorder for replay and for convertor. Now, the default implementation uses SQLite because we believe that it's a very well-known technology now. It's deployed in many systems and Android actually uses that as the base for their data storage.

It is a well-known technology. It's well maintained and it is performant. It allows SQL access, which is something that we didn't want to miss. With this re-architecture, one thing that we also wanted to keep is the compatibility. If you used Recording Service in the past and you want to use the new one now, there is a compatibility layer where replay or convertor would read all the  databases and they are then replayed and converted. We call them legacy databases.

Steven Onzo:

Can you talk a little bit about the most popular use cases for this?

Juanlu Jimenez:

Lately, we've seen an increase of Recording Service being used in autonomous cars and the automotive industry in general. I know it's been used as a black box system there, and it's also being used as a black box in many other systems, like boats and mining vessels. There are many other applications in that regard.

Another thing that I've seen in the past and is increasing in popularity is Recording Service for testing. Customers record data of their live systems, and then we replay a converter. They can then analyze it or they can replay it back into a test system where they are developing their systems. That allows them to correct defects to the normal development process for our system of systems. As for the simulation use case, the customer will use the tool to record data, then replay that data to compliment the assimilated environment. While they're simulating, they're using real data in their simulated environment. It's a very interesting use case, certainly.

Steven Onzo:

I have a quick question. Is there a reason why we see Recording Service used in transportation more frequently? Like you said, boats, automotive and mining vessels, or is that just a coincidence?

Juanlu Jimenez:

No, I don't think it's a coincidence. All of those systems have needs to log the data. Planes have a need to log the data that's in their systems as well as boats and applications in A&D. Recording data and having a place to store that data may not only be a good thing to do, but it may be necessary for certain system compliances. 

Steven Onzo:

Is Recording Service used differently across development, testing and deployment of distributed systems? These are all different phases. Is the application itself being used differently among those three different development stages?

Juanlu Jimenez:

Yes, I think so. When you are using Recording Service for testing, they want real data. Let's go back to the car case. They want real data because they have expensive LiDAR and radar information. Getting the car on the street is not always cheap, so they want the data because it's a representation of the real data. Then they can dump that data in the system and can adjust their development. They can detect drivers, they can fix and they can test.

This is very different than if you have deployed Recording Service, for example, as an activity logger or as a black box. Which is interesting because it allows for post-mission analysis, which can be interesting to do. Not to mention statistics. There are many reasons why you would want to deploy it. I've mentioned that before. But the use cases are very different. It has to be able to work, persist and be resilient. 

Steven Onzo:

You mentioned Routing Service before. How does Recording Service integrate with other RTI tools like Routing Service?

Juanlu Jimenez:

So as I mentioned, it's built on top of it. Although Routine Service is very complex tool, it's extremely powerful, but it's complex. We wanted to abstract users from that, even though we were building Recording Service on top of it. Even though within that, it does express some of the functionality in Routine Service like transformations. It also allows you to thread through control. It does not expose adapters, but it does expose an API in a similar fashion.

There are plans to expose processes. There are many things that Routing Service and Recording Service have in common in terms of functionality. Although, as I said, we wanted to simplify the two.

Steven Onzo:

We've described Recording Service a bit. I want to transition over to how users can get their hands on it and start using Recording Service. What is the easiest way for our listeners to start using Recording Service?

Juanlu Jimenez:

Well, I would say with any piece of software, you just have to download it from www.rti.com, install it and try it out. It's easily accessible through RTI Launcher, and anyone that has downloaded our bundles know's that it's integrated with Launcher. It's easy to configure there are built-in configurations that work out of the box and  configured to record any topic. That can, of course be changed. Also, we have a good amount of examples in the RTI community website, and the RTI GitHub repository. I would say download it, check it out and try it. I'm sure you're going to love it.

Steven Onzo:

It's as easy as that. Well, I have one last question for you and that's simply, what is your favorite part about Recording Service?

Juanlu Jimenez:

I really love the new architecture. It was a huge improvement. From a technical point of view, this may not be necessarily useful for people, but it's more maintainable, which means it's going to be easier to make resilient. It's also multi-threaded, which the old version wasn't and it could become cumbersome. I love that it's pluggable, this was something I really wanted to put in the product, the plug-ability aspect, because we had some customers in the past ask, "Okay, yeah, can I move away from SQLite?"

"No, I'm afraid not." So now we can answer, "Yes you can." It's quite interesting. It's more performant and more resilient. We've done performance testing. Worst case scenario is doubling the performance of the old one. I really like it and I hope our users like it too.

Steven Onzo:

I hope so, I think they will. Thank you for your time today, Juanlu. It's very much appreciated.

Juanlu Jimenez:

Thanks, Steven.

Steven Onzo:

And to all of the listeners, again, accessing Recording Service is as simple as downloading the bundle from rti.com, installing and trying it out. Thanks for listening, everybody. And we'll see you on the next episode of The Connext Podcast.