A Next Gen Recording Service for the IIoT Databus

In Episode 9 of The Connext Podcast:

• [1:03] What is Recording Service? What does it allow an IIoT dev to do? 
• [2:05] Recording Service comes with Replay Service, which is very helpful when debugging
• [3:15] What Recording Services is going to be when it grows up (Next Gen!)
• [8:44] What is DDS?
• [10:30] Rose explains DDS Like I’m 5 (ELI5) 

Related Content:

• [Blog] Databus vs. Database: The 6 Questions Every IIoT Developer Needs to Ask
• [Datasheet] Recording Service
• [Product Page] Record & Replay Services

Podcast Transcription:

Lacey Trebaol: Hi everyone. Welcome to Episode Nine of the Connext Podcast. I'm Lacey Trebaol and today I'm joined by my co-host, Niheer Patel. We're going to be talking to Rose Wahlin, a principal software engineer here at RTI. Rose has been involved in developing RTI's case plus code use cases, based on the requirements and challenges of the industrial internet. She has provided architecture and design guidance for many real world industrial internet systems in the power, automotive, military, aerospace and financial industries. Rose is currently leading a team here at RTI building the next generation recording service. Wow, go Rose. So, hi Rose, how's it going?

Rose Wahlin: Good, how are you?

Niheer Patel: Glad to have you here.

Rose Wahlin: Oh, thank you.

Niheer Patel: I think I need to learn a little bit about recording services too.

Lacey Trebaol: I do too.

Niheer Patel: I'm still learning about what RTI has to offer and this is on my to do list, so this is perfect for me.

[1:03] What is Recording Service? What does it allow an IIoT dev to do?

Lacey Trebaol: Yeah, right. So why don't you tell us first what recording service is, and then maybe what the delta kind of is between what people now know and can use as recording service, and what it's going to be when it grows up.

Rose Wahlin: Well, if you think of DDS as really providing all your real time data in motion, what the recording service does is it takes that in motion real time data and it records it to a database fairly simply. It allows you to define which of your topics or data streams you actually want to record, and then right now with the existing recording service, it records it to an SQLite database. The idea for a recording service is that it can be embeddable and can act like the black box in your system. So if you have an industrial application, where let's say you need to in some cases record every type of alert or alarm that happens in your system and go back later and see what went wrong, you'd use the recording service for that.

[2:05] Recording Service comes with Replay Service, which is very helpful when debugging

The recording service, it also comes with a replay service. So initially, so you record a database on your running system, and later you can either look at that using tools that come with SQLite, or you can actually replay it at the original rate that this data was recorded. It's really cool. So it lets you actually debug your system, and you can do things ... There's a GUI console that lets you pause and step sample by sample, so you can actually see as your say this original recorded system, and you want to debug it or maybe do some analysis of what happened, and you can step through and see, "Oh, this data was sent, this data was sent, this data was sent."

Lacey Trebaol: All the different events.

Rose Wahlin: Yeah, something went terribly wrong at this point, and you can potentially ...

Niheer Patel: It's almost like a DVR for DDS system. You can record and then you can play back, and then you can even play back step at a time so you can see what's going on.

Rose Wahlin: Exactly.

Niheer Patel: That's really cool.

[3:15] What Recording Services is going to be when it grows up (Next Gen!)

Rose Wahlin: Yeah and the console application, as it looks now, looks a lot like that very intentionally. The next gen recording service, the big piece for that that we're doing, is we're making it plugable. So right now the recording service, it's very tied to this SQLite embedded database. For the next generation, we've gotten a lot of requests for various databases, and so to support that, we're allowing people to actually plug in their own database. So we're providing a very simple API that lets you open your database table, write to the database and so on. We'll be doing the same with the next gen replay to let you plug in your own database that you can read from. That's a big feature.

Niheer Patel: That would be really cool.

Lacey Trebaol: That is a great feature.

Niheer Patel: Well, I can imagine, right. I mean, we're in distributed systems and, you know, they have all these distributed databases, so maybe something like Hadoop or whatever, you'd want to record and save your data on a distributed database like that.

Rose Wahlin: Oh, absolutely. We've gotten some people expressing interest in exactly that.

Niheer Patel: Oh, cool.

Lacey Trebaol: That and we also do a lot of integration projects. Customers have legacy systems that they're not changing. While one of their databases might be using SQLite, you know the other ... Gosh knows however many aren't. All of that data needs to be used in present for these things to be inter operating. So the idea, now they can throw everything into their DDS systems. Use all the data, right? That's a great thing for our customers and users.

Rose Wahlin: And yeah, the other thing that's interesting, it's not the primary purpose of it, but I think that's very interesting, is that this is being built on top of our product, the routing service. One of the features of the routing service is you can actually plug in data transformations into it.

So this lets you ... Not only are you recording your real time data, you have the possibility of actually manipulating it before it's recorded. I mentioned that the replay service, a lot of times that's for analyzing something. Let's say you have a test run of your system and you want to be able to analyze a particular test run. One use case is that you record this data and you would like every part of this data tagged with an identifier of this test run as it goes into the database. You could do that in post analysis, but this is something you can now do, you can create a data transformation and add that in at that time. So you have a lot of interesting possibilities.

Niheer Patel: So it's just, yeah, so you just take care of the data tagging at run time. You don't have to worry about any kind of post processing. In fact, it'd be ready for you to actually post process with a lot more intelligence now that you've got that tag built in.

Lacey Trebaol: And then it's fewer things to change about that piece of data later on. So data integrity I know is a huge issue in a lot of things. So like you can't go back and change a piece of data. It loses the aspect that a lot of different industries ... I know medical needs it when they do their testing. But you can't go back in and start touching the data and put it back and have everyone know that this is the data that came from this test at this point, because now there's changes made to it, right?

Rose Wahlin: So imagine that some sensor doesn't have a patient ID associated with that data and you want to make sure that that patient ID is associated with the sensor reading exactly at the time it's records, you can do that now.

Lacey Trebaol: That's great.

Niheer Patel: Insane. And you said built on routing service referred from others that routing service is kind of a basis. This is using the adapter tool kit, I presume?

Rose Wahlin: Exactly.

Niheer Patel: Which we also refer to as the gateway tool kit, but in this case, I mean, I don't know, I guess it's kind of a gateway. But really it's allowing you to do data transformation using the same technology that we share with our customers, and another example of how we just build on top of our own technology and use our own technology.

Rose Wahlin: Yeah, all of the ability to plug in adapters, you can record some other data as well as DDS data by plugging in your own input adapter.

Lacey Trebaol: When is this available?

Rose Wahlin: So, for the recording side, right now we have an early prototype. We have just a few people looking at it. Our goal is to have a set of customers from a few different industries take an in-depth look, which is a challenge in itself because everybody has their own schedules and their, "We're doing something else right now. We don't have a chance to look at this." But we would like people to give their feedback, which we can incorporate and then ship.

Right now, so we have the early version of the recording service incorporate this to make an early version of a replay service, which we're looking at second half of this year. Then the actual next step of really productizing it. I should note that this early prototype, it records data. We have test results. We have performance test results for it right now. It's just very limited in features, and intentionally so because we don't have a way of knowing what features were used by some existing customers or not. That's why we're asking for feedback. We could simply re-implement everything that we have, but then-

Lacey Trebaol: But why.

Rose Wahlin: Yeah, why have the complexity.

Niheer Patel: Focus on what the customers want.

Rose Wahlin: Exactly.

Niheer Patel: What are they going to use and just deliver that, and deliver that well.

Rose Wahlin: Yeah, and so yeah, our initial ... So we have the proof of concept version. Then our initial shipment is probably also going to be limited feature, but the idea is to have it be absolutely solid, high performance and, yeah, built on routing service so it's more plugable.

[8:44] What is DDS?

Lacey Trebaol: All right. So we're going to end this interview with the same two questions that we ask all of the RTI staff that we have in here. The first one is what is DDS?

Rose Wahlin: DDS is a real time middleware. What that means is it's a set of libraries that we provide that you link into your application. We do the networking piece for you. So, if you have an application that needs to communicate with other applications, and specifically if you need any sort of real time behavior or any interesting behavior about how your data moves. So, either your data has to get from point A to point B at a certain time and you need a guarantee. Or, maybe you have a network with incredibly high latency. I know that some of us have actually debugged messages from the International Space Station, which is the coolest thing.

Lacey Trebaol: So cool.

Rose Wahlin: But yeah, that of course has very high latency for your data. This middleware, it's built for distributed systems and it allows you to send and receive this data when you have these interesting requirements. So requirements for guaranteed delivery at a certain time or as close as you can get depending on your network, or very high latency, very low latency.

Lacey Trebaol: Handling jitter, all this stuff.

Rose Wahlin: Exactly, yeah exactly. So the middleware's doing that piece for you. You link it into your application. You define data types. You define data streams that you name. Then the middle, and you define the qualities of service you need for those data streams, and then the middleware does that piece for you.

[10:30] Rose explains DDS Like I’m 5 (ELI5)

Lacey Trebaol: It's fancy. All right, so the second question that we're asking everyone is now you've explained what DDS is. Can you explain it to me like I'm five?

Rose Wahlin: So if you have multiple computers that need to communicate with each other and if you have something like ... Let's say if you have a robot and you need to, from your computer, send messages to that robot to say, "Hey, turn left right now." Then what we do is we create that piece that's actually sending those messages to that robot.

Niheer Patel: Great, I think every kid loves to be able to control a robot or a drone. Can you make it a drone?

Rose Wahlin: Yeah, can make it a drone. Make anything you want, absolutely, yeah.

Niheer Patel: I'm the happiest five year old.

Rose Wahlin: I know, right. Well and we really do. I've been with customers who were controlling robots through a satellite. It's not a fake example at all.

Lacey Trebaol: Like ESA, did their telerobotic one with the Space Station. That was from the lab in the Netherlands to the Space Station. Then there was that underwater robot, Schneider.

Rose Wahlin: Oh yes, those are great, yeah. So, yeah, I got to work with underwater robots in some ... Yeah, there-

Niheer Patel: Underwater, outer space.

Lacey Trebaol: So cool. We do it all.

Niheer Patel: Everything, everywhere.

Rose Wahlin: Yeah, I've worked with at least two. One was at NASA and one was somebody else. But, yeah, they were actually looking at sending satellite messages to a robot. In one case I think near the North Pole to get an idea of latency and actually control it and make it move and so, yeah. The five year old likes robots, we like robots.

Lacey Trebaol: We're really just like big five year olds.

Thanks so much for listening to Episode Nine of the Connext Podcast. We hope you enjoyed it. If you're looking for more information about the next generation recording service, reach out to your local account rep or email us at info@RTI.com. Have a great day.