Debug, test and design your system to help you meet your operational requirements with Connext Tools! This week we kick off the first of a three-part podcast with RTI Principal Software Engineer, Ken Brophy. Learn about the Connext Tools product suite and how organizations are using them to increase productivity and results in their Industrial IoT environments.
In Episode 25 of The Connext Podcast:
- [0:27] From power user to RTI engineer: How Ken went from RTI customer to shaping the software
- [4:20] Challenges in developing distributed systems vs. traditional areas of software development
- [9:22] Overview of Connext Tools: What they are and how they are used
- [13:27] Using
WireSharkto debug distributed systems
- [14:50] Common debugging use cases and how Admin Console can help
- [Blog] Implementing Simple Introspection with Connext DDS in C++14
- [Datasheet] RTI Connext Tools
- [Webinar] Accelerate Distributed Systems Development using Connext Tools
- [Webpage] RTI Labs
Steven Onzo: Hello everybody and thanks for joining us for another episode of The Connext Podcast. Today we're talking to RTI Connext Tools technical lead, Ken Brophy. Ken will discuss how Connext Tools can provide deep visibility into your distributed systems. He'll also share common customer use cases where Connext Tools have eased the development
[0:27] From power user to RTI engineer: How Ken went from RTI customer to shaping the software
Lee Johnson: Thanks for sitting down, you're in a lot of ways the spirit of Connext Tools at RTI.
Ken Brophy: You're too kind.
Lee Johnson: So, I was really interested to pick your brain about a lot of topics that relate to tools as they apply to distributed systems development, from
Ken Brophy: I am the Tools team lead, so I am kind of the technical lead but also the internal champion for advocation of tools on behalf of customers. So I'm a former RTI customer, myself.
Lee Johnson: And where did you come from?
Ken Brophy: I used to work at General Dynamics in Pittsfield, where I live, in Massachusetts, and we used DDS on a pretty big project, a new class of ship design - a surface ship - and it was just terrific. DDS itself did everything we wanted it to do, but it didn't do it in a way that was
We actually had XML-based QoS long before RTI had it, and we had a
Lee Johnson: Great, so you were looking to solve a couple of problems, one making DDS more accessible, more consumable to a set of software developers, with whom you were working, as well as provide aids to de-bugging systems as you were developing them.
Ken Brophy: Exactly. We had some software we developed in-house at General Dynamics for the ship, but we had vendors that came in and they would bring their DDS software and sometimes things didn't work and it was like ‘where do we go to figure this out’? So that's where some of the ideas came from to ease that integration, to help us in development and tuning, and to log things so we could go back after and do an analysis of what happened during that use case.
Lee Johnson: So you were obviously successful in your quest to come to RTI, and you're now that engineering manager for our tools team, which is great.
[4:20] Challenges in developing distributed systems vs. traditional areas of software development
Lee Johnson: So let's talk a little bit about some of the underlying motivation for tools, as we're terming them. Distributed system development is a software development discipline, as we see it. How would you describe it as being different from "traditional" areas of software development? And what are some of the challenges that go along with developing distributed systems?
Ken Brophy: So I think distributed systems are quite a bit different than other disciplines in software because inevitably to solve really big problems, you either need to make really big computers, which is not practical, or you need to spread the workload around to a number of smaller computers. And that spreading of the workload and the coordination of solving that bigger problem, I thought was always interesting. It's a non-trivial problem to solve because there are technical factors, how much data to send vs. how big do we make a packet or what protocols do we use and all that sort of thing. Yet it's really almost a human engineering problem too because you have to get people with different viewpoints, "We should do x," "We should do y", they have to be able to talk to each other because software has to talk to each other and so, fundamentally, you have a technical issue and you have a people issue to solve and get people to communicate.
Lee Johnson: Absolutely. And just to put some context to the size, the magnitude of these systems, can you speak to the types of systems and the number of nodes or participants in these systems that you've dealt with earlier in your career and working with customers now that you're with RTI?
Ken Brophy: Yes, so back at my former work, I can't divulge any company secrets, that sort of thing, but you know it was a fairly sizable system, you're talking about a 300-foot ship. So there's plenty of human beings that have to run the ship and they, of course, need the computers to run systems, to run defensive systems, to run everything to email and everything else on the ship. They were fairly good sized at that point, but when I came to RTI, I got exposed to much bigger customer systems and consequently bigger challenges to solve in scaling up.
My first couple of years
Lee Johnson: Certainly, and during the course of developing a system of that size, can you give some examples of types of problems, common scenarios that users would run into in
Ken Brophy: Right, it really varies but as a general rule of thumb, every time you increase the system size and order of magnitude, you run into new and interesting challenges to solve. So, most often the standard three bottlenecks are going to bite you in some way or another: memory, CPU and throughput. That tends to be the limiting factor of any computing system. Having in mind the end goal of where your system is going to be when you start designing, you can really avoid some of that. Using the products we provide, we try to help people locate those challenges, those issues, earlier on than they might have thought, that's one of the things we really like to do is to provide the information and let them be able to see what's happening.
[9:22] Overview of Connext Tools: What they are and how they are used
Lee Johnson: Excellent, so I don't think we can go too much longer without talking more specifically about the Connext Tools offering. What are
Ken Brophy: Well it's a suite of products that help customers
Lee Johnson: And give some specific examples, I don't know if we'll have time to go through every single bit of detail here but we can certainly talk to the more commonly used tools that our customers take advantage of.
Ken Brophy: Right, and it really depends on how you define a tool,
Lee Johnson: The runtime service
Ken Brophy: The runtime services exactly and so my team specifically, though we do things like the Shapes Demo, which is a learning tool to learn the concepts of DDS through an interactive application. The RTI Monitor, which is kind of a deep system de-bugger for tuning and system awareness. And then the Administration Console which does the distributed de-bugging administration and some monitoring of your DDS systems.
Lee Johnson: So what do you mean by
Ken Brophy: Well it provides some commanding control for the RTI services, primarily. But also of our distributed logger component, and we do ship command-line administration shelves for each of our services but this is a UI. The Admin Console is a graphical interface so I think it's easier to work with, and each service has its own peculiarities so you can use the UI and not have to write a bunch of code yourself to interact with the service. It just saves people time and energy.
Lee Johnson: Now you mentioned a handful of tools, and I know there are a number
Ken Brophy: Yes we have a variety of de-bugging offerings and depending on what you are trying to do and
WireShark to debug distributed systems
Lee Johnson: At some
Ken Brophy: Absolutely,
[14:50] Common debugging use cases and how Admin Console can help
Lee Johnson: So let's dig into more of the common de-bugging use cases that you see because these are cases that apply at a level much higher, typically, than de-bugging at the protocol level. Can you talk through a few of the core use cases for which Admin Console was used?
Ken Brophy: Sure. One of the big things that we see customers do with Admin Console is to de-bug QoS mismatches. The DDS API has a lot of quality of service settings, QoS, or "Quas" depending on
Another very common scenario that we see is that Admin Console can be used to visualize the data that's flowing in your distributed system. If you are suspecting that a particular topic has data on it that doesn't make sense or might be in some way corrupted or erroneous, you can subscribe to that data and bring it up in different visualizations to
Lee Johnson: And by
Ken Brophy: Exactly.
Steven Onzo: Ken, thank you very much, that was some terrific information. And thanks to all you listeners for tuning in to another episode of the Connext Podcast. Next week we'll continue our conversation with Ken and explore the world of administration and monitoring of DDS systems. If you have suggestions or feedback on this or other episodes, please contact us at podcast@RTI.com. Thanks, and have a great day!