How to setup ROS2 Fast-DDS Discovery Server

If you have worked with ROS2 enough and tried to run your ROS2 nodes across multiple machines, like your host computer and a robot computer like Raspberry Pi or Jetson, you most likely ran across issues like this, where you are running the nodes on one machine but the nodes and topics are not being recognized on the other machine, even though both the machines are on the same WiFi network.

There are certainly many other issues related to network that can cause this behavior but given that everything in networking setup is correct and you have setup your ROS_DOMAIN_ID to be same on both machines, you should not be running into these issues. However, it happens and I have experienced that first hand.

One of the workarounds I tried was to restart the ROS2 daemon on both machines before I launched my nodes on both machines and I even created a little bash script and alias on both machines so it would restart the daemon each time I compiled/built the workspace. However, this is more of a hack and not a real solution.

ros2 daemon stop;ros2 daemon start;

Recently, I tried to dig into a different mechanism that is available with ROS2 foxy onward called Fast-DDS Discovery Server and this article is about documenting my experience with it so someone else can learn from my trials and tribulations.

Background on Fast-DDS Discovery Server

If you are interested in reading more about it you can dig into this very detailed documentation here.

As the documentation above states, the main reason for this behavior is that by default DDS implements Simple Discovery Protocol, which has following limitations to paraphrase the documentation:

• It does not scale as network traffic increases
• It may not work reliably in specific scenarios e.g. Wifi

On the other hand the Fast-DDS Discovery server basically acts as a middle man that each node connects to like client/server and hence reduces the network traffic significantly. You can see it here in the test scripts comparison.

In addition, it has some more benefits like you can isolate your network with multiple discovery servers that only subset of your robots are connecting to and hence keeping the rqt graphs clean and partitioned so the nodes that are connecting to one Discovery server are not visible to other nodes and vice-versa.

This article is not supposed to be a duplication of the documentation linked above so we are not going to describe all the concepts here, you can read up on it directly from that link. Here we will just focus on simple steps you need to do to configure the Fast-DDS discovery server to avoid all the issues we run into when we use the default Simple Discovery Protocol.

The basic config steps are fairly straightforward:

1. Export the environment variable indicating which Fast DDS server(s) you are connecting to
2. Setup the super client XML configuration file — you can download it from here — Super Client XML Config file
3. Start the Discovery server
4. Launch the nodes or apps like Rqt from the terminal window where you have already exported the environment variable [in step #1]

That’s it.

Lets do a real world example of a physical robot that runs its nodes on two machines — a host machine and a robot machine [Raspberry Pi].

Steps to perform on both Host Machine and Robot Machine

Step 1 — Either in your ~/.bashrc or on a new terminal export the environment variable [assuming host machine IP address is 192.168.1.XX1, you can substitute your own IP addr there]. Assumption here is we are running only one discovery server on host machine and both the machines are pointing to that host machine IP address for discovery server. However, it is possible to setup multiple discovery servers as backup or for partitioning reasons, you can read more in original docs linked earlier.

export FASTRTPS_DEFAULT_PROFILES_FILE=~/super_client_configuration_file.xml
export ROS_DISCOVERY_SERVER=192.168.1.XX1:11811

Step 2— Download the Super Client XML Config file and copy it to your home directory ~. This is required if you use the same export command above, you can adjust the export command above if you choose to use a different directory but make sure the environment variable points to your correct file path.

Step 3— start the discovery server by running following command. This will start the server with id=0 and on port 11811 which is default port, so if you don’t supply -p with port number it will default to 11811.

fastdds discovery --server-id 0 -p 11811

Step 4— You can launch a new node from the same terminal [or a terminal that you have setup the bashrc file to export on start] and you can also launch another terminal that has the environment variables set as described in Step1 and launch Rqt app from there and you’ll see that only the nodes that are launched from terminals where that env variable is set will be visible.

If you launch a new terminal where the environment variable for discovery server is not set, those nodes will not be visible to the Rqt.

A detailed demo is also documented in original documentation link above if you want to test the concepts out with talker and listener examples.

You may have to restart the ROS2 daemon on both machines one time before the configuration takes effect but this is one time only. If you set the env variables in bashrc, you will not have to worry about it again. The only thing you do have to worry about is to launch the discovery server in Step 3 unless you have automated that as part of your startup scripts as well.

That’s it. You’ll see that nodes and topics show up consistently from now on without messing with the ROS2 daemon.

One assumption worth pointing out in this article is that you are working with machines with static IP addresses, if that is not the case you will need to figure out a way to automatically set the IP address in the export environment variable scripts in your bashrc file at startup.

I have tried to keep the steps to very simple but the documentation also describes how you can setup backup discovery server in case one of them goes down, the backup will continue to connect the nodes.

Enjoy!!