Learning NVidia Isaac ROS2 — Part 1 — Dev Setup
Until recently every time I tried to learn NVIDIA Isaac and related technologies, I did not have good luck and I always ended up spending many days banging my head trying to get everything installed and working on my old Jetson Nano and spent enormous amount of time trying to figure out compatibility issues between Ubuntu, Jetpack and ROS versions and finally gave up. Believe me, I’m not the one to easily give up on things and I tried many times. You must be thinking why is this guy singing the sad song and not getting to the point.
Well, as you may have guessed from the title of this article things have changed a bit after getting back to it after a decent break and trying it one more time and I have had some initial success getting basic things work in NVidia Isaac world/system and I would like to share my learning experience with you gentle folks as I always have!!
So what changed this time:
- I tried starting from a different path this time — instead of banging my head on Jetson Nano I realized that there is a path of least resistance, to try to get things to work on my laptop which has NVIDIA GeForce RTX 3050 Card with 6 GB ram and not even bother with Jetson Nano. It’s an MSI GP66 Leopard 11UG that I bought about a year ago.
- Instead of trying to install everything on bare metal Ubuntu, I tried to follow NVIDIA’s advice and used their docker Container Toolkit which I’m starting to like a lot, I must say.
- Also, I tried to watch a bunch of NVIDIA Isaac Webinars and blog articles [most of them are from Rafaello Bonghi @raffaello86]!!!
Alright, with all that off my chest let me describe the hardware and software I used as my base setup to get these things going.
Hardware
Laptop with Ubuntu 22.04 Jammy with ROS2 Humble desktop installed as base setup. I have also recently upgraded my laptop and maxed out its RAM to 64Gb. I also added a second 1Tb SSD drive in empty slot as part of upgrade [this is optional] since these laptops are pretty hard to open, I figured why not.
That’s really all you need for hardware, rest is all software configuration on the laptop.
Basic Software Setup
- NVIDIA Graphics Card Drivers
First thing after getting your basic Ubuntu 22.04 running will be to make sure to install latest NVIDIA drivers. This is a very highly discussed and confusing topic on the web and I would recommend following Ubuntu’s own instructions to install it properly. I must say this is something you’ll have to figure out for your own hardware combination as it varies a lot and I had been in the jungle of installing NVIDIA drivers on Ubuntu and almost breaking my Ubuntu install where I had to get into Rescue mode with root a couple of times and it was pretty ugly. So save yourself trouble and follow this article here — https://ubuntu.com/server/docs/nvidia-drivers-installation. Follow the section named — Installing the drivers for generic use (e.g. desktop and gaming). It is basically a single command that makes it very simple.
sudo ubuntu-drivers installYou may have to mess around with secure boot options in your BIOS settings until things work correctly.
2. ROS2 Humble
I’m assuming you are familiar with ROS2 and how to install ROS2 Humble on your Ubuntu 22.04 machine — if not you can run a single script from Linorobot/Ros2me git repository and it will install it for you.
3. Installing NVIDIA Container Toolkit
You can follow instructions here to setup the NVIDIA Container Toolkit — https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html
The guide above is a bit confusing and it gives 3 different ways to do the same thing so just make sure to follow only one path. I chose Installing with Apt and Configuring docker.
At this point you should be ready to run sample workload by running nvidia-smi image like this:
sudo docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smiIt should give you an output like this if everything worked correctly:
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.129.03 Driver Version: 535.129.03 CUDA Version: 12.2 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 NVIDIA GeForce RTX 3070 ... Off | 00000000:01:00.0 On | N/A |
| N/A 52C P8 11W / 125W | 635MiB / 8192MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
+---------------------------------------------------------------------------------------+If you get an error then something was not setup during steps you performed so try to retrace your steps slowly one more time.
After everything was installed I was initially not able to launch GUI applications like RViz2 from the running container and I had to follow steps here to setup CDI Configuration. I had to reboot and after that everything started working properly.
If you are savvy with docker you can see additional steps here to customize your docker setup — https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_common/index.html#isaac-ros-docker-development-environment
4. Installing Omniverse Launcher and Installing apps
At this point you can follow the Workstation Setup instructions from NVIDIA docs and you can follow the steps to install all Apps except Nucleus, you don’t need that for now.
- Confirm your system meets system requirements and driver requirements for Isaac SIM [you’ll be able to run nvidia-sim check to see all versions of drivers installed in previous steps].
- Mine is 11th Gen Intel i7–11800H and you can see my drivers and other specs in nvidia-sim output above.
You can follow a good youtube video here. You will need to fill out developer info form and you will have to create/have an NVIDIA developer account to login after it launches. The Omniverse launcher setup takes a bit of getting used to but once you have it running you can install other apps like Cache and Isaac Sim from within the launcher. Even though it says launcher it also serves as installer app for all other apps from NVIDIA.
The only two apps we need to install are Cache and Isaac Sim as shown in the youtube video referenced above. It will take sometime to download and install everything. After that first you need to make sure the Cache is started and then launch the Isaac Sim app.
First time you launch the Isaac Sim app it will take a while to come up but be patient and keep clicking Wait. Some of the additional steps that can help you make the Isaac Sim go faster are :
- Make sure in nvidia-setting GUI app within Ubuntu, you have selected Performance Mode [not on-demand mode]
- Make sure you have enough RAM [my reason for RAM upgrade]
- Also, make sure you have added decent amount of swap file space for Ubuntu. You can see additional instructions here.
At this point all the setup steps are done and you can play with Isaac Sim by following official tutorials.
In the next few articles I’ll try to dig in deeper and show how I was able to run the isaac-ros-common packages to try out various isaac-ros packages like April Tag and even import Turtlebot3 URDF and drive around the robot within Isaac Sim using standard ROS2 tools like rqt_robot_steering.
Hope you have success like I finally did with Isaac Sim, have fun!!!
