Follow-Me Tracking on Arduino VENTUNO Q

ROS 2 Jazzy

Arduino®️ VENTUNO™ Q

<html><body><p> </p><figure><img src="https://projects.arduinocontent.cc/ce261b54-1702-48fb-b806-4a9292522499.png"/><figcaption>Pipeline architecture diagram</figcaption></figure><p>"Follow me" is the friendliest possible introduction to closed-loop robot control: a robot that keeps a target roughly centered and at a comfortable distance, using nothing but a camera and a velocity command. It's also a deceptively good teaching example, because it forces you to connect three things that usually get built in isolation — perception, a simple control law, and a simulated robot base — into one working loop, before you ever risk a real motor. </p><h3>Prerequisites</h3><figure><img src="https://projects.arduinocontent.cc/ac4f2df9-eb1a-432e-b437-0feb75999b12.png"/><figcaption>Prerequisites table</figcaption></figure><h3>Setup and installation</h3><p>Operating mode: desktop for Isaac Sim, headless/SSH is fine for the ROS 2 side</p><p>This sample spans two machines, and they don't need the same operating mode:</p><ol><li>Simulation host: run Isaac Sim in its normal desktop ("SBC" style) setup — keyboard, mouse, and a display — because you need to see the viewport to press play and visually confirm the robot's behavior in Step 4 below. Isaac Sim can be driven headless/remotely with extra configuration, but that's outside the scope of this walkthrough; if you need it, start from <a href="https://docs.isaacsim.omniverse.nvidia.com/">NVIDIA's Isaac Sim documentation</a>.</li><li>Device/host side (tracker + controller): this is a plain ROS 2 process with no GUI dependency, so an SSH session into the board/PC running tracker and controller works with no extra steps — you only need a working ROS 2 Jazzy install (see below) and network reachability to the simulation host.</li></ol><p>There's no ADB step in this sample — nothing here runs on an Android-based device.</p><h3>Additional hardware</h3><p>None. This sample is fully simulated: the "camera" and "robot base" are Isaac Sim nodes, not physical devices, so there's no camera to plug in and no motor controller to wire up. The only physical requirement is a workstation capable of running Isaac Sim at a usable frame rate — a discrete GPU is recommended, since a software-rendered Isaac Sim session will make the tracker's frame rate (and therefore the whole control loop) noticeably worse.</p><h3>Getting the source onto each machine</h3><p>Both the simulation host and the device/host side need a copy of the same ROS 2 workspace. There's no zip archive to download for this sample — you build it from source with colcon:</p><figure><img src="https://projects.arduinocontent.cc/060a53c2-b988-477d-907c-04fe037e6680.png"/><figcaption>Clone and build command</figcaption></figure><p>No manual unzip step or special file placement is needed — colcon build reads the package directly from src/qrb_ros_samples/robotics/simulation_follow_me and installs everything under ~/qrb_ros_sample_ws/install. You don't need to chmod anything by hand either: colcon build preserves/sets the executable bit on the launch files and any Python entry points it installs. Run source ~/qrb_ros_sample_ws/install/setup.bash before any ros2 launch command below.</p><h3>Software prerequisites</h3><figure><img src="https://projects.arduinocontent.cc/8daff13c-fa30-498c-9be0-55c7c0e91749.png"/><figcaption>Software prerequisites table</figcaption></figure><p>No extra Python packages beyond what a standard ROS 2 Jazzy desktop install already provides are required for the tracker/controller nodes themselves. If the package you build declares its own dependencies, satisfy those first with :package.xmlrosdep</p><figure><img src="https://projects.arduinocontent.cc/a0239b17-cf22-4fe2-866f-751e3ac7f43e.png"/><figcaption>Rosdep install command</figcaption></figure><h3>What the sample does</h3><p>The simulation_follow_me sample runs entirely in Isaac Sim against a simulated robot base, so you can iterate on the tracking and control logic safely before deploying to hardware. As the architecture diagram shows, the simulated world exposes exactly the sensors a real robot would: color and depth image topics plus camera info, all published by the simulator's camera and depth-camera nodes. </p><figure><img src="https://projects.arduinocontent.cc/7e3d36e9-4098-45d5-873e-6aeb085623d8.png"/><figcaption>Topic table</figcaption></figure><p>The Tracker module fuses the color and depth streams to keep a lock on the target and estimate how far away it is; the Controller module turns that into a Twist command — turn to keep the target centered, move forward or back to hold distance. Everything on the "Device" side of the diagram is the sample's own code; everything on the "Simulation" side is provided for free by Isaac Sim's built-in camera and root-base nodes. </p><h3>Why depth matters for the control loop, not just the tracker</h3><p>A color-only tracker can tell you where a target is in the image, but not how far away it is — and "how far away" is exactly what the controller needs to decide whether to move forward, hold position, or back up. That's why the depth stream isn't optional here: the controller's forward/back velocity term is driven directly by the depth estimate at the target's tracked pixel location, while the angular velocity term is driven by the target's horizontal offset from image center. Decoupling these two control axes — depth for linear velocity, image-plane offset for angular velocity — is a simple but effective pattern worth reusing in any vision-based following behavior. </p><h3>Try it yourself, step by step</h3><p>Because this sample targets Isaac Sim rather than a physical camera, the fastest path to a first result is the simulated one. Note that exact package/launch names may vary slightly by repository release — check ros2 pkg prefix simulation_follow_me and the package's own launch directory if a step below doesn't match exactly. </p><p><strong>Step 1 — Launch the Isaac Sim scene </strong></p><p>Open the bundled Isaac Sim scene for this sample and press play. This brings up the simulated robot base and starts publishing /camera/color/image_raw, /camera/color/image_info, /camera/depth/image_raw, and subscribing to /cmd_vel. </p><p>Expected result: confirm the topics are live from a terminal on the same ROS domain: </p><figure><img src="https://projects.arduinocontent.cc/390fa192-2de5-46c9-b9d6-e59193fccc9b.png"/><figcaption>Check camera topics command</figcaption></figure><p><strong>Step 2 — Launch the tracker and controller on the device/host side </strong></p><figure><img src="https://projects.arduinocontent.cc/a892871d-aa5e-4bce-90c0-be440ef68883.png"/><figcaption>Check cmd vel rate command</figcaption></figure><p>Expected result: both nodes start with no errors, and the tracker begins logging that it has locked onto a target as soon as one enters the simulated camera's view. </p><p><strong>Step 3 — Confirm velocity commands are flowing </strong></p><figure><img src="https://projects.arduinocontent.cc/db977ec5-aefb-493a-a87f-228ebaeee612.png"/><figcaption>Check cmd vel rate command</figcaption></figure><p><strong>Step 4 — Observe the behavior in the simulation viewport </strong></p><p>Expected result: the simulated robot base turns to keep the target roughly centered and moves forward/backward to hold a comfortable following distance, without oscillating wildly or losing the target during normal walking-speed movement. </p><h3>Troubleshooting</h3><figure><img src="https://projects.arduinocontent.cc/676d6016-787c-4796-9460-67ba10fbf1cd.png"/><figcaption>Troubleshooting table</figcaption></figure><h3>From simulation to a real robot</h3><p>The value of keeping this as a simulation-first sample is that the tracker and controller nodes don't know or care that /camera/color/image_raw and /cmd_vel are coming from Isaac Sim. Swap the simulated camera and base nodes for the real qrb_ros_camera driver and your robot's actual motor controller, and the exact same tracking and control logic should carry over — that's the whole point of validating the behavior in simulation first. </p><p> </p><p> </p><p> </p></body></html>

Follow-Me Tracking on Arduino VENTUNO Q

Real-time person tracking (color + depth fusion) driving closed-loop velocity control to follow a target in simulation on Arduino VENTUNO Q.