Depth Multitool on the Arduino VENTUNO Q

Arduino®️ VENTUNO™ Q

Arduino App Lab

<html><body><h3>Introduction</h3>Close one eye and look around the room. You still know the mug is nearer than the door, and that you could reach the chair but not the wall. Your brain works it out from a single flat picture, using cues it has learned over a lifetime: how big things usually are, where they touch the floor, which one hides the other. A camera has none of that. To it, every pixel is just a colour, and a photo of a hallway looks no different from a painting of one. So when robots and phones need distance, they add hardware: a second camera for stereo, an infrared projector, a LiDAR scanner. Neural networks have now learned the one-eyed trick. Monocular depth models look at one ordinary image and estimate how far away every pixel is, and the good ones can pick a person out from the chair behind them. The catch has been compute. These models are vision transformers, the kind that normally wants a laptop GPU, not a small board that runs from a power bank. This project puts one on the Arduino VENTUNO Q, feeds it a plain USB webcam, and turns what it sees into four tools you can play with at 20 frames per second. <figure><img src="https://projects.arduinocontent.cc/f97b4e62-afa8-4ccc-805a-4aa4f1f04065.jpg"/><figcaption></figcaption></figure><h3>What it does</h3>There is no depth sensor here. A USB webcam takes an ordinary picture, and Depth Anything V2, a neural network running on the VENTUNO Q's NPU, estimates how far away every pixel is from that picture alone. The app turns that estimate into four tools, one per number key. <figure><img src="https://projects.arduinocontent.cc/ba7a0b12-76ed-4ce1-9dc9-383369dac0f3.jpg"/><figcaption></figcaption></figure> Heatmap (key 1) shows distance as colour. Near is bright, far is dark, with the camera image blended underneath. <figure><img src="https://projects.arduinocontent.cc/f0139c52-223f-4e91-b6d8-ac2321f33148.jpg"/><figcaption></figcaption></figure>Slicer (key 2) lights up only what sits at one distance, like a slice through an MRI scan. The arrow keys sweep the slice through the room and change its thickness. <figure><img src="https://projects.arduinocontent.cc/15b9489e-c9df-440c-93c2-68acbc9b23b4.jpg"/><figcaption></figcaption></figure>Flood (key 3) fills the room with virtual water from the back wall forward. Whatever is close stays dry. <figure><img src="https://projects.arduinocontent.cc/4d15d24b-deb3-4749-89fa-85967960c1e1.jpg"/><figcaption></figcaption></figure>Sonar (key 4) measures how crowded each side of the view is, and you can hear it through the display's speaker: low clicks for the left, high clicks for the right, faster as things get closer. Walk toward a wall and the clicks speed up; turn toward an open doorway and they stop. <h3>Why the VENTUNO Q</h3><figure><img src="https://projects.arduinocontent.cc/e94ccf4f-d926-44ac-a689-d37549dec496.jpg"/><figcaption></figcaption></figure> Depth Anything V2 Small has 25 million parameters. The VENTUNO Q's Qualcomm Dragonwing NPU runs it in 16 ms, with every layer on the NPU. That leaves the eight CPU cores free to decode the camera, draw the tools and generate the sonar sound at the same time. It is also a regular Ubuntu computer with a desktop, and the board, the 10 inch screen and the camera all run from one USB-C power bank. The board has its own power monitor, which made it easy to tune for battery life: see How it works. <h3>Things used in this project</h3><figure><img src="https://projects.arduinocontent.cc/47991a5a-d0dc-4ddc-a39d-3f6fa0812343.png"/><figcaption></figcaption></figure> Software: the Ubuntu 24.04 image the VENTUNO Q ships with, Qualcomm AI Runtime, LiteRT and OpenCV. Qualcomm AI Hub is only needed if you want to rebuild the model yourself. <h3>Build it</h3><h3></h3><figure><img src="https://projects.arduinocontent.cc/779daa4f-e3b3-4c90-87bd-8526dd60b60c.jpg"/><figcaption></figcaption></figure><figure><img src="https://projects.arduinocontent.cc/8388e455-1d45-4775-a402-85de3fc0a2db.jpg"/><figcaption></figcaption></figure><figure><img src="https://projects.arduinocontent.cc/0bfb4adb-bbaa-4111-898b-29156591c7ae.jpg"/><figcaption></figcaption></figure>1. Set up the VENTUNO Q and connect the display, webcam and keyboard. Arduino's official VENTUNO Q getting started page: <a href="https://docs.arduino.cc/tutorials/ventuno-q/first-setup/">https://docs.arduino.cc/tutorials/ventuno-q/first-setup/ </a>2. Open a terminal on the board and run: <pre>git clone https://github.com/SamuelAlexander/ventuno-q-depth-multitool.git ~/depth-multitool </pre><pre>cd ~/depth-multitool </pre><pre>./install.sh </pre>3. Double-click the Depth Multitool on the desktop. It is live in about 4 seconds. <figure><img src="https://projects.arduinocontent.cc/2f5f5fa1-7cbc-44e4-af4e-0df3a6a4b320.jpg"/><figcaption></figcaption></figure>The installer makes the NPU available to ordinary programs, builds a Python environment, fetches the model, prepares it on the NPU once and creates the icon. The first of those is the one worth knowing about: the VENTUNO Q keeps its NPU libraries inside the Arduino App Lab containers, and one package from the preinstalled Qualcomm repository puts them where any Python program can load them. <h3>Controls</h3><figure><img src="https://projects.arduinocontent.cc/cd704652-3a82-46ba-864a-f82eaf07b8b9.png"/><figcaption></figcaption></figure><h3>How it works</h3>Three threads share the work. One reads the camera, one runs the model on the NPU, and the main thread draws. While the tools draw one frame, the NPU is already working on the next, so drawing never adds to the frame time. The app is tuned for a battery. The NPU can run this model at 30 frames per second, but the board then draws 15.2 W. Capped at 20 frames per second, with the NPU in its balanced power mode and the webcam asked for only 20 frames, it draws 11.8 W, measured on the board's own power monitor. Preparing a model for the NPU takes about 13 seconds, so the installer does it once and the delegate caches the result on disk. Every launch after that loads the prepared model in under half a second. Mirroring and flipping happen before the model sees the picture. That matters for the flip: depth models expect the floor at the bottom of the image, and an upside-down frame gives a worse estimate. <h3>The model</h3>The app runs Depth Anything V2 Small in full float precision at a 336×252 input. Two findings shaped that. The Qualcomm NPU delegate's default setting keeps the NPU at low clocks: at a 336×252 input the model took 58 ms per frame by default and 12.7 ms with the delegate's sustained high performance mode, a single option when the model is loaded. And for this vision transformer, a smaller input made it faster more cheaply than the usual 8-bit quantization: fewer image patches cut the attention cost faster than the pixel count fell, and staying in float avoided quantization's artifacts, including a flattened near field. Ultralytics' YOLO26-depth was tested too. It is faster, but scored against the NYU Depth V2 benchmark its slices matched the true surfaces far less well (16.2% overlap against 24.4% for the input used here), and the Slicer depends on exactly that. <figure><img src="https://projects.arduinocontent.cc/f893b394-02d5-40e7-b18b-1ab8a5ff490d.jpg"/><figcaption></figcaption></figure>To rebuild the model, run <code class="code">research/export_model.py </code>on any computer with a free Qualcomm AI Hub account. <h3>Going further</h3>This project uses only the VENTUNO Q's Linux side. Its STM32 microcontroller is still free: physical knobs, buttons or vibration motors could talk to the app through Arduino's Router Bridge. </body></html>

Depth Multitool on the Arduino VENTUNO Q

One ordinary webcam, a depth model on the VENTUNO Q's NPU, and four ways to see how far away everything is