Visual Tracking Gimbal

OpenCV (cv2)

Arduino App Lab

3D printer

Rechargeable Lithium Battery Pack

3D Printed parts 

Personal Computer

MG996R Dual-Axis Pan-Tilt

 USB C to USB C Cable 100W Fast Charging

Screwdriver set

type-a usb camera

python IDE

Arduino® UNO™ Q 2GB

Chrome Web Browser

<html><body><h3></h3><figure><iframe src="https://youtu.be/D9BSSyZQYkg?si=9mglaEt25pHdKOZR"></iframe></figure> <h3>Introduction</h3>Color tracking is one of the most classic entry-level projects in computer vision: it enables a camera to "follow" an object of a specific color, meaning the camera rotates wherever the object moves. This project implements this concept as a complete closed-loop control system, forming a full visual servoing closed-loop pipeline from color recognition, error calculation, and PID control to servo output. Unlike traditional color tracking projects, this system has five outstanding features: First, PID closed-loop control. Most traditional solutions use Bang-Bang control (rotating left by a fixed step if the target is on the left), resulting in stiff tracking and obvious overshoot. This project implements a complete PID controller: the proportional term provides fast response, the integral term eliminates steady-state error, and the derivative term suppresses overshoot and oscillation. Combined with dead-zone design and integral anti-windup, the tracking motion is smooth and natural. Second, Web-based dynamic color tuning. The tracking target can be switched without modifying the code. The web interface provides a Hue slider and six preset color blocks (yellow, red, green, blue, orange, purple). Dragging the slider changes the target tracking color in real time. The HSV color space makes color selection intuitive and robust to lighting variations. Third, real-time PID tuning panel. The web interface embeds a PID parameter adjustment panel, allowing real-time modification of Kp, Ki, Kd, and dead-zone size during tracking. Four preset modes are also provided (Default / P-Only / Conservative / Aggressive). This visualizes and interactively simplifies the PID tuning process, eliminating the need to repeatedly edit and re-upload code. Fourth, manual PWM servo driving. Instead of using the Arduino Servo library, the system manually generates 50Hz PWM pulse signals to directly drive the MG996R servos. This approach helps you understand the working principle of servos at the signal level: how angles map to pulse widths, and how pulse widths form the PWM cycle. Fifth, servo center calibration. The web interface includes a dedicated servo calibration panel: after one-click centering to 90°, the physical midpoint of the servo can be accurately found via ±1°/±5° fine-tuning buttons before installing the servo arm. This removes guesswork from assembly and ensures precise repositioning every time the arm is reinstalled. <figure><img src="https://projects.arduinocontent.cc/8c970de2-d265-4192-b597-2bce60bdbfac.png"/><figcaption></figcaption></figure><h3>Schematic Diagram and Pin Configuration</h3>Wiring for this project is extremely straightforward: two servos use only 2 digital pins, plus one USB‑C cable to connect the expansion board to the USB camera. <figure><img src="https://projects.arduinocontent.cc/ec9a989c-8d86-4c3d-8a36-1ccf696e47e5.png"/><figcaption></figcaption></figure> <figure><img src="https://projects.arduinocontent.cc/e2c4122b-abe6-4e08-b8c6-6bedf1d96973.png"/><figcaption></figcaption></figure> | Function | Description | |------|------| | 🎨 Dynamic Color Selection | Hue slider + 6 color presets (Yellow/Red/Green/Blue/Orange/Purple) for real-time tracking color switching | | 🤖 PID Auto-Tracking | Dual-axis independent PID closed-loop control, including deadband hysteresis, integral anti-windup, and EMA coordinate filtering | | 📊 Online PID Tuning | Kp/Ki/Kd/deadband four-parameter sliders + 4 preset schemes (Default/P-only/Conservative/Aggressive), with real-time effect | | 🕹️ Manual Control | Direct control via Pan 0-180° / Tilt 30-150° sliders | | 🔧 Servo Center Calibration | ±1°/±5° fine-tuning + one-click return to 90° center, for easy installation of the swing arm | | 📹 Real-Time Video | Base64 JPEG streaming (~25fps), with tracking markers displayed directly in the browser | <figure><img src="https://projects.arduinocontent.cc/3551e7d7-08eb-48e7-af51-f6f0497300b5.png"/><figcaption></figcaption></figure> <h3>project structure</h3><pre>color_tracked_plz/ </pre><pre>├── app.yaml # Arduino App configuration </pre><pre>├── assets/ </pre><pre>│ └── index.html # Single-file Web console (CSS + HTML + JS, ~930 lines) </pre><pre>├── python/ </pre><pre>│ └── main.py # OpenCV vision + PID control + WebUI service (~270 lines) </pre><pre>└── sketch/ </pre><pre>	 ├── sketch.ino # Manual 50Hz PWM servo driver (~65 lines) </pre><pre>	 └── sketch.yaml # Sketch configuration (Arduino Zephyr UNO Q) </pre><h3>PID Closed-Loop Control</h3>Dual-axis independent PID controller that converts pixel error into servo angle correction values. <figure><img src="https://projects.arduinocontent.cc/fb26263e-15fb-4a90-adc3-a44ae9ceff9e.png"/><figcaption></figcaption></figure><ol><li>Proportional (P): The larger the error, the greater the correction, providing fast response. </li></ol><ol><li>Integral (I): Accumulates historical error to eliminate steady-state deviation, with Anti-Windup limiting (±500). </li></ol><ol><li>Derivative (D): Senses the rate of error change to suppress overshoot and oscillation. </li></ol><ol><li>Deadband Hysteresis: No action when error &lt; 20px; reactivates only when error exceeds 26px to prevent edge jitter. </li></ol><ol><li>EMA Low-Pass Filter: Applies Exponential Moving Average to detected coordinates (α=0.4) to smooth inter-frame noise. </li></ol><h3>Color Threshold Segmentation</h3>Color segmentation and centroid calculation based on HSV color space: <pre>BGR → HSV conversion → inRange threshold segmentation → Erosion/Dilation denoising </pre><pre>→ findContours contour extraction → Largest contour → minEnclosingCircle centroid positioning </pre>Red color handles hue wrapping automatically (H ≈ 170°~10°, dual masks merged). <h3>Manual PWM Servo Driver</h3>No Servo library used; 50Hz pulses generated manually in loop(): <pre>Angle → Pulse width: pulse = 500 + (angle × 2000 / 180) μs </pre><pre>0° → 500μs | 90° → 1500μs | 180° → 2500μs </pre><h3>Usage Instructions</h3><ol><li>Upload Sketch — Compile and upload the sketch/ folder to UNO Q </li><li>Start App — Run this project in Arduino App Lab </li><li>Open Console — Visit http://&lt;UNO_Q_IP&gt;:7000 in your browser </li><li>Select Color — Choose the target color with the Hue slider or preset color blocks </li><li>Start Tracking — Enable the "Auto Tracking" switch, and the gimbal will automatically follow the target </li></ol><h3>Servo Calibration</h3>For first assembly or after replacing the servo arm: <ol><li>Click "Start Calibration" → Both axes automatically center to 90° </li><li>Use the ±1 / ±5 buttons to fine-tune to the physical center position </li><li>Attach the servo arm at the current angle </li><li>Click "Finish Calibration" to exit </li></ol>Auto-tracking is forcibly disabled during calibration to ensure manual adjustment is not disturbed. <figure><iframe src="https://www.youtube.com/watch?v=Pa46g3mHI1E"></iframe></figure> <h3>Usage Points and Instructions</h3>Initial Setup: <ol><li>Connect the two MG996R servos to pins D3 and D11 of the Arduino UNO Q according to the wiring diagram. </li><li>Connect the USB camera to the Arduino UNO Q. </li><li>Open the Arduino App Lab and load the color_tracked_plz project. </li><li>Upload the sketch firmware and Python program. </li><li>Configure the WiFi credentials. </li></ol>⚠️ Important: Boot Sequence Precautions Before powering on, if using a USB Hub for expansion, pay attention to the camera connection sequence. It is recommended to plug in the camera USB cable after the motherboard has finished booting. Program Startup: Method 1: Via App Lab Click "Run" in the App Lab to start the program. Method 2: Via SSH <ol><li>Log in to the Arduino UNO Q via SSH. </li><li>Navigate to the application directory: cd ArduinoApps </li><li>Start the program: arduino-app-cli app start color_tracked_plz </li><li>The message 🚀 System Ready indicates a successful startup. </li></ol>Access the Control Interface: <ol><li>After startup, visit http://&lt;robot-IP&gt;:7000 in a web browser. </li><li>The status light in the top-right corner turns green to confirm a successful connection. </li><li>The video area should display a real-time camera feed (with a crosshair reticle). </li></ol>Select Tracking Color: <ol><li>Find a brightly colored object (e.g., yellow tennis ball, red cup). </li><li>Drag the Hue slider in the Tracking Color area, or click a preset color block. </li><li>The color circle on the left previews the selected color in real time. </li><li>The current HSV range is displayed at the bottom right (e.g., H:20~40 S:80~255 V:80~255). </li></ol>Enable Auto-Tracking: <ol><li>Turn on the Auto-Tracking switch. </li><li>The Pan/Tilt sliders will gray out (under automatic control). </li><li>Place the target object in the camera view; the pan-tilt mechanism will track it automatically. </li><li>Move the object and observe how the pan-tilt follows. </li></ol>PID Tuning (Advanced): <ol><li>Click PID Tuning to expand the panel. </li><li>First test the 4 preset schemes to experience different parameter effects. </li><li>If tracking is not ideal with default values, fine-tune using these guidelines: </li></ol>- Slow response → Increase Kp - Jitter / oscillation → Increase Kd, or decrease Kp - Steady offset from center → Increase Ki - Jitter near center → Increase dead zone Manual Mode: <ol><li>Turn off the Auto-Tracking switch. </li><li>Drag the PAN slider to control the camera left/right view. </li><li>Drag the TILT slider to control the camera up/down tilt. </li><li>Default position: PAN=90° (straight ahead), TILT=90° (horizontal). </li></ol>Servo Center Calibration (For First Assembly): <ol><li>Click the Start Calibration button → Both axes automatically center to 90°. </li><li>Auto-tracking is forced off to ensure unobstructed manual adjustment. </li><li>Use the ±1° / ±5° fine-tune buttons to precisely adjust Pan and Tilt to the physical servo center. </li><li>Mount the servo horn (cross or single arm) at the current angle. </li><li>Click Finish Calibration to exit; the tuning controls will hide. </li><li>For recalibration, click 🎯 Reset to 90° at any time. </li></ol> <figure><img src="https://projects.arduinocontent.cc/2a440034-f5e4-4b21-8a2e-e68019aeadfe.png"/><figcaption></figcaption></figure> <h3>Troubleshooting FAQ</h3>Q: The gimbal doesn't respond after enabling auto-tracking? A: Check if the tracking color matches the target object. Drag the Hue slider near the target object's color and ensure the object is visible in the video feed. Recognition may fail if lighting is too dim or the object's color is not saturated enough. Q: The gimbal shakes violently during tracking? A: PID parameters may be too aggressive. Switch to the "Conservative (Smooth)" preset, or manually decrease Kp and increase Kd and Dead Zone. Q: "Wrong type parameter" error in the log? A: The PID output float value was passed to Bridge without int conversion. Ensure int() conversion is applied before Bridge.call("set_pan", current_pan) . Q: Video feed is stuttering or has high latency? A: Base64 streaming consumes significant bandwidth at high resolutions. Ensure camera resolution is set to 320×240 and JPEG quality does not exceed 70. WiFi signal strength also affects streaming frame rate. Q: Camera shows a blank frame? A: Check if the camera is recognized by the system. Run ls -l /dev/video* via SSH — if no device nodes exist, try reinserting the USB camera. Q: Red color tracking doesn't work well? A: Red straddles the H=0 boundary in the HSV color wheel. The code already handles this case (dual mask merge), but large red background areas in the environment may interfere with tracking. Try narrowing the Hue tolerance or adjusting S/V lower bounds. </body></html>

Visual Tracking Gimbal

This project is featured in the Robotics Hub. This is a very interesting open-source project. Its main features include: network monitoring and control, running a web server, parsing HTTP/Socket requests, and handling MJPEG video stream transmission.