Offline ESP32 Voice Assistant with Gemini AI

Google Gemini

DOIT ESP32 DevKit v1

Diode Led Red 5mm

Breadboard 100x160

MAX98357A

Breadboard Jumper Wire Pack (200mm&100mm)

5mm Green LED

0.96 OLED SSD1306

Speaker 3W  4 ohms

<html><body><h3><p></p><figure><iframe src="https://youtu.be/EO-1ZwN6LNo"></iframe></figure><p></p></h3><h3>Story </h3><p>In an era of always-listening smart devices, I wanted to create an alternative: a voice assistant that is deliberately controlled by the user and doesn't rely on a microphone. This project is my ESP32 Voice Assistant, a powerful, privacy-focused device that uses a button to trigger a request to a server. </p><p>The server, running on Python, uses Google's Gemini API to generate responses and then converts that text into speech using gTTS. The audio is streamed back to the ESP32, which plays it through an I2S amplifier and speaker. This design is not only privacy-conscious but also highly efficient with AI token usage. It's a fantastic project for anyone looking to build a self-contained AI device. </p><h3>Pinout and connection diagram</h3><figure><img src="https://projects.arduinocontent.cc/201fca1a-7fc8-4547-b67c-b577a931c983.jpg"/><figcaption></figcaption></figure><p> </p><figure><img src="https://projects.arduinocontent.cc/4e77215a-514e-44b6-8cb4-1c6f6e566d03.png"/><figcaption></figcaption></figure><p>(All GND pins should be connected to a common ground rail.) </p><h3>Instructions</h3><p>All the code could be found at <a href="https://github.com/circuitsmiles/ai-chat-bot-v0.1">github repo </a></p><ol><li>Step 1 The Build: Follow the schematic above to connect all your components on the breadboard. Pay close attention to the I2C and I2S pinouts on the ESP32. </li><li>Step 2 Flash the Arduino Code </li></ol><pre><p>Follow the steps in the arduino_firmware.ino file comments to install the ESP32 board manager and required libraries. Select your board and port, then upload the code to your ESP32. <br/></p></pre><ol><li>Step 3 Set Up the AI Server: This Python server acts as the brain of the operation. </li></ol><pre><p>Install Dependencies: Run pip install -r requirements.txt to get all the necessary Python libraries. <br/></p></pre><pre><p>Get Your API Key: Obtain a GEMINI_API_KEY from the Google AI Studio. <br/></p></pre><pre><p>Run the Server: Run python server.py to start the server. <br/></p></pre><h3>Operation</h3><p>Your Offline AI Is Ready! </p><ol><li>Power Up: Connect power to your ESP32. It should connect to Wi-Fi, and the OLED will display "Ready" with the green LED solid. </li><li>"Next" Button: Press this button to cycle through the predefined phrases on the OLED display. </li><li>"Speak" Button: When you've found the phrase you want, press "Speak." </li><li>The OLED will show "Thinking..." (red LED solid) as the ESP32 contacts the server. </li><li>Once the server responds, it will switch to "Speaking..." (green LED solid, red LED blinks) as the audio plays. </li><li>After playback, it returns to "Ready." </li></ol><p>The Token-Saving Trick: Remember, the Python server deliberately limits the length of the Gemini response to keep your API token usage (and potential costs!) down. It's an efficient little system! </p><h3>Conclusion</h3><p>This project is a great way to explore the world of AI-driven hardware while maintaining a focus on user control and privacy. The design is easily expandable with more sensors, prompts, or even a custom web interface. </p><p>I hope you've enjoyed this build. Please leave a comment if you have any questions! </p><p></p></body></html>

Code Repo

Pinout

Connection Diagram

Offline ESP32 Voice Assistant with Gemini AI

A button-controlled AI device that talks back (no microphone needed!)