XR2ino: the first app to visualize Arduino data on Meta Quest in Mixed Reality

Unity

Arduino IDE

Arduino® UNO R4 WiFi

Meta Quest

<html><body> <figure><iframe src="https://www.youtube.com/watch?v=-qsrfZyFsLg&amp;ab_channel=Metagate"></iframe></figure> <h3>The Idea</h3><a href="https://discover.themetagate.it/en?srsltid=AfmBOooLApw-peNGrk8VDjxz6vdVRuirzSVY6DVyhvRxB1apvg6hy60n">Metagate </a>is an Italian startup developing Mixed Reality experiences for the Meta Quest 3 and 3S . Our team creates customized applications for clients and original experiences released directly by the company. In this case, we wanted to experiment with using sensors and Arduino to make Mixed Reality even more engaging by connecting it to the physical world, embracing the potential of Extended Reality. <h3>The Process</h3>Our applications are built on Unity , so the first step was figuring out how to easily connect Arduino to Unity to retrieve sensor data. After experimenting with different methods, we decided that Bluetooth was the most straightforward option. For this project, we used an Arduino Uno R4 WiFi, as it was readily available to us. However, the general availability of Bluetooth modules for Arduino made this approach a strong choice in terms of usability. The next step was deciding which data to send, so we could create the Arduino and Unity code. From the beginning, our goal was to build an application that the Arduino community could use to easily experiment with Mixed Reality. This meant the code we released had to be simple to read and modify. This was the biggest challenge because connecting the headset requires working on both the Arduino and Unity code. We wanted the app to be adjustable by modifying only the Arduino code. Achieving this required extensive testing, from variable selection (currently all floats ) to creating and configuring the JSON file that sends sensor data to Unity. As you can see from the released code, some sections are fixed and depend on the Unity connection. However, by following the comments in the code, we’re happy to confirm that the final application can now be adjusted directly from Arduino . <h3>UI/UX Design</h3>After defining what we wanted the app to do, we had to decide how to display the data. For this project, we developed a straightforward Mixed Reality app that we plan to improve based on user feedback. Currently, the app features panels that can be grabbed and resized , and their names can be changed through the Arduino code. Each panel's appearance can be customized via an edit menu, with four visual styles to choose from. <figure><img src="https://projects.arduinocontent.cc/7750456b-b293-4cf7-898e-586692a0440a.gif"/><figcaption></figcaption></figure> Settings also allow users to adjust the soundtrack and sound effects volume. The number of panels spawned in the experience corresponds to the number of variables in use on Arduino, up to a total of ten . Panels can be opened and closed through a horizontal menu , which can be activated and deactivated by pinching the left-hand index finger and thumb. <h3>Implementations and Support</h3>The XR2ino application is available on the <a href="https://vr.meta.me/s/1KVRVO3efRSOYcY"></a><a href="https://vr.meta.me/s/1KVRVO3efRSOYcY">Meta Store</a>. We’ve published the Arduino code on GitHub under an open license. For questions or support, you can join our <a href="https://discord.com/invite/bghRyt4DRs">Discord </a>support channel or reach out to us <a href="https://discover.themetagate.it/pages/xr2ino">directly </a>. We can’t wait to see how you use the app! </body></html>

XR2ino: the first app to visualize Arduino data on Meta Quest in Mixed Reality

XR2ino is a Mixed Reality app developed by Metagate that permits the BLE connection between Arduino and MR headsets.