HackMIT 2026

Keiko

Mini breadboard - White

Arduino® UNO™ Q 4GB

10 jumper wires 150mm male

Arduino App Lab

<html><body><h3>Inspiration</h3><p>A couple of years ago, we [Matthew and Taha] played around with training a machine learning model on Cornell University's North Atlantic Right Whale noise dataset. After doing some digging around, we found that whale conservation was a huge problem with around two-thirds of all deaths attributed to human interactions, namely ship collisions or net entanglments. Initially, we decided to create a product that could prevent (what we saw) as easily preventable whale deaths if the location was known. But more than that, both of us found the idea of finding the position of whales - and by extension any marine animal - fascinating. The possibility is a worldwide open-source database available for conservationists, academics, and marine animal enthusiasts.</p><p></p><h3>What it does</h3><p>The entire system tracks the location of whales (and marine animals) through the sounds they produce through the water. A hydrophone picks up continuous underwater audio, sending it to a 3D-printed SLA printed buoy that sends it to a cloud to get processed through a custom ensemble model. If the model confidently detects a whale then it will update the website along with the open-source database on Github.</p><p></p><h3>How we built it</h3><p>The first large hurdle we had to overcome was the hardware aspect. We had to 3D-print the buoy to fit compactly and settled on using SLA for a water-tight finish. We then built the hardware schematics using an Arduino UNO Q 4GB (ABX00173) with a small breadboard and wires. Then came construction of the DIY hydrophone itself. Due to not being allowed to access a hydrophone, we used the Piezo Disc Transducers to convert physical vibrations to electrical sensing. In order to maximize its sensitivity and ensure waterproofing, we utilized hot glue to encase the transducer in it. After wiring and finishing everything, we went through a couple variations of machine learning models before settling down on our own custom ensemble model tailored towards the specificities of whale vocals. </p><p>We scraped Watkins Marine Mammal Sound Database, ReefSet v1.0, Cornell/Marinexplore Whale Detection Challenge, Orcasound Pod.Cast, ToadFishFinder v4, BEANS hiceas minke boing set for the training data. We then trained the ensemble model on 5-fold cross-validation achieving an AUROC score of 0.9816. We built a website and open source database to accompany this then tied everything up together. </p><p></p><h3>Challenges we ran into</h3><p>There were many challenges that we faced, especially along the hardware side. Due to not being able to access a pre-made hydrophone, we had to construct one from scratch which proved to be incredibly difficult. It required testing, breaking, and threading many wires together or apart to create a custom op-amp and piezo disk.</p><p>Along with this the data on whale vocalizations is heavily limited with high class imbalance. Some whale species such have unusually high rates such as the North Atlantic Right, Humpback, or Bowhead whales, while others have only a couple audio clips due to their rarity. Combined with this, the data was scraped from many datasets causing a large volume of data. Pre-processing was necessary to cut down and ensure cleanliness of the data before running it through the model. We initially saved each as a mel-spectrogram, but realized that information can be stored as a gray-scaled tensor before running it through the model, which cut down time training immensely. </p><p></p><h3>Accomplishments that we're proud of</h3><p>We were proud of many of our features in both the software and hardware components. On the software side, we took in an extremely large volume of data of audio clips all sourced from different data sets with their own clip times, recording methodology, and frequency bandwith captured. Parsing and cleaning through the data was a major milestone, and our model uses one of the most robust sets of data. Along with this, we created our own custom ensemble model, that included multiple branches to process both 1D and 2D branches which was an insight we realized was useful after looking at audio of dolphin clicks.</p><p></p><h3><strong>What's next for our project</strong></h3><p>We want to expand our project to not just whales, but all marine animals. Currently, the closest relatives are dolphins and porpoises but due limitations in audio data about these animals it makes it incredibly difficult to measure. However, the buoy provides the potential for continuous learning by training the model live on data that it is consuming - double checked by researchers to ensure alignment. With multiple buoys across the world this could quickly produce an enormous amount of data. </p></body></html>

Keiko

Tracking whales in Cape Cod using underwater vocalizations!