r/Cetacea • u/antonkulaga • 12h ago
We built an orchestra using whale and dolphin recordings from different oceans
Enable HLS to view with audio, or disable this notification
Livia Zaharia and I developed Whale & Dolphin Orchestra, a science-art project that combines recordings of different cetaceans into music.
We selected short excerpts of humpback songs, dolphin whistles, sperm-whale click patterns and orca calls from open research collections: PANGAEA, NOAA DCLDE, OpenWhistle and the Dominica Sperm Whale Project datasets.
You pick recordings on a world map and arrange them together or in sequence. We measure the timing and pitch contours of the calls, map tonal contours to notes and clicks to percussion, and use the combined audio guide to generate music with an AI model.
I recorded a demo: https://youtu.be/CI_CTcqpJ_Q
Try the prototype: https://whales.liviazaharia.com/
We made the project as a proposal for the HIFMB × HWK ArtWaves residency. Livia also made a jewelry generator that uses the recordings’ amplitude and frequency to shape her ring designs. She drafted a physical installation for the project.
Everything is open-sourced at https://github.com/Livia-Zaharia/whales
We’re still deciding which generative music model to use by default. In the video you can hear ACE-Step and Stable Audio 3, which one do you like more?
