Diffusion-Based Sound Synthesis in Music Production
In this paper, we explore the usability of generative artificial intelligence in music production through the development of a digital instrument that incorporates diffusion-based sound synthesis in its sound generation. Current text-to-audio models offer a novel method of defining sounds, which we aim to render utilizable in a music-production environment. Selected pretrained latent diffusion models, enable the synthesis of playable sounds through textual descriptions, which we incorporated into a digital instrument that integrates with standard music production tools. The resultant user interface not only allows generating but also modifying the sounds by editing model and instrument-specific parameters. We evaluated the applicability of current diffusion models with their parameters as well as the fitness of possible prompts for music production scenarios. Adapting published diffusion model pipelines for integration into the instrument, we facilitate experimentation and exploration of this innovative sound synthesis method. Our findings show that despite facing some limitations in the models’ responsiveness to specific music production contexts and the instrument’s functionality, the tool allows the development of novel and intriguing soundscapes. The instrument and code is published under https://anonymous.4open.science/r/WaveGenSynth/
Mon 2 SepDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
14:00 - 15:30 | |||
14:00 22mTalk | Demo: A Geometric Approach to Generate Musical Rhythmic Patterns in Haskell FARM Xavier Góngora Universidad Nacional Autónoma de México Link to publication DOI | ||
14:22 22mTalk | Diffusion-Based Sound Synthesis in Music Production FARM Pierre-Louis Suckrow Berlin University of the Arts, Technical University of Berlin, Christoph Johannes Weber University of Television and Film Munich, LMU Munich, Sylvia Rothe University of Television and Film Munich | ||
14:45 10mTalk | Demo: Functional Sound Design FARM Stefano Panelli Conservatorio A. Vivaldi Alessandria | ||
14:55 35mTalk | A Progressive-Adaptive Music Generator (PAMG): An Approach to Interactive Procedural Music for Videogames FARM Alvaro Lopez Duarte University of California Riverside |