Comparative Analysis of cVAE, cGAN, and LSTM Models for Music Generation Based on Weather Conditions

Generative Artificial Intelligence is gaining popularity every day and uses contextual data to personalize and enhance user experiences. This paper explores music generation that is conditioned on weather. It influences musical compositions by connecting MIDI music data with corresponding weather attributes, for example sunny and cloudy weather. In this paper, three generative models are compared for the task of weather based music generation; Conditional Variational Autoencoder (cVAE), a Conditional Generative Adversarial Network (cGAN) and Long Short-Term Memory (LSTM) network. Mentioned models are implemented and trained on combination of large MIDI corpus with historical weather information. Performance of models in this paper are evaluated using metrics that capture musical diversity and quality, such as pitch range, unique pitches and pitch variance, and for fidelity to real data is measured by mean squared error and KL divergence. Results of the paper showed that the cVAE produced the most diverse music for music that is context sensitive. Using cVAE in this approach helped the model in achieving the widest pitch range and variety of notes with low error. Runner ups in this comparison was cGAN that generated relevant music, but with slightly less diversity while LSTM showed higher error and inability to integrate the weather context. Contributions to this work is made of the first comparative analysis of these architectures for weather based music generation, exploration of their strength and limits and evidence that environmental data can be used to influence AI generated music. Results showed the relevance of blending weather data with music generation which can serve as a foundation for applications such as adaptive game soundtracks, mood based music therapy or dynamic background music that is being generated according to the user's environment.