Package: dlf-nova-play-my-own-music
DLF-Nova Play My Own MusicTheDLF novaplaylistis just a few hours long, leading to a lot of repetitions if you listen long enough. This is kind of annoying, because the news-bits are quite interesting.
So I programmed a script that detects music/news and switches to spotify during the music parts on the radio.This project is work in progress, I want to make it easier to use.RequirementsVLChttps://www.videolan.org/vlc/spotify accounthttps://www.spotify.comspotify client ID:https://developer.spotify.com/dashboard/applicationspython 3.7Get startedTo use the Spotify API aclient idandclient secretare needed, it can be createdhereSaveSPOTIPY_CLIENT_IDin your environment (example1a2ef9787cc1638261545d6dadb2315)SaveSPOTIPY_CLIENT_SECRETin your environment (example2c12e5161ace0476c8f2abc70a924ac76)Find your spotify username and insert it inspotify_usernameinonline_prototype.pyInstall withpip install DLF-Nova-Play-My-Own-Musicdlfnova -su my.spotifyusernameto startAfter some setup messages it should generate logs like this19:20:34: Class:  music, prob.:  96%, filtered signal:   0%, AI performance:  3.3Hz
19:20:36: Class:  music, prob.:  66%, filtered signal:   0%, AI performance:  3.5Hz
19:20:39: Class:   news, prob.:  84%, filtered signal:  70%, AI performance:  3.3Hz
19:20:42: Class:   news, prob.:  99%, filtered signal:  91%, AI performance:  3.3Hz switch to radio
[00000282eb2c49d0] prefetch stream error: unimplemented query (264) in control
19:20:44: Class:   news, prob.: 100%, filtered signal:  97%, AI performance:  2.8Hz
19:20:47: Class:   news, prob.: 100%, filtered signal:  99%, AI performance:  3.4Hz
19:20:49: Class:   news, prob.: 100%, filtered signal:  100%, AI performance:  3.3HzTechnical detailsI trained a simple deep learning model to classify an audio single as "music" or "news", using this classifications, I switch between my spotify device and the online radio stream.DatasetThe dataset consists of:500MB of DLF nova podcasts fromhttps://www.deutschlandfunknova.de/podcaststhe DLF nova spotify playlistusingspotdlstream chunksTo split training (80%) and test (20%) datasets, the files are separated randomly before splitting into chunks.
The mp3 files are split into 10s chunks (no overlapping) at 44100Hz.
Each chunk is converted to the feature vector, making the dataset files relatively small.Input vector/featuresBased on thisthis medium articleandthis articleI chose three features:MFCC(related to speech/music separation, fromlibrosa),GFCC(typical used for speaker recognition, fromspafe) andonset_strength(related to beat detection in music, fromlibrosa). It results in a 256 dimensional vector.# feature vectormfccs=librosa.feature.mfcc(y=audio,sr=sample_rate,n_mfcc=20)mfccs_scaled=np.mean(mfccs.T,axis=0)try:gfccs=gfcc(audio,num_ceps=20)gfccs_scaled=np.mean(gfccs,axis=0)exceptException:gfccs_scaled=np.zeros(20)hop_length=2048oenv=librosa.onset.onset_strength(y=audio,sr=sample_rate,hop_length=hop_length)returnnp.hstack((mfccs_scaled,gfccs_scaled,oenv))Whereaudiois always a 10 second audio chunk at 44100Hz sample rate.Deep learning modelI used theKerasmodel fromthis medium articlemodel=Sequential()model.add(Dense(256,input_shape=(input_size,)))model.add(Activation("relu"))model.add(Dropout(0.5))model.add(Dense(256))model.add(Activation("relu"))model.add(Dropout(0.5))model.add(Dense(num_labels))model.add(Activation("softmax"))model.compile(loss="categorical_crossentropy",metrics=["accuracy"],optimizer="adam")whereinput_sizeis 256.It trained for 100 epochs, default learning rate, batch size 256 and saved the weights with the lowest validation loss.PerformanceIt achieved 99.8% accuracy during training and 98% for the validation set.SwitchingThe scriptonline_prototype.pyregisters through the spotify API and requests with OAUTH permission to see the devices and modify the player state. It downloads the DLF stream in 40kB chunks and outputs an classification every 2 to 3 seconds. This signal is smoothed and it triggers switching events. The radio is streamed via VLC.Known issuesSome songs are classified as news sometimes, due to the simple model. Happens with rap especially. The news sometimes have a beat in the background as well.[00000282eb2c49d0] prefetch stream error: ...logs, they come from VLCTODOTrain better model for fewer switching errorsFading the sound during the switch event