Azure Trial Hackathon - EmotionalSound - perfect spotify song for your emotions
Tags: azuretrialhack, hackathon, showdev, javascript
Published: 2022-03-08 00:47:24

**Project Links**
[Demo](https://black-desert-0f8deaa10.1.azurestaticapps.net)
[Frontend](https://github.com/marco-bertelli/emotional-sound-frontend)
[BackEnd](https://bitbucket.org/becodebg/emotionsound/src/master/)

**Overview of our project**
This project goal is to suggest real Spotify songs by understanding face emotions.
This project help you to find perfect sound for your actual mood!
You only have to enter in the demo, take a photo of your face and see which new Spotify song **EmotionalSound** suggest for you.
If you click the button you can real time listen the song on Spotify!.
Every 5 minutes the back-end search for new real time released and add it to song list.

**Preview**

![Azure live demo](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0qa9mfix94w43r5y5h0r.gif)

**Mobile**
This project have full mobile support (also Safari), and if you click the Spotify Button you can open the Song in your mobile application

**Submission Category**:
The category for this project is: AI Aces.
I am using **Azure Face API** for detecting the face emotions and a custom classification neural network made with **Tensorflow** and compiled to be used by **TensorFlowJs**.

In addition, to publish the frontend i am using an **Azure Static Webapp** (great Angular support) and for the backend an **Azure App Service** with custom Docker images.

For docker images i use **Azure Container Registry**.

For frontend i have implemented CI/CD with GitHub Actions.
Instead for back-end i prefer to use Bitbucket with Custom Pipelines for build And Deploy the Docker images.

**Neural Network Model**:
the neural network model is a quite simple classification model that use the normalized data from **Spotify API** to understand the mood of a song Based on Song Background Frequency and other values.
To better Understand this part i suggest you to visit and try the **Google Collab** project here!: [Collab Url](https://colab.research.google.com/drive/1Orj8wboKwYqoJhAyEQ5CU19va4Vtw7t-?usp=sharing)

**Tech Stack**
The frontend is written in Angular 11.
From the homepage of the application,you can take picture from your webcam and the image is send as a base64 string directly to the Backend, through an API.

**YOUR DATA ARE NOT STORED IN ANY WAY**

The backend is written in **Node js 16** and **Express**.
For the database to store the song retrived by Spotify Api i use **MongoDb 4.4**.
an example of Database doc is: 

```
{ 
    "_id" : ObjectId("6224cff6eff1e8fefdf67e45"), 
    "name" : "Don’t Forget My Love", 
    "__v" : NumberInt(0), 
    "image" : "https://i.scdn.co/image/ab67616d0000b2732cfa4b40e2d7b56fb171dd53", 
    "mood" : "2", 
    "spotifyUrl" : "https://open.spotify.com/track/5L7ddKqhj6xZFeb8ZOoI4r"
}
```
For interact with Mongodb i use **mongoose**, an npm library for easy wrap all.
The API receives the base64 string image from the Frontend and sends it to the Azure Face API directly using a simple **axios** call.
The Back-end get the response from azure and understand the max mood (es happiness) and send it to the client with the saved songs generated from The Neural Network and Spotify Api.
Every 5 minutes the back-end get new spotify releases from the API and **Neural Network** analise the data for understand the mood of the new song, and save it to **Mongo**.
On the results page, we display a “dynamic” mood label with the founded songs and a button to real-time listen it.

**Azure resources**
This is the list of our services deployed on Azure:

![Azure resources](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7gaxcz978rsg5bpzlhue.png)

**CI/CD**
I have automated the Frontend builds through CI/CD thanks to two GitHub Actions generated from Azure.

For the back-end i need a more complicated one that make a docker image and release it on the **Azure Container Registry**

```
image: atlassian/default-image:2
clone:
  depth: full
pipelines:
  branches:
  
    master:
      - step:
          name: Deploy to Azure
          caches:
            - docker
          services:
            - docker
          script:
            - docker login emotionalsound.azurecr.io -u $AZURE_USER -p $AZURE_PASS
            - docker build -t emotionalsound.azurecr.io/prod:latest .
            - docker push emotionalsound.azurecr.io/prod:latest
definitions:
  services:
    docker:
      memory: 3072  # increase docker memory for unit testing into the pipe
```

**RoadMap**
the project is official on Beta stage, my roadMap is to improve before the end of **Hackathon** the import system to get more song to suggest, and random shuffle it for get more result variety.

If anyone encounters any kind of bug, please report it in the comments

**About Me**
My name is Marco Bertelli, i am a 21 years old back-end developer that work in [Runelab](https://www.runelab.it).
This is my first **Hackathon**, if you have suggestions or comments are Welcome.
If you like the project **Comment** and **Share** this article on Linkedin or with your friend!

If anyone has any questions or suggestions, don't hesitate to leave a comment!