HealthLingo | AI Agents Enabling Multilingual Doctor consultation via WhatsApp
Tags: devchallenge, twiliochallenge, ai, twilio
Published: 2024-06-21 17:57:07

*This is a submission for [Twilio Challenge v24.06.12](https://dev.to/challenges/twilio)*

## What I Built
<!-- Share an overview about your project. -->
People can directly contact doctors from WhatsApp in their native language.
Based on patients query a suitable doctor from the list of doctor will be selected by AI.
The Doctor replies in English while the patient's will receive the reply in their preferred language.

## Demo
![Small gif showing translation](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/i9zr61tlzas43j0n2nka.gif)

Video explanation of the project and the codebase
{% embed https://youtu.be/5pdg0Vm-4X8 %}
<!-- Share a link to your app and include some screenshots here. -->

![whatsapp sandbox link](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/o24nm2gt4cpbo7algwpo.PNG)

## Working of the App
The app consists of two layers
1. Business layer
2. AI layer

The business layer is responsible for handling the incoming msg and Database interactions while the AI layer consists of multiple small agents which interacts with Open AI API. I have used MongoDB as the database to store information like patient details, chat summary and doctor information.

![explanation of code base](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/67csuz8czb0pdjx8yz14.png)

## Twilio and AI
<!-- Tell us how you leveraged Twilio’s capabilities with AI -->
### AI
- Multiple Agents for handling specific tasks using OpenAI's GPT-4o model
- Translate Agent
- Chat Summary Agent
- Two Agents for Details Collection
     - Basic Details Agent
     - Medical Details Agent

### Twilio
- Twilio Whatsapp service.  
- Initially the app was deployed as Twilio Functions but was as I added more conditions I was hitting timeouts sometimes. So yeah, I just deployed in AWS using docker. And yes only one Twilio service.

## Additional Prize Categories

<!-- Does your submission qualify for any additional prize categories (Twilio Times Two, Impactful Innovators, Entertaining Endeavors)? Please list all that apply.  -->
**Impactful Innovators**: People who are visiting a country for first time or immigrants who still find it difficult to express especially their medical condition in the foreign language can use this kind of app to communicate with doctors and express their symptoms and conditions.
Also the AI will pick the right doctor based on their condition.
The privacy of both doctor and patient is maintained as no phone numbers are shared.

> The vision of such a app in real life would be
> - Social service mined Doctors signing up for such a service and give some details about their specialty
> - Patients who are looking for answers for some rare or specific condition could potentially find the right doctor and explain their problem in the language they are comfortable with

If you've made it this far, thanks for reading the post! Leaving a like would mean a lot, as it supports my work and encourages me to keep creating more content. Have a great day !

### GitHub
{% embed https://github.com/ashiqsultan/twilio-whatsapp-ai-bot %}

<!-- Team Submissions: Please pick one member to publish the submission and credit teammates by listing their DEV usernames directly in the body of the post. -->

<!-- Don't forget to add a cover image (if you want). -->