metadata

title: Audio and Video Content Analyzer
emoji: 🎥
colorFrom: blue
colorTo: green
sdk: streamlit
python_version: '3.8'
sdk_version: 1.27.2
app_file: app.py
pinned: false

Summarize Audio and Video with Semantic Retrieval, Chatbot, and LLM

Description

This project is an all-in-one solution for audio and video content analysis:

Summarization: Generates concise summaries using advanced Natural Language Processing, powered by HuggingFace Transformers.
Semantic Retrieval: Enables you to find specific words, phrases, or segments, thanks to Whisper Timestamped by Linto.
Chatbot Interface: Features a chatbot that can answer queries about the audio or video content, leveraging Language Models for Machines (LLM).

The project is built using Python and integrates various libraries including Whisper Timestamped, HuggingFace Transformers, and Streamlit for a seamless user experience.

Project Structure

.
├── Dockerfile                   # Dockerfile for setting up the environment
├── LICENSE                      # License file
├── README.md                    # This README file
├── YoutubeAudios                # Directory containing YouTube audio files
├── _requirements.txt            # Requirements file
├── app.py                       # Streamlit application file
├── config.py                    # Configuration file
├── keyword_retriever            # Module for keyword retrieval
│   └── keyword_retreiver.py     # Keyword retriever script
├── logger.py                    # Logging utility
├── notebooks                    # Jupyter notebooks for development and testing
├── pdf_test.py                  # PDF testing script
├── query_service                # Query service module
│   └── query_engine.py          # Query engine script
├── requirements.txt             # Requirements file
├── resource_loader              # Resource loader module
│   ├── json_loader.py           # JSON loader script
│   ├── linkedin_loader.py       # LinkedIn loader script
│   ├── uploaded_video_loader.py # Uploaded video loader script
│   ├── video_loader_interface.py# Video loader interface script
│   └── youtube_loader.py        # YouTube loader script
├── summarization_service        # Summarization service module
│   └── summarizer.py            # Summarizer script
├── transcription_service        # Transcription service module
│   └── transcriber.py           # Transcriber script
└── utils.py                     # Utility functions

Prerequisites

Python 3.8+
Docker (optional)

Built With

Llama_index Framework for LLM application
Whisper Timestamped - For semantic retrieval and timestamping
HuggingFace Transformers - For summarization and NLP
Streamlit - For the web interface

Setup and Installation

There are two methods to get the project up and running:

Method 1: Using Docker

Clone the repository.
Navigate to the project directory.
Build the Docker image:
```
docker build -t summarizer .
```
Run the Docker container:
```
docker run -p 8501:8501 summarizer
```
Open your web browser and go to http://localhost:8501.

Method 2: Using Python Environment

Clone the repository.
Navigate to the project directory.
Install the requirements:
```
pip install -r requirements.txt
```
Run the Streamlit app:
```
streamlit run app.py
```
Open your web browser and go to http://localhost:8501.

Using Python Environment

Clone the repository.
Navigate to the project directory.
Install the requirements:
```
pip install -r requirements.txt
```
Run the Streamlit app:
```
streamlit run app.py
```

Usage

Open the Streamlit app in your web browser.
Follow the instructions on the screen to upload or specify your audio/video content.
Click "Submit" to generate a summary, perform semantic retrieval, or interact with the chatbot.

Contributing

Please read CONTRIBUTING.md for details on our code of conduct, and the process for submitting pull requests.

License

This project is licensed under the MIT License - see the LICENSE.md file for details.