initial commit

.gitignore

@@ -117,6 +117,9 @@ dmypy.json
 # MacOS specific
 .DS_Store
 
+# Specific model files
+yolov8n.pt 
+
 # Keep empty models dir
 models/*
 !models/.gitkeep

README.md

@@ -1,5 +1,5 @@
 ---
-title: Computer Vision Playground
+title: YOLO v8 Playground
 emoji: 🦀
 colorFrom: indigo
 colorTo: blue
@@ -10,218 +10,13 @@ pinned: false
 license: mit
 ---
 
-# Computer Vision Playground
+# Yolo v8 Playground
 
-This Streamlit application streams video from the webcam, analyzes facial sentiment, and displays the results in real-time. It serves as a playground for computer vision projects, with an example facial sentiment analysis demo.
-
-## How to Use
-
-1. Clone the repository.
-2. Ensure you have the necessary packages installed: `pip install -r requirements.txt`
-3. Run the application: `streamlit run app.py`
-
-## Create Your Own Analysis Space
-
-Follow these steps to set up and modify the application for your own image analysis:
-
-### Step 1: Clone the Repository
-
-First, you need to clone the repository to your local machine. Open your terminal or command prompt and run:
-
-```sh
-git clone https://huggingface.co/spaces/eusholli/computer-vision-playground
-cd computer-vision-playground
-```
-
-
-### Step 2: Install Dependencies
-
-Make sure you have Python installed on your machine. You can download it from [python.org](https://www.python.org/).
-
-Next, install the required packages. In the terminal, navigate to the cloned repository directory and run:
-
-```sh
-pip install -r requirements.txt
-```
-
-This will install all the necessary libraries specified in the \`requirements.txt\` file.
-
-### Step 3: Run the Application
-
-To start the Streamlit application, run:
-
-```sh
-streamlit run app.py
-```
-
-This will open a new tab in your default web browser with the Streamlit interface.
-
-## Using the Application
-
-#### Webcam Stream
-
-- Allow access to your webcam when prompted.
-- You will see the live stream from your webcam in the "Input Stream" section.
-- The application will analyze the video frames in real-time and display the sentiment results in the "Analysis" section.
-
-#### Uploading Images
-
-- In the "Input Stream" section, under "Upload an Image", click on the "Choose an image..." button.
-- Select an image file (jpg, jpeg, png) from your computer.
-- The application will analyze the uploaded image and display the sentiment results.
-
-#### Image URL
-
-- In the "Input Stream" section, under "Or Enter Image URL", paste an image URL and press Enter.
-- The application will download and analyze the image from the provided URL and display the sentiment results.
-
-#### Uploading Videos
-
-- In the "Input Stream" section, under "Upload a Video", click on the "Choose a video..." button.
-- Select a video file (mp4, avi, mov, mkv) from your computer.
-- The application will analyze the video frames and display the sentiment results.
-
-#### Video URL
-
-- In the "Input Stream" section, under "Or Enter Video Download URL", paste a video URL and press Enter.
-- The application will download and analyze the video from the provided URL and display the sentiment results.
-
-## Customize the Analysis
-
-You can customize the analysis function to perform your own image analysis. The default function \`analyze_frame\` performs facial sentiment analysis. To use your own analysis:
-
-1. Replace the contents of the \`analyze_frame\` function in \`app.py\` with your custom analysis code.
-2. Update any necessary imports at the top of the \`app.py\` file.
-3. Adjust the \`ANALYSIS_TITLE\` variable to reflect your custom analysis.
-
-Example:
-
-```python
-ANALYSIS_TITLE = "Custom Analysis"
-
-def analyze_frame(frame: np.ndarray):
-    # Your custom analysis code here
-    ...
-```
-
-### Troubleshooting
-
-If you encounter any issues:
-
-- Ensure all dependencies are correctly installed.
-- Check that your webcam is working and accessible.
-- Verify the URLs you provide are correct and accessible.
-
-For more detailed information, refer to the comments in the \`app.py\` file.
-
-
-### Debugging using Vscode
-
-If you are using Vscode as your IDE you can use the following launch.json file to debug the current file (e.g. app.py) in your editor.
-
-```json
-{
-    "version": "0.2.0",
-    "configurations": [
-        {
-            "name": "Python:Streamlit",
-            "type": "debugpy",
-            "request": "launch",
-            "module": "streamlit",
-            "args": [
-                "run",
-                "${file}",
-                "--server.port",
-                "2000"
-            ]
-        }
-    ]
-}
-```
-
-
-# How to Create a New Huggingface Space and Push Code to It
-
-## Step 1: Create a New Huggingface Space
-1. Log in to your [Huggingface](https://huggingface.co/) account.
-2. Go to the [Spaces](https://huggingface.co/spaces) section.
-3. Click on the **Create new Space** button.
-4. Fill in the details for your new Space:
-    - **Space name**: Choose a unique name for your Space.
-    - **Owner**: Ensure your username is selected.
-    - **Visibility**: Choose between Public or Private based on your preference.
-    - **SDK**: Select the SDK you will use (in this case`streamlit`).
-5. Click on the **Create Space** button to create your new Space.
-
-## Step 2: Change the Local Git Remote Repo Reference
-1. Open your terminal or command prompt.
-2. Navigate to your local project directory:
-    ```bash
-    cd /path/to/your/project
-    ```
-3. Remove the existing remote reference (if any):
-    ```bash
-    git remote remove origin
-    ```
-4. Add the new remote reference pointing to your newly created Huggingface Space. Replace `<your-username>` and `<your-space-name>` with your actual Huggingface username and Space name:
-    ```bash
-    git remote add origin https://huggingface.co/spaces/<your-username>/<your-space-name>.git
-    ```
-
-## Step 3: Add, Commit, and Push the Code to the New Space
-1. Stage all the changes in your local project directory:
-    ```bash
-    git add .
-    ```
-2. Commit the changes with a meaningful commit message:
-    ```bash
-    git commit -m "Initial commit to Huggingface Space"
-    ```
-3. Push the changes to the new Huggingface Space:
-    ```bash
-    git push origin main
-    ```
-
-> **Note**: If your default branch is not `main`, replace `main` with the appropriate branch name in the push command.
-
-## Conclusion
-You have now successfully created a new Huggingface Space, updated your local Git remote reference, and pushed your code to the new Space. You can verify that your code has been uploaded by visiting your Huggingface Space's URL.
-
-## Webcam STUN/TURN Server
-
-When running remotely on Huggingface, the code needs to access your remote webcam. It does this using the [streamlit-webrtc](https://github.com/whitphx/streamlit-webrtc) module but requires a Twilio account to be established and the credentials uploaded to the Huggingface space.
-
-### How to Create a Free Twilio Account and Add Credentials to Huggingface Space Settings
-
-#### Step 1: Create a Free Twilio Account
-1. Go to the [Twilio Sign-Up Page](https://www.twilio.com/try-twilio).
-2. Fill in your details to create a new account.
-3. Verify your email address and phone number.
-4. After verification, log in to your Twilio dashboard.
-
-#### Step 2: Obtain `TWILIO_ACCOUNT_SID` and `TWILIO_AUTH_TOKEN`
-1. In the Twilio dashboard, navigate to the **Console**.
-2. Look for the **Account Info** section on the dashboard.
-3. Here, you will find your `Account SID` (referred to as `TWILIO_ACCOUNT_SID`).
-4. To obtain your `Auth Token` (referred to as `TWILIO_AUTH_TOKEN`), click on the **Show** button next to the `Auth Token`.
-
-#### Step 3: Add Twilio Credentials to Huggingface Space Settings
-1. Log in to your [Huggingface](https://huggingface.co/) account.
-2. Navigate to your Huggingface Space where you need to add the credentials.
-3. Go to the **Settings** of your Space.
-4. In the **Variables and secrets** section:
-    - Click on the **New variable** button to add `TWILIO_ACCOUNT_SID`:
-        - Name: `TWILIO_ACCOUNT_SID`
-        - Value: Copy your `Account SID` from the Twilio dashboard and paste it here.
-    - Click on the **New secret** button to add `TWILIO_AUTH_TOKEN`:
-        - Name: `TWILIO_AUTH_TOKEN`
-        - Value: Copy your `Auth Token` from the Twilio dashboard and paste it here.
-
-5. Save the changes.
-
-You have now successfully added your Twilio credentials to the Huggingface Space settings. Your application should now be able to access and use the Twilio API for WebRTC functionality.
+This Streamlit application uses the Computer Vision Playground as its base and has modified the analyze_frame function, replacing the example face/sentiment detection with object detection.  
 
+To learn how to do the same yourself and start playing with computer vision models read [here](https://huggingface.co/spaces/eusholli/computer-vision-playground).
 
+A massive thanks to [Ultralytics](https://docs.ultralytics.com/modes/track/#python-examples)
 
 
 ### Contributing

app.py

@@ -1,11 +1,11 @@
+from ultralytics import YOLO
 import torch
 import tensorflow as tf
 import time
 import os
 import logging
-import queue
 from pathlib import Path
-from typing import List, NamedTuple
+from typing import List
 
 import av
 import cv2
@@ -24,86 +24,56 @@ import requests
 from io import BytesIO  # Import for handling byte streams
 
 
-# Named tuple to store detection results
-class Detection(NamedTuple):
-    class_id: int
-    label: str
-    score: float
-    box: np.ndarray
-
-
-# Queue to store detection results
-result_queue: "queue.Queue[List[Detection]]" = queue.Queue()
-
 # CHANGE CODE BELOW HERE, USE TO REPLACE WITH YOUR WANTED ANALYSIS.
 # Update below string to set display title of analysis
 
-# Appropriate imports needed for analysis
+# Default title - "Facial Sentiment Analysis"
 
-# Initialize MTCNN for face detection
-mtcnn = MTCNN()
+ANALYSIS_TITLE = "YOLO-8 Object Detection Analysis"
 
-# Initialize the Hugging Face pipeline for facial emotion detection
-emotion_pipeline = pipeline("image-classification",
-                            model="trpakov/vit-face-expression")
 
+# Load the YOLOv8 model
+model = YOLO("yolov8n.pt")
 
-# Default title - "Facial Sentiment Analysis"
-
-ANALYSIS_TITLE = "Facial Sentiment Analysis"
 
 # CHANGE THE CONTENTS OF THIS FUNCTION, USE TO REPLACE WITH YOUR WANTED ANALYSIS.
 #
-#
-# Function to analyze an input frame and generate an analyzed frame
-# This function takes an input video frame, detects faces in it using MTCNN,
-# then for each detected face, it analyzes the sentiment (emotion) using the analyze_sentiment function,
-# draws a rectangle around the face, and overlays the detected emotion on the frame.
-# It also records the time taken to process the frame and stores it in a global container.
-# Constants for text and line size in the output image
-TEXT_SIZE = 1
-LINE_SIZE = 2
 
 
 # Set analysis results in img_container and result queue for display
 # img_container["input"] - holds the input frame contents - of type np.ndarray
 # img_container["analyzed"] - holds the analyzed frame with any added annotations - of type np.ndarray
 # img_container["analysis_time"] - holds how long the analysis has taken in miliseconds
-# result_queue - holds the analysis metadata results - of type queue.Queue[List[Detection]]
+# result_queue - holds the analysis metadata results - of type dictionary
 def analyze_frame(frame: np.ndarray):
     start_time = time.time()  # Start timing the analysis
     img_container["input"] = frame  # Store the input frame
     frame = frame.copy()  # Create a copy of the frame to modify
 
-    results = mtcnn.detect_faces(frame)  # Detect faces in the frame
-    for result in results:
-        x, y, w, h = result["box"]  # Get the bounding box of the detected face
-        face = frame[y: y + h, x: x + w]  # Extract the face from the frame
-        # Analyze the sentiment of the face
-        sentiment = analyze_sentiment(face)
-        result["label"] = sentiment
-        # Draw a rectangle around the face
-        cv2.rectangle(frame, (x, y), (x + w, y + h), (0, 0, 255), LINE_SIZE)
-        text_size = cv2.getTextSize(sentiment, cv2.FONT_HERSHEY_SIMPLEX, TEXT_SIZE, 2)[
-            0
-        ]
-        text_x = x
-        text_y = y - 10
-        background_tl = (text_x, text_y - text_size[1])
-        background_br = (text_x + text_size[0], text_y + 5)
-        # Draw a black background for the text
-        cv2.rectangle(frame, background_tl, background_br,
-                      (0, 0, 0), cv2.FILLED)
-        # Put the sentiment text on the image
-        cv2.putText(
-            frame,
-            sentiment,
-            (text_x, text_y),
-            cv2.FONT_HERSHEY_SIMPLEX,
-            TEXT_SIZE,
-            (255, 255, 255),
-            2,
-        )
+    # Run YOLOv8 tracking on the frame, persisting tracks between frames
+    results = model.track(frame, persist=True)
+
+    # Initialize a list to store Detection objects
+    detections = []
+    object_counter = 1
+
+    # Iterate over the detected boxes
+    for box in results[0].boxes:
+        detection = {}
+        # Extract class id, label, score, and bounding box coordinates
+        class_id = int(box.cls)
+
+        detection["id"] = object_counter
+        detection["label"] = model.names[class_id]
+        detection["score"] = float(box.conf)
+        detection["box_coords"] = [round(value.item(), 2)
+                                   for value in box.xyxy.flatten()]
+
+        detections.append(detection)
+        object_counter += 1
+
+    # Visualize the results on the frame
+    frame = results[0].plot()
 
     end_time = time.time()  # End timing the analysis
     execution_time_ms = round(
@@ -112,27 +82,13 @@ def analyze_frame(frame: np.ndarray):
     # Store the execution time
     img_container["analysis_time"] = execution_time_ms
 
-    result_queue.put(results)  # Put the results in the result queue
+    # store the detections
+    img_container["detections"] = detections
     img_container["analyzed"] = frame  # Store the analyzed frame
 
     return  # End of the function
 
 
-# Function to analyze the sentiment (emotion) of a detected face
-# This function converts the face from BGR to RGB format, then converts it to a PIL image,
-# uses a pre-trained emotion detection model to get emotion predictions,
-# and finally returns the most dominant emotion detected.
-def analyze_sentiment(face):
-    # Convert face to RGB format
-    rgb_face = cv2.cvtColor(face, cv2.COLOR_BGR2RGB)
-    pil_image = Image.fromarray(rgb_face)  # Convert to PIL image
-    results = emotion_pipeline(pil_image)  # Run emotion detection on the image
-    dominant_emotion = max(results, key=lambda x: x["score"])[
-        "label"
-    ]  # Get the dominant emotion
-    return dominant_emotion  # Return the detected emotion
-
-
 #
 #
 # DO NOT TOUCH THE BELOW CODE (NOT NEEDED)
@@ -157,7 +113,8 @@ logging.getLogger("torch").setLevel(logging.ERROR)
 logging.getLogger("streamlit").setLevel(logging.ERROR)
 
 # Container to hold image data and analysis results
-img_container = {"input": None, "analyzed": None, "analysis_time": None}
+img_container = {"input": None, "analyzed": None,
+                 "analysis_time": None, "detections": None}
 
 # Logger for debugging and information
 logger = logging.getLogger(__name__)
@@ -294,12 +251,6 @@ def analysis_init():
 # This function retrieves the latest frames and results from the global container and result queue,
 # and updates the placeholders in the Streamlit UI with the current input frame, analyzed frame, analysis time, and detected labels.
 def publish_frame():
-    if not result_queue.empty():
-        result = result_queue.get()
-        if show_labels:
-            labels_placeholder.table(
-                result
-            )  # Display labels if the checkbox is checked
 
     img = img_container["input"]
     if img is None:
@@ -318,6 +269,15 @@ def publish_frame():
     # Display the analysis time
     analysis_time.text(f"Analysis Time: {time} ms")
 
+    detections = img_container["detections"]
+    if detections is None:
+        return
+
+    if show_labels:
+        labels_placeholder.table(
+            detections
+        )  # Display labels if the checkbox is checked
+
 
 # If the WebRTC streamer is playing, initialize and publish frames
 if webrtc_ctx.state.playing:
@@ -361,13 +321,6 @@ def process_video(video_path):
         )  # Analyze the frame for face detection and sentiment analysis
         publish_frame()  # Publish the results
 
-        if not result_queue.empty():
-            result = result_queue.get()
-            if show_labels:
-                labels_placeholder.table(
-                    result
-                )  # Display labels if the checkbox is checked
-
     cap.release()  # Release the video capture object
 
 

object_detection.py

@@ -1,420 +0,0 @@
-import torch
-import tensorflow as tf
-import time
-import os
-import logging
-import queue
-from pathlib import Path
-from typing import List, NamedTuple
-
-import av
-import cv2
-import numpy as np
-import streamlit as st
-from streamlit_webrtc import WebRtcMode, webrtc_streamer
-
-from utils.download import download_file
-from utils.turn import get_ice_servers
-
-from PIL import Image, ImageDraw  # Import PIL for image processing
-from transformers import pipeline  # Import Hugging Face transformers pipeline
-
-import requests
-from io import BytesIO  # Import for handling byte streams
-
-
-# Named tuple to store detection results
-class Detection(NamedTuple):
-    class_id: int
-    label: str
-    score: float
-    box: np.ndarray
-
-
-# Queue to store detection results
-result_queue: "queue.Queue[List[Detection]]" = queue.Queue()
-
-# CHANGE CODE BELOW HERE, USE TO REPLACE WITH YOUR WANTED ANALYSIS.
-# Update below string to set display title of analysis
-
-# Appropriate imports needed for analysis
-
-MODEL_URL = "https://github.com/robmarkcole/object-detection-app/raw/master/model/MobileNetSSD_deploy.caffemodel"
-MODEL_LOCAL_PATH = Path("./models/MobileNetSSD_deploy.caffemodel")
-PROTOTXT_URL = "https://github.com/robmarkcole/object-detection-app/raw/master/model/MobileNetSSD_deploy.prototxt.txt"
-PROTOTXT_LOCAL_PATH = Path("./models/MobileNetSSD_deploy.prototxt.txt")
-
-CLASSES = [
-    "background",
-    "aeroplane",
-    "bicycle",
-    "bird",
-    "boat",
-    "bottle",
-    "bus",
-    "car",
-    "cat",
-    "chair",
-    "cow",
-    "diningtable",
-    "dog",
-    "horse",
-    "motorbike",
-    "person",
-    "pottedplant",
-    "sheep",
-    "sofa",
-    "train",
-    "tvmonitor",
-]
-
-# Generate random colors for each class label
-
-
-def generate_label_colors():
-    return np.random.uniform(0, 255, size=(len(CLASSES), 3))
-
-
-COLORS = generate_label_colors()
-
-# Download model and prototxt files
-
-
-def download_file(url, local_path, expected_size=None):
-    if not local_path.exists() or (expected_size and local_path.stat().st_size != expected_size):
-        import requests
-        with open(local_path, "wb") as f:
-            response = requests.get(url)
-            f.write(response.content)
-
-
-download_file(MODEL_URL, MODEL_LOCAL_PATH, expected_size=23147564)
-download_file(PROTOTXT_URL, PROTOTXT_LOCAL_PATH, expected_size=29353)
-
-# Load the model
-net = cv2.dnn.readNetFromCaffe(str(PROTOTXT_LOCAL_PATH), str(MODEL_LOCAL_PATH))
-
-
-# Default title - "Facial Sentiment Analysis"
-
-ANALYSIS_TITLE = "Object Detection Analysis"
-
-# CHANGE THE CONTENTS OF THIS FUNCTION, USE TO REPLACE WITH YOUR WANTED ANALYSIS.
-#
-
-# Set analysis results in img_container and result queue for display
-# img_container["input"] - holds the input frame contents - of type np.ndarray
-# img_container["analyzed"] - holds the analyzed frame with any added annotations - of type np.ndarray
-# img_container["analysis_time"] - holds how long the analysis has taken in miliseconds
-# result_queue - holds the analysis metadata results - of type queue.Queue[List[Detection]]
-
-
-def analyze_frame(frame: np.ndarray):
-    start_time = time.time()  # Start timing the analysis
-    img_container["input"] = frame  # Store the input frame
-    frame = frame.copy()  # Create a copy of the frame to modify
-
-    # Run inference
-    blob = cv2.dnn.blobFromImage(
-        cv2.resize(frame, (300, 300)), 0.007843, (300, 300), 127.5
-    )
-    net.setInput(blob)
-    output = net.forward()
-
-    h, w = frame.shape[:2]
-
-    # Filter the detections based on the score threshold
-    score_threshold = 0.5  # You can adjust the score threshold as needed
-    output = output.squeeze()  # (1, 1, N, 7) -> (N, 7)
-    output = output[output[:, 2] >= score_threshold]
-    detections = [
-        Detection(
-            class_id=int(detection[1]),
-            label=CLASSES[int(detection[1])],
-            score=float(detection[2]),
-            box=(detection[3:7] * np.array([w, h, w, h])),
-        )
-        for detection in output
-    ]
-
-    # Render bounding boxes and captions
-    for detection in detections:
-        caption = f"{detection.label}: {round(detection.score * 100, 2)}%"
-        color = COLORS[detection.class_id]
-        xmin, ymin, xmax, ymax = detection.box.astype("int")
-
-        cv2.rectangle(frame, (xmin, ymin), (xmax, ymax), color, 2)
-        cv2.putText(
-            frame,
-            caption,
-            (xmin, ymin - 15 if ymin - 15 > 15 else ymin + 15),
-            cv2.FONT_HERSHEY_SIMPLEX,
-            0.5,
-            color,
-            2,
-        )
-
-    end_time = time.time()  # End timing the analysis
-    # Calculate execution time in milliseconds
-    execution_time_ms = round((end_time - start_time) * 1000, 2)
-    # Store the execution time
-    img_container["analysis_time"] = execution_time_ms
-
-    result_queue.put(detections)  # Put the results in the result queue
-    img_container["analyzed"] = frame  # Store the analyzed frame
-
-    return  # End of the function
-
-#
-#
-# DO NOT TOUCH THE BELOW CODE (NOT NEEDED)
-#
-#
-
-
-# Suppress FFmpeg logs
-os.environ["FFMPEG_LOG_LEVEL"] = "quiet"
-
-# Suppress TensorFlow or PyTorch progress bars
-
-tf.get_logger().setLevel("ERROR")
-os.environ["TF_CPP_MIN_LOG_LEVEL"] = "3"
-
-# Suppress PyTorch logs
-
-logging.getLogger().setLevel(logging.WARNING)
-torch.set_num_threads(1)
-logging.getLogger("torch").setLevel(logging.ERROR)
-
-# Suppress Streamlit logs using the logging module
-logging.getLogger("streamlit").setLevel(logging.ERROR)
-
-# Container to hold image data and analysis results
-img_container = {"input": None, "analyzed": None, "analysis_time": None}
-
-# Logger for debugging and information
-logger = logging.getLogger(__name__)
-
-
-# Callback function to process video frames
-# This function is called for each video frame in the WebRTC stream.
-# It converts the frame to a numpy array in RGB format, analyzes the frame,
-# and returns the original frame.
-def video_frame_callback(frame: av.VideoFrame) -> av.VideoFrame:
-    # Convert frame to numpy array in RGB format
-    img = frame.to_ndarray(format="rgb24")
-    analyze_frame(img)  # Analyze the frame
-    return frame  # Return the original frame
-
-
-# Get ICE servers for WebRTC
-ice_servers = get_ice_servers()
-
-# Streamlit UI configuration
-st.set_page_config(layout="wide")
-
-# Custom CSS for the Streamlit page
-st.markdown(
-    """
-    <style>
-        .main {
-            padding: 2rem;
-        }
-        h1, h2, h3 {
-            font-family: 'Arial', sans-serif;
-        }
-        h1 {
-            font-weight: 700;
-            font-size: 2.5rem;
-        }
-        h2 {
-            font-weight: 600;
-            font-size: 2rem;
-        }
-        h3 {
-            font-weight: 500;
-            font-size: 1.5rem;
-        }
-    </style>
-    """,
-    unsafe_allow_html=True,
-)
-
-# Streamlit page title and subtitle
-st.title("Computer Vision Playground")
-
-# Add a link to the README file
-st.markdown(
-    """
-    <div style="text-align: left;">
-        <p>See the <a href="https://huggingface.co/spaces/eusholli/sentiment-analyzer/blob/main/README.md" 
-        target="_blank">README</a> to learn how to use this code to help you start your computer vision exploration.</p>
-    </div>
-    """,
-    unsafe_allow_html=True,
-)
-
-st.subheader(ANALYSIS_TITLE)
-
-# Columns for input and output streams
-col1, col2 = st.columns(2)
-
-with col1:
-    st.header("Input Stream")
-    st.subheader("input")
-    # WebRTC streamer to get video input from the webcam
-    webrtc_ctx = webrtc_streamer(
-        key="input-webcam",
-        mode=WebRtcMode.SENDRECV,
-        rtc_configuration=ice_servers,
-        video_frame_callback=video_frame_callback,
-        media_stream_constraints={"video": True, "audio": False},
-        async_processing=True,
-    )
-
-    # File uploader for images
-    st.subheader("Upload an Image")
-    uploaded_file = st.file_uploader(
-        "Choose an image...", type=["jpg", "jpeg", "png"])
-
-    # Text input for image URL
-    st.subheader("Or Enter Image URL")
-    image_url = st.text_input("Image URL")
-
-    # File uploader for videos
-    st.subheader("Upload a Video")
-    uploaded_video = st.file_uploader(
-        "Choose a video...", type=["mp4", "avi", "mov", "mkv"]
-    )
-
-    # Text input for video URL
-    st.subheader("Or Enter Video Download URL")
-    video_url = st.text_input("Video URL")
-
-# Streamlit footer
-st.markdown(
-    """
-    <div style="text-align: center; margin-top: 2rem;">
-        <p>If you want to set up your own computer vision playground see <a href="https://huggingface.co/spaces/eusholli/computer-vision-playground/blob/main/README.md" target="_blank">here</a>.</p>
-    </div>
-    """,
-    unsafe_allow_html=True
-)
-
-# Function to initialize the analysis UI
-# This function sets up the placeholders and UI elements in the analysis section.
-# It creates placeholders for input and output frames, analysis time, and detected labels.
-
-
-def analysis_init():
-    global analysis_time, show_labels, labels_placeholder, input_placeholder, output_placeholder
-
-    with col2:
-        st.header("Analysis")
-        st.subheader("Input Frame")
-        input_placeholder = st.empty()  # Placeholder for input frame
-
-        st.subheader("Output Frame")
-        output_placeholder = st.empty()  # Placeholder for output frame
-        analysis_time = st.empty()  # Placeholder for analysis time
-        show_labels = st.checkbox(
-            "Show the detected labels", value=True
-        )  # Checkbox to show/hide labels
-        labels_placeholder = st.empty()  # Placeholder for labels
-
-
-# Function to publish frames and results to the Streamlit UI
-# This function retrieves the latest frames and results from the global container and result queue,
-# and updates the placeholders in the Streamlit UI with the current input frame, analyzed frame, analysis time, and detected labels.
-def publish_frame():
-    if not result_queue.empty():
-        result = result_queue.get()
-        if show_labels:
-            labels_placeholder.table(
-                result
-            )  # Display labels if the checkbox is checked
-
-    img = img_container["input"]
-    if img is None:
-        return
-    input_placeholder.image(img, channels="RGB")  # Display the input frame
-
-    analyzed = img_container["analyzed"]
-    if analyzed is None:
-        return
-    # Display the analyzed frame
-    output_placeholder.image(analyzed, channels="RGB")
-
-    time = img_container["analysis_time"]
-    if time is None:
-        return
-    # Display the analysis time
-    analysis_time.text(f"Analysis Time: {time} ms")
-
-
-# If the WebRTC streamer is playing, initialize and publish frames
-if webrtc_ctx.state.playing:
-    analysis_init()  # Initialize the analysis UI
-    while True:
-        publish_frame()  # Publish the frames and results
-        time.sleep(0.1)  # Delay to control frame rate
-
-
-# If an image is uploaded or a URL is provided, process the image
-if uploaded_file is not None or image_url:
-    analysis_init()  # Initialize the analysis UI
-
-    if uploaded_file is not None:
-        image = Image.open(uploaded_file)  # Open the uploaded image
-        img = np.array(image.convert("RGB"))  # Convert the image to RGB format
-    else:
-        response = requests.get(image_url)  # Download the image from the URL
-        # Open the downloaded image
-        image = Image.open(BytesIO(response.content))
-        img = np.array(image.convert("RGB"))  # Convert the image to RGB format
-
-    analyze_frame(img)  # Analyze the image
-    publish_frame()  # Publish the results
-
-
-# Function to process video files
-# This function reads frames from a video file, analyzes each frame for face detection and sentiment analysis,
-# and updates the Streamlit UI with the current input frame, analyzed frame, and detected labels.
-def process_video(video_path):
-    cap = cv2.VideoCapture(video_path)  # Open the video file
-    while cap.isOpened():
-        ret, frame = cap.read()  # Read a frame from the video
-        if not ret:
-            break  # Exit the loop if no more frames are available
-
-        # Display the current frame as the input frame
-        input_placeholder.image(frame)
-        analyze_frame(
-            frame
-        )  # Analyze the frame for face detection and sentiment analysis
-        publish_frame()  # Publish the results
-
-        if not result_queue.empty():
-            result = result_queue.get()
-            if show_labels:
-                labels_placeholder.table(
-                    result
-                )  # Display labels if the checkbox is checked
-
-    cap.release()  # Release the video capture object
-
-
-# If a video is uploaded or a URL is provided, process the video
-if uploaded_video is not None or video_url:
-    analysis_init()  # Initialize the analysis UI
-
-    if uploaded_video is not None:
-        video_path = uploaded_video.name  # Get the name of the uploaded video
-        with open(video_path, "wb") as f:
-            # Save the uploaded video to a file
-            f.write(uploaded_video.getbuffer())
-    else:
-        # Download the video from the URL
-        video_path = download_file(video_url)
-
-    process_video(video_path)  # Process the video

requirements.txt

@@ -3,9 +3,9 @@ opencv-python-headless
 numpy
 transformers
 torch
-mtcnn
 setuptools
 tensorflow
 tf-keras
 streamlit_webrtc
-twilio
+twilio
+ultralytics

sentiment.py

@@ -1,141 +0,0 @@
-import threading
-
-import streamlit as st
-import cv2
-import numpy as np
-from transformers import pipeline
-from PIL import Image, ImageDraw
-from mtcnn import MTCNN
-from streamlit_webrtc import webrtc_streamer
-import logging
-
-# Suppress transformers progress bars
-logging.getLogger("transformers").setLevel(logging.ERROR)
-
-lock = threading.Lock()
-img_container = {"webcam": None,
-                 "analyzed": None}
-
-# Initialize the Hugging Face pipeline for facial emotion detection
-emotion_pipeline = pipeline("image-classification", model="trpakov/vit-face-expression")
-
-# Initialize MTCNN for face detection
-mtcnn = MTCNN()
-
-# Function to analyze sentiment
-def analyze_sentiment(face):
-    # Convert face to RGB
-    rgb_face = cv2.cvtColor(face, cv2.COLOR_BGR2RGB)
-    # Convert the face to a PIL image
-    pil_image = Image.fromarray(rgb_face)
-    # Analyze sentiment using the Hugging Face pipeline
-    results = emotion_pipeline(pil_image)
-    # Get the dominant emotion
-    dominant_emotion = max(results, key=lambda x: x['score'])['label']
-    return dominant_emotion
-
-TEXT_SIZE = 3
-
-# Function to detect faces, analyze sentiment, and draw a red box around them
-def detect_and_draw_faces(frame):
-    # Detect faces using MTCNN
-    results = mtcnn.detect_faces(frame)
-    
-    # Draw on the frame
-    for result in results:
-        x, y, w, h = result['box']
-        face = frame[y:y+h, x:x+w]
-        sentiment = analyze_sentiment(face)
-        cv2.rectangle(frame, (x, y), (x+w, y+h), (0, 0, 255), 10)  # Thicker red box
-        
-        # Calculate position for the text background and the text itself
-        text_size = cv2.getTextSize(sentiment, cv2.FONT_HERSHEY_SIMPLEX, TEXT_SIZE, 2)[0]
-        text_x = x
-        text_y = y - 10
-        background_tl = (text_x, text_y - text_size[1])
-        background_br = (text_x + text_size[0], text_y + 5)
-        
-        # Draw black rectangle as background
-        cv2.rectangle(frame, background_tl, background_br, (0, 0, 0), cv2.FILLED)
-        # Draw white text on top
-        cv2.putText(frame, sentiment, (text_x, text_y), cv2.FONT_HERSHEY_SIMPLEX, TEXT_SIZE, (255, 255, 255), 2)
-    
-    return frame
-
-# Streamlit UI
-st.markdown(
-    """
-    <style>
-        .main {
-            background-color: #FFFFFF;
-        }
-        .reportview-container .main .block-container{
-            padding-top: 2rem;
-        }
-        h1 {
-            color: #E60012;
-            font-family: 'Arial Black', Gadget, sans-serif;
-        }
-        h2 {
-            color: #E60012;
-            font-family: 'Arial', sans-serif;
-        }
-        h3 {
-            color: #333333;
-            font-family: 'Arial', sans-serif;
-        }
-        .stButton button {
-            background-color: #E60012;
-            color: white;
-            border-radius: 5px;
-            font-size: 16px;
-        }
-    </style>
-    """,
-    unsafe_allow_html=True
-)
-
-st.title("Computer Vision Test Lab")
-st.subheader("Facial Sentiment")
-
-# Columns for input and output streams
-col1, col2 = st.columns(2)
-
-with col1:
-    st.header("Input Stream")
-    st.subheader("Webcam")
-    video_placeholder = st.empty()
-
-with col2:
-    st.header("Output Stream")
-    st.subheader("Analysis")
-    output_placeholder = st.empty()
-
-sentiment_placeholder = st.empty()
-
-def video_frame_callback(frame):
-    try:
-        with lock:
-            img = frame.to_ndarray(format="bgr24")
-            img_container["webcam"] = img
-            frame_with_boxes = detect_and_draw_faces(img)
-            img_container["analyzed"] = frame_with_boxes
-
-    except Exception as e:
-        st.error(f"Error processing frame: {e}")
-
-    return frame
-
-ctx = webrtc_streamer(key="webcam", video_frame_callback=video_frame_callback)
-
-while ctx.state.playing:
-    with lock:
-        print(img_container)
-        img = img_container["webcam"]
-        frame_with_boxes = img_container["analyzed"]
-
-    if img is None:
-        continue
-
-    video_placeholder.image(img, channels="BGR")
-    output_placeholder.image(frame_with_boxes, channels="BGR")