omni_bot / docs /examples /omni_agent.md
Zack Zitting Bradshaw
Upload folder using huggingface_hub
4962437

A newer version of the Gradio SDK is available: 4.36.1

Upgrade

A Comprehensive Guide to Setting Up OmniWorker: Your Passport to Multimodal Tasks**

Introduction

  • Introduction to OmniWorker
  • Explanation of its use-cases and importance in multimodal tasks
  • Mention of prerequisites: Git, Python 3.x, Terminal or Command Prompt access

Chapter 1: Cloning the Necessary Repository

  • Explanation of Git and its use in version control
  • Step-by-step guide on how to clone the OmniWorker repository
    !git clone https://github.com/kyegomez/swarms
    

Chapter 2: Navigating to the Cloned Directory

  • Explanation of directory navigation in the terminal
    %cd /swarms
    

Chapter 3: Installing the Required Dependencies

  • Explanation of Python dependencies and the purpose of requirements.txt file
  • Step-by-step installation of dependencies
    !pip install -r requirements.txt
    

Chapter 4: Installing Additional Dependencies

  • Discussion on the additional dependencies and their roles in OmniWorker
    !pip install git+https://github.com/IDEA-Research/GroundingDINO.git
    !pip install git+https://github.com/facebookresearch/segment-anything.git
    !pip install faiss-gpu
    !pip install langchain-experimental
    

Chapter 5: Setting Up Your OpenAI API Key

  • Explanation of OpenAI API and its key
  • Guide on how to obtain and set up the OpenAI API key
    !export OPENAI_API_KEY="your-api-key"
    

Chapter 6: Running the OmniModal Agent Script

  • Discussion on the OmniModal Agent script and its functionality
  • Guide on how to run the script
    !python3 omnimodal_agent.py
    

Chapter 7: Importing the Necessary Modules

  • Discussion on Python modules and their importance
  • Step-by-step guide on importing necessary modules for OmniWorker
    from langchain.llms import OpenAIChat
    from swarms.agents import OmniModalAgent
    

Chapter 8: Creating and Running OmniModalAgent Instance

  • Explanation of OmniModalAgent instance and its role
  • Guide on how to create and run OmniModalAgent instance
    llm = OpenAIChat()
    agent = OmniModalAgent(llm)
    agent.run("Create a video of a swarm of fish")
    

Conclusion

  • Recap of the steps taken to set up OmniWorker
  • Encouragement to explore more functionalities and apply OmniWorker to various multimodal tasks