Spaces:
Running
Running
| title: lang-ground | |
| app_file: app_langloc.py | |
| sdk: gradio | |
| sdk_version: 3.41.2 | |
| # Language Grounding | |
| Localize and keep tracking things based on natural | |
| language sepecfication is a good idea but remains challenging due to the scarcity of large-scale annotated datasets. | |
| This repository provides a practical solution to identify relevant objects in the view and continuously tracking them, which is particularly beneficial for robotics applications. | |
| We offer several demos that showcase the following functionalities: | |
| ## Language Localize | |
| Identify and locate objects based on natural language queries. | |
| Try our demo to experience an interesting use case: When analyzing food items, the model demonstrates contextual understanding: | |
| - For food nearing expiration: Suggests storing in the cabinet | |
| - For expired food: Recommends disposal in the trash can | |
|  | |
| ## **Lang Ground (Localize + Track):** | |
| Not only find but also continuously track objects of interest | |
| We prioritize making this project highly accessible and customizable: | |
| - **Open Box Design:** All components are modular and well-documented for easy understanding | |
| - **Customizable Pipeline:** Easily adapt the system for different use cases | |
| - **Extensible Framework:** Simple integration with other vision or language models | |
| ## 🛠️ Install | |
| ```bash | |
| git clone https://github.com/jing-bi/lang-ground.git && cd lang-ground | |
| mamba create -n lang-ground python=3.11 | |
| mamba activate lang-ground | |
| pip install -e . | |
| ``` | |
| ## Acknowledgments | |
| This project is built upon and inspired by the following repositories: | |
| - [Segment-Anything](https://github.com/facebookresearch/segment-anything-2) | |
| - [Supervision](https://github.com/roboflow/supervision) | |
| ## License | |
| This project is licensed under the Apache 2.0 License |