metadata

title: lang-ground
app_file: app_langloc.py
sdk: gradio
sdk_version: 3.41.2

Language Grounding

Localize and keep tracking things based on natural language sepecfication is a good idea but remains challenging due to the scarcity of large-scale annotated datasets.

This repository provides a practical solution to identify relevant objects in the view and continuously tracking them, which is particularly beneficial for robotics applications.

We offer several demos that showcase the following functionalities:

Language Localize

Identify and locate objects based on natural language queries.

Try our demo to experience an interesting use case: When analyzing food items, the model demonstrates contextual understanding:

For food nearing expiration: Suggests storing in the cabinet
For expired food: Recommends disposal in the trash can

Lang Ground (Localize + Track):

Not only find but also continuously track objects of interest

We prioritize making this project highly accessible and customizable:

Open Box Design: All components are modular and well-documented for easy understanding
Customizable Pipeline: Easily adapt the system for different use cases
Extensible Framework: Simple integration with other vision or language models

🛠️ Install

git clone https://github.com/jing-bi/lang-ground.git && cd lang-ground

mamba create -n lang-ground python=3.11
mamba activate lang-ground

pip install -e .

Acknowledgments

This project is built upon and inspired by the following repositories:

License

This project is licensed under the Apache 2.0 License