Spaces:
Runtime error
Runtime error
title: m4-dialogue | |
emoji: 🐨 | |
colorFrom: red | |
colorTo: indigo | |
sdk: gradio | |
sdk_version: 3.38.0 | |
app_file: app_dialogue.py | |
pinned: false | |
# M4 Visualization | |
For visualizations, we have a main [app](https://huggingface.co/spaces/HuggingFaceM4/m4-demo) which calls multiple child apps to retrieve generations via [Gradio API](https://gradio.app/using-blocks-like-functions/). This allows us to parallelize calls to multiple models at the same time instead of running them sequentially. | |
## How to? | |
The process of adding a model to the main space: | |
- Use `huggingface-cli login` to login with an auth token that has a read/write access to the `HuggingFaceM4` org on the hub. | |
- Use `./upload_checkpoint_to_hub_gcs.sh` script to upload a checkpoint from GCP store to the hub. An example command to upload checkpoint for step 3000 from `tr_121ter` to the hub: `./m4/visualization/upload_checkpoint_to_hub_gcs.sh gs://hf-science-m4-cold/local_experiment_dir/tr_121ter/opt_step-3000`. This will create model repo under the `HuggingFaceM4` repo on the hub. If you are on the cluster, use `./upload_checkpoint_to_hub_s3.sh` instead. I recommend being on a compute node to avoid disk space issues (uploading to the hub consists in downloading locally the checkpoint, creating a repo on the hub, copying it locally, filling it with the weights and commiting the weights to the hub repo). | |
- [MANUAL] Go to the hub, create a repo of type `space` with the same name as the model. In the space's settings, add a secret `HF_AUTH_TOKEN` with a token which has read access to the `HuggingFaceM4` repo. This step can be potentially automated in the future. | |
- [MANUAL] Edit `m4/visualization/app_dialogue.py`'s three dictionary to include your model in the existing formats of those dictionaries. | |
- Run `m4/visualization/sync-repo.sh <name_of_the_space_on_the_hub>` to sync the repo with the local setting. This will automatically update the space to have the latest code as in the `m4/visualization/app_dialogue.py`. | |
- Run `m4/visualization/sync-repo.sh main` to update the main repo as well with the new model. | |