Remotely upload files to a Hugging Face repo using access token
Published: 2023-10-24 07:31:03

# Uploading Files to Hugging Face with Python

Hugging Face provides easy access to its repos for storing and sharing machine learning models through simple Python APIs. You can upload files without having them stored locally by leveraging the `huggingface_hub` Python library. 

## Prerequisites

- Python 3.x
- `huggingface_hub` library installed (`pip install huggingface_hub`)
- Hugging Face account and access token

## Getting an Access Token

To upload files, you need an access token for your Hugging Face account:

1. Go to your [Hugging Face profile settings](https://huggingface.co/settings/token)
2. Click "New token" and give it a description like "Python uploads"
3. Copy the generated token - this will be used later

## Uploading a File 

With the access token, uploading a file just takes a few lines of code:

```python
from huggingface_hub import login, upload_file

# Login using your access token 
login(token="hf_abc123") 

# File data to upload
file_data = read_binary_file(...) 

# Upload file data
upload_file(
    path_or_fileobj=file_data,
    path_in_repo="files/image.jpg",
    repo_id="username/repo", 
    repo_type="space"
)
```

The key steps:

1. Login with `login()` using your access token
2. Get the binary file data to upload 
3. Call `upload_file()`, passing the data and destination file path

The file data can come from anywhere - downloading a remote file, reading a database BLOB, etc. As long as you have the binary data in memory, it can be uploaded.

## Uploading from a URL

To upload a file from a remote URL, you can use the Requests library:

```python
import requests
from huggingface_hub import login, upload_file

url = "https://example.com/image.jpg"

# Download file data
data = requests.get(url).content

# Upload data
upload_file(data, "images/remote_image.jpg", "username/repo") 
```

This provides a simple way to upload public files without needing local storage.

## Getting File Data

There are a few options for getting the binary file data to upload:

**User Input**

You can ask the user to provide a file path and read the data:

```python
file_path = input("Enter file path to upload: ")
file_data = open(file_path, "rb").read()
```

**Web Form**

If building a web app with Flask, you can get the data from the file upload:

```python
from flask import request

@app.route("/upload", methods=["POST"])
def upload_file():
    file_data = request.files["file"].read() 
    # Upload file...
```

**Downloading** 

You can download data from a remote URL:

```python 
import requests

url = "https://example.com/file.jpg" 
file_data = requests.get(url).content
```

This provides some options forsourcing the binary file data before uploading to Hugging Face. The `huggingface_hub` library handles the upload seamlessly once you have the data in memory.

## Conclusion

The `huggingface_hub` Python library makes it easy to upload files to your Hugging Face repositories with just a few lines of code using your access token. By passing binary file data directly, you can upload files from anywhere without requiring them to be stored locally.