ilyali034/ruT5-base-smart-home

ruT5 smart home function calling

Fine-tuned ai-forever/ruT5-base. Takes a Russian smart home command and returns a JSON list of function calls in Home Assistant style (HassTurnOn, HassTurnOff, HassLightSet, HassClimateSetTemperature and others).

Модель превращает русскую команду для умного дома в JSON со списком вызовов функций.

Usage

The tokenizer vocabulary lacks some characters ({ } Q Y Z & é →), so the model works with an escaped text format. inference.py in this repository handles it. Do not call generate directly without it.

from huggingface_hub import snapshot_download
import sys

path = snapshot_download("ilyali034/ruT5-base-smart-home")
sys.path.insert(0, path)
from inference import SmartHomeModel

model = SmartHomeModel(path)
print(model.predict("включи свет на кухне"))

Output example:

{"text": "[{\"name\": \"HassTurnOn\", \"arguments\": {\"name\": \"Kitchen Light\"}}]", "calls": [{"name": "HassTurnOn", "arguments": {"name": "Kitchen Light"}}]}

Training

• Base model: ai-forever/ruT5-base
• Data: RockMan256/needle-ru-opt, split 90/5/5 (train/val/test), seed 42
• 5 epochs, Adafactor, learning rate 3e-4, batch size 16, fp16
• The tools list is identical in every example of the dataset, so it is not given to the model: the input is query: <command>

Results (test, 1290 examples)

Metric	Value
Exact match (canonical JSON)	0.828
Valid JSON rate	1.000
Function name F1 (micro)	0.993
Main function accuracy	0.997
Arguments accuracy	0.792
Token F1	0.956
BLEU	97.3
chrF	96.9

Limitations

• Almost all errors are in the name argument (device name). The model does not see the list of devices in the home, so when a command does not say which device is meant (for example, several thermostats), or a device has several valid English names, it picks one. Use it together with a list of real devices, or map its output to the closest known device name.
• Sometimes the model skips the last call in commands with several actions.
• Device names in the data are mostly English, commands are Russian.
• Trained on synthetic data. Quality on real user phrases may be lower.