License & Attribution

This repository contains weights or code derived from the SmolVLA foundational architecture developed by Hugging Face and the LeRobot Authors.

License: Distributed under the https://apache.org.
Original Model Base: https://huggingface.co/lerobot/smolvla_base
Copyright Notice: Copyright 2025-2026 Hugging Face & The LeRobot Authors.
Research paper introducing this model: https://arxiv.org/pdf/2506.01844

This is SmolVLA-Base model cloned from Hugginface "lerobot/smolvla_base" repository. This was createed for ready-to-use custom model for easy inference during Hackathon challenge.

Script to load model from HuggingFace:

from lerobot.policies.smolvla.modeling_smolvla import SmolVLAPolicy
from lerobot.policies.factory import make_pre_post_processors
import torch

device = "cuda" if torch.cuda.is_available() else "cpu"

# Switch this to your custom Hugging Face model repository or local directory path
my_custom_model_id = "Man1103/SmolVLA-Base-0.45B"

# 2. Load Model instance skeleton onto device
model = SmolVLAPolicy.from_pretrained(my_custom_model_id).to(device)

print("Model successfully loaded!")

Script for loading model specific preprocess and postprocess assets:

# 3. Initialize pre/post-processors using your loaded model config 
preprocess, postprocess = make_pre_post_processors(
    policy_cfg=model.config,      # Passes the structural configuration matrix
    pretrained_path=my_custom_model_id # Resolves dataset statistics from your repository
)

print("Model successfully initialized!")

Script to perform random sample inference:

import numpy as np
import pandas as pd
from PIL import Image
import torch
import torchvision.transforms.functional as TF

# 1. Create a dummy image frame (RGB, 256x256 as specified by the model's expected shape)
mock_image = Image.fromarray(np.random.randint(0, 255, (256, 256, 3), dtype=np.uint8))
mock_instruction = "Grasp the red block and place it in the tray."
mock_robot_state = torch.zeros((1, 6)) # Example: 6-DoF robot arm state

# 2. Build the observation map using the EXACT keys the model is looking for
observation_frame = {
    "observation.images.camera1": mock_image,
    "observation.images.camera2": mock_image,
    "observation.images.camera3": mock_image,
    "observation.state": mock_robot_state,
    "task": mock_instruction
}

# 3. Process and format
processed_observation = preprocess(observation_frame)

# 4. Push tensors to the GPU, add batch dimensions, and convert PIL Images
for key, value in processed_observation.items():
    if isinstance(value, Image.Image):
        # Convert to tensor (C, H, W) and add batch dimension -> (1, C, H, W)
        tensor_val = TF.to_tensor(value).to(device)
        processed_observation[key] = tensor_val.unsqueeze(0)
    elif isinstance(value, torch.Tensor):
        # Ensure tensor values have a batch dimension at index 0
        if value.ndim == 1:
            processed_observation[key] = value.unsqueeze(0).to(device)
        elif value.ndim == 2 and key == "observation.state":
            # State is already (1, 6), keep it or make sure it handles batching correctly
            processed_observation[key] = value.to(device)
        else:
            processed_observation[key] = value.to(device)

# 5. Execute VLA Policy Inference
model.eval()
with torch.no_grad():
    print("Predicting action sequence with batched streams...")
    predicted_action = model.select_action(processed_observation)
    final_robot_commands = postprocess(predicted_action)

# 6. Output Result
print("\n--- INFERENCE SUCCESS ---")
print("Predicted Robot Action Matrix shape:", final_robot_commands.shape)
print(f"Final robot command: {final_robot_commands}")