SceneViT-Nano-P8-64

A Nano Vision Transformer (ViT) model built from scratch for scene classification tasks, trained on the Intel Image Classification dataset.

Build and Train pipeline here:

Model Specifications

• Patch Size: 8x8
• Image Size: 224x224
• Hidden Size: 64
• Attention Heads: 4
• Hidden Layers: 8
• Validation Accuracy: ~84.30%

Training Summary & Progression

The custom Vision Transformer model was trained for 30 epochs on the Intel Image Classification dataset using a Cosine Annealing learning rate scheduler and early stopping.

Training and Validation Metrics Table

Epoch	Train Loss	Train Acc (%)	Val Loss	Val Acc (%)
1	1.3781	44.61%	1.1608	52.27%
2	1.0283	59.30%	0.9267	64.20%
3	0.9045	64.72%	0.8021	69.47%
4	0.8076	69.26%	0.6912	73.77%
5	0.7190	73.10%	0.6661	74.13%
6	0.6658	75.53%	0.6352	76.37%
7	0.6380	76.80%	0.5872	78.27%
8	0.6035	78.14%	0.5618	78.73%
9	0.5919	78.33%	0.5418	79.70%
10	0.5717	78.68%	0.5262	80.53%
11	0.5518	79.70%	0.5132	80.57%
12	0.5345	80.67%	0.5064	80.93%
13	0.5277	80.84%	0.4982	81.17%
14	0.5159	81.31%	0.4949	81.77%
15	0.5020	81.36%	0.4901	82.03%
16	0.4985	81.71%	0.4837	81.57%
17	0.4781	82.43%	0.4949	81.77%
18	0.4781	82.53%	0.4634	82.57%
19	0.4637	83.06%	0.4699	82.23%
20	0.4595	83.37%	0.4644	82.70%
21	0.4567	83.36%	0.4557	82.80%
22	0.4449	83.60%	0.4520	83.20%
23	0.4412	84.10%	0.4500	83.50%
24	0.4432	83.93%	0.4456	83.53%
25	0.4286	84.39%	0.4407	84.30%
26	0.4302	84.42%	0.4412	83.83%
27	0.4225	84.65%	0.4409	83.90%
28	0.4194	84.75%	0.4393	84.00%
29	0.4241	84.55%	0.4382	83.97%
30	0.4155	84.72%	0.4381	83.90%

---

Training Curves Visualization

The graphs below illustrate the progression of Loss and Accuracy across all 30 training epochs:

The model achieves stable convergence with a peak validation accuracy of 84.30% and minimal overfitting.

Final Evaluation Results from TestSet

• Test Loss: 0.4381
• Test Accuracy: 83.90%
• Macro F1-Score: 83.95%

---

Usage

from PIL import Image
from torchvision import transforms
import torch.nn.functional as F
import matplotlib.pyplot as plt
import glob
import random
import torch
from transformers import AutoModelForImageClassification

# Step 1: Download and load the model from Hugging Face Hub first
print("Loading model from Hugging Face...")
model = AutoModelForImageClassification.from_pretrained(
    "TheSon2202/SceneViT-Nano-P8-64", trust_remote_code=True
)
model.eval()
print("Model loaded successfully!")


# Step 2: Define a clean prediction and visualization function using direct categories list
def predict_scene(
    model,
    image_search_path="/kaggle/input/datasets/thangcpd/intelimageclassification/data/seg_train/seg_train/**/*.jpg",
):
  # Get image size from the model's configuration
  image_size = getattr(model.config, "image_size", 224)

  # Define image preprocessing pipeline
  test_transform = transforms.Compose([
      transforms.Resize((image_size, image_size)),
      transforms.ToTensor(),
      transforms.Normalize(
          mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]
      ),
  ])

  # Search for image paths in the dataset
  image_paths = glob.glob(image_search_path, recursive=True)
  if len(image_paths) == 0:
    image_paths = glob.glob("/kaggle/input/**/*.jpg", recursive=True)

  if len(image_paths) == 0:
    print("No image files found in the specified path.")
    return

  # Randomly pick an image and preprocess it
  img_path = random.choice(image_paths)
  raw_image = Image.open(img_path).convert("RGB")
  input_tensor = test_transform(raw_image).unsqueeze(0)

  # Run inference
  with torch.no_grad():
    outputs = model(input_tensor)
    logits = outputs.logits
    probabilities = F.softmax(logits, dim=-1)

    pred_idx = torch.argmax(probabilities, dim=-1).item()
    confidence = probabilities[0][pred_idx].item()

  # Directly use the fixed categories list matching model training
  categories = ["buildings", "forest", "glacier", "mountain", "sea", "street"]
  predicted_label = (
      categories[pred_idx] if pred_idx < len(categories) else str(pred_idx)
  )

  # Display the image with result shown on the plot title
  plt.figure(figsize=(4, 4))
  plt.imshow(raw_image)
  plt.title(f"Pred: {predicted_label} ({confidence * 100:.1f}%)", color="green")
  plt.axis("off")
  plt.show()


# Example usage: Call the function using the pre-loaded model
predict_scene(model)

How to use ONNX

!pip install onnxruntime
!pip install onnxscript

onnx_file_path = hf_hub_download(
    repo_id="TheSon2202/SceneViT-Nano-P8-64", filename="vit_model.onnx"
)

# read model through ort_session
ort_session = ort.InferenceSession("vit_model.onnx")