Update README.md

README.md

@@ -1,3 +1,79 @@
----
-license: apache-2.0
----
+---
+license: apache-2.0
+language: en
+library_name: pytorch
+pipeline_tag: object-detection
+tags:
+- rtdetr
+- object-detection
+
+---
+license: apache-2.0
+language: en
+library_name: pytorch
+pipeline_tag: object-detection
+tags:
+- rtdetr
+- object-detection
+- knowledge-distillation
+- taco-dataset
+- dinov3
+- vit
+---
+
+# RT-DisDINOv3-ViT: A Distilled RT-DETR-L Model
+
+This model is an **RT-DETR-L** whose backbone and encoder have been pre-trained using knowledge distillation from a powerful **DINOv3 ViT-Base** teacher model. The distillation process was performed on feature maps from the [TACO (Trash Annotations in Context)](https://tacodataset.org/) dataset using the `lightly-train` framework.
+
+This pre-trained checkpoint contains the "distilled knowledge" and is intended to be used as a starting point for fine-tuning on downstream object detection tasks.
+
+This work is part of the **RT-DisDINOv3** project. For full details on the training pipeline, baseline comparisons, and analysis, please visit the [main GitHub repository](https://github.com/your-username/your-repo-name). <!--- <<< TODO: Add your GitHub repo link here -->
+
+## How to Use
+
+You can load these distilled weights and apply them to the original RT-DETR-L model's backbone and encoder before fine-tuning.
+
+```python
+import torch
+from torch.hub import load_state_dict_from_url
+
+# 1. Load the original RT-DETR-L model architecture
+# Make sure you have the 'rtdetr' repository cloned locally or installed
+rtdetr_l = torch.hub.load('lyuwenyu/RT-DETR', 'rtdetrv2_l', pretrained=True)
+model = rtdetr_l.model
+
+# 2. Load the distilled weights from this Hugging Face Hub repository
+MODEL_URL = "https://huggingface.co/hnamt/RT-DisDINOv3-ViT-Base/resolve/main/distilled_rtdetr_vit_teacher_BEST.pth"
+distilled_state_dict = load_state_dict_from_url(MODEL_URL, map_location='cpu')['model']
+
+# 3. Load the weights into the model's backbone and encoder
+# The `strict=False` flag ensures that only matching keys (backbone + encoder) are loaded.
+model.load_state_dict(distilled_state_dict, strict=False)
+
+print("Successfully loaded and applied distilled knowledge from ViT teacher!")
+
+# Now the 'model' is ready for fine-tuning on your own dataset.
+# For example:
+# optimizer = torch.optim.AdamW(model.parameters(), lr=1e-4)
+# model.train()
+# ... your fine-tuning loop ...
+```
+
+## Training Details
+
+- **Student Model**: RT-DETR-L (`rtdetrv2_l` from [lyuwenyu/RT-DETR](https://github.com/lyuwenyu/RT-DETR)).
+- **Teacher Model**: DINOv3 ViT-Base (`dinov3/vitb16` via Lightly).
+- **Dataset for Distillation**: TACO dataset images.
+- **Distillation Procedure**: The student model's backbone and encoder were trained to minimize the Mean Squared Error (MSE) between their output feature maps and those of the teacher model, orchestrated by the `lightly-train` library.
+
+## Evaluation Results
+
+After the distillation pre-training, the model was fine-tuned on the TACO dataset. In our experiments, this particular teacher did not yield an improvement over the baseline.
+
+| Model                         | mAP@50-95 | mAP@50 | Speed (ms) | Notes                               |
+| ----------------------------- | :-------: | :----: | :--------: | ----------------------------------- |
+| RT-DETR-L (Baseline)          |   2.80%   | 4.60%  |   50.05    | Fine-tuned from COCO pre-trained.   |
+| **RT-DisDINOv3 (w/ ViT)**     |   2.80%   | 4.20%  |   49.80    | No performance improvement observed.|
+
+## License
+The weights in this repository are released under the Apache 2.0 License. Please be aware that the models used for training (RT-DETR, DINOv3) have their own licenses.