Update README.md

README.md

@@ -3,4 +3,41 @@ license: mit
 language:
 - en
 pipeline_tag: image-to-text
----
+---
+
+## **Description**
+
+It is a ViT model that has been fine-tuned on a **Stable Diffusion 2.0** image dataset and applied **LORA**.   
+It produces optimal results in a reasonable time. Moreover, its implementation with Pytorch is straightforward.
+
+
+<img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/lora-assets/latent-diffusion.png" alt="Image" width="600">
+
+* Reference: *https://huggingface.co/blog/lora*
+
+## **Usage**
+
+```python
+# Libraries
+from transformers import ViTFeatureExtractor, AutoTokenizer, VisionEncoderDecoderModel
+
+# Model
+model = VisionEncoderDecoderModel.from_pretrained("nttdataspain/vit-gpt2-coco-lora")
+tokenizer = AutoTokenizer.from_pretrained("nttdataspain/vit-gpt2-coco-lora")
+feature_extractor = ViTFeatureExtractor.from_pretrained("nttdataspain/vit-gpt2-coco-lora")
+
+# Predict function
+def predict_prompts(list_images, max_length=16):
+    model.eval()
+    pixel_values = feature_extractor(images=list_images, return_tensors="pt").pixel_values
+    with torch.no_grad():
+        output_ids = model.generate(pixel_values, max_length=max_length, num_beams=4, return_dict_in_generate=True).sequences
+
+    preds = tokenizer.batch_decode(output_ids, skip_special_tokens=True)
+    preds = [pred.strip() for pred in preds]
+    return preds
+
+# Get an image and predict
+img = Image.open(image_path).convert('RGB')
+pred_prompts = predict_prompts([img], max_length=16)
+```