Update README.md

README.md

@@ -32,6 +32,76 @@ The following Colab notebook showcases how you can finetune the model with your
 - This model is a multimodal language model fine-tuned for the task of detecting tables in images given textual prompts. The model utilizes a combination of image and text inputs to predict bounding boxes around tables within the provided images. 
 - The primary purpose of this model is to assist in automating the process of table detection within images. It can be utilized in various applications such as document processing, data extraction, and image analysis, where identifying tables within images is essential.
 
+## How to Get Started with the Model
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+In Transformers, you can load the model as follows:
+
+```python
+from transformers import AutoProcessor, AutoModelForCausalLM
+import matplotlib.pyplot as plt
+import matplotlib.patches as patches
+
+model_id = "ucsahin/Florence-2-large-TableDetection"
+model = AutoModelForCausalLM.from_pretrained(model_id, trust_remote_code=True, device_map="cuda") # load the model on GPU
+processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
+
+def run_example(task_prompt, image, max_new_tokens=128):
+    prompt = task_prompt
+    inputs = processor(text=prompt, images=image, return_tensors="pt")
+    generated_ids = model.generate(
+      input_ids=inputs["input_ids"].cuda(),
+      pixel_values=inputs["pixel_values"].cuda(),
+      max_new_tokens=max_new_tokens,
+      early_stopping=False,
+      do_sample=False,
+      num_beams=3,
+    )
+    generated_text = processor.batch_decode(generated_ids, skip_special_tokens=False)[0]
+    parsed_answer = processor.post_process_generation(
+        generated_text,
+        task=task_prompt,
+        image_size=(image.width, image.height)
+    )
+    return parsed_answer
+
+def plot_bbox(image, data):
+   # Create a figure and axes
+    fig, ax = plt.subplots()
+
+    # Display the image
+    ax.imshow(image)
+
+    # Plot each bounding box
+    for bbox, label in zip(data['bboxes'], data['labels']):
+        # Unpack the bounding box coordinates
+        x1, y1, x2, y2 = bbox
+        # Create a Rectangle patch
+        rect = patches.Rectangle((x1, y1), x2-x1, y2-y1, linewidth=1, edgecolor='r', facecolor='none')
+        # Add the rectangle to the Axes
+        ax.add_patch(rect)
+        # Annotate the label
+        plt.text(x1, y1, label, color='white', fontsize=8, bbox=dict(facecolor='red', alpha=0.5))
+
+    # Remove the axis ticks and labels
+    ax.axis('off')
+
+    # Show the plot
+    plt.show()
+
+########### Inference
+from datasets import load_dataset
+
+dataset = load_dataset("ucsahin/pubtables-detection-1500-samples")
+
+example_id = 5
+image = dataset["train"][example_id]["image"]
+
+parsed_answer = run_example("<OD>", image=image)
+
+plot_bbox(image, parsed_answer["<OD>"])
+```
+
 
 ### Training hyperparameters