Improve Model Card with Model Details and Links

README.md

@@ -33,125 +33,72 @@ model-index:
       name: Accuracy
 ---
 
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
+# Model Card for LLaVA-1.6-Mistral-7B-Offensive-Meme-Singapore
 
+This model is described in the paper [Detecting Offensive Memes with Social Biases in Singapore Context Using Multimodal Large Language Models](https://arxiv.org/abs/2502.18101). It classifies memes as offensive or not offensive, specifically within the Singaporean context.
 
 ## Model Details
 
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
+This model is a fine-tuned Vision-Language Model (VLM) designed to detect offensive memes in the Singaporean context. It leverages the strengths of VLMs to handle the nuanced and culturally specific nature of meme interpretation, addressing the limitations of traditional content moderation systems. The model was fine-tuned on a dataset of 112K memes labeled by GPT-4V. The fine-tuning process involved a pipeline incorporating OCR, translation, and a 7-billion parameter VLM (LLaVA-v1.6-Mistral-7b-hf). The resulting model demonstrates strong performance in offensive meme detection, achieving high accuracy and AUROC scores on a held-out test set.
 
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
+- **Developed by:** Cao Yuxuan, Wu Jiayang, Alistair Cheong Liang Chuen, Bryan Shan Guanrong, Theodore Lee Chong Jen, and Sherman Chann Zhi Shen
+- **Model type:** Fine-tuned Vision-Language Model (VLM)
+- **Language(s) (NLP):** English (with multilingual capabilities through the pipeline)
+- **License:** MIT
+- **Finetuned from model:** llava-hf/llava-v1.6-mistral-7b-hf
+- **Repository:** https://github.com/aliencaocao/vlm-for-memes-aisg
+- **Paper:** [Detecting Offensive Memes with Social Biases in Singapore Context Using Multimodal Large Language Models](https://arxiv.org/abs/2502.18101)
 
 ## Uses
 
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
 ### Direct Use
 
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
+The model can be used directly for classifying memes as offensive or non-offensive. Input is expected to be a meme image. The model processes this using OCR and translation where necessary, then utilizes a VLM for classification.
 
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+### Downstream Use
 
-[More Information Needed]
+This model can be integrated into larger content moderation systems to enhance the detection of offensive memes, specifically targeting the Singaporean context.
 
 ### Out-of-Scope Use
 
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
+This model is specifically trained for the Singaporean context. Its performance may degrade significantly when applied to memes from other cultures or regions. It is also not suitable for general-purpose image classification tasks.
 
 ## Bias, Risks, and Limitations
 
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
+The model's performance is inherently tied to the quality and representativeness of the training data. Biases present in the training data may be reflected in the model's output, particularly regarding the interpretation of culturally specific humor or references. The model may misclassify memes due to ambiguities in language or visual representation. It is crucial to use this model responsibly and acknowledge its limitations.
 
 ### Recommendations
 
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+Users should be aware of the potential biases and limitations of the model. Human review of the model's output is strongly recommended, especially in high-stakes scenarios. Further research into mitigating bias and enhancing robustness is needed.
 
 ## How to Get Started with the Model
 
-Use the code below to get started with the model.
-
 [More Information Needed]
 
 ## Training Details
 
 ### Training Data
 
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
 [More Information Needed]
 
 ### Training Procedure
 
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
 [More Information Needed]
 
 ## Evaluation
 
-<!-- This section describes the evaluation protocols and provides the results. -->
-
 ### Testing Data, Factors & Metrics
 
 #### Testing Data
 
-<!-- This should link to a Dataset Card if possible. -->
-
 [More Information Needed]
 
 #### Factors
 
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
 [More Information Needed]
 
 #### Metrics
 
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
 [More Information Needed]
 
 ### Results
@@ -160,67 +107,43 @@ Use the code below to get started with the model.
 
 #### Summary
 
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
 [More Information Needed]
 
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
+## Model Examination
 
 [More Information Needed]
 
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
+## Environmental Impact
 
 [More Information Needed]
 
-#### Software
+## Technical Specifications
 
 [More Information Needed]
 
-## Citation [optional]
+## Citation
 
 ```
 @misc{yuxuan2025detectingoffensivememessocial,
-      title={Detecting Offensive Memes with Social Biases in Singapore Context Using Multimodal Large Language Models}, 
+      title={Detecting Offensive Memes with Social Biases in Singapore Context Using Multimodal Large Language Models},
       author={Cao Yuxuan and Wu Jiayang and Alistair Cheong Liang Chuen and Bryan Shan Guanrong and Theodore Lee Chong Jen and Sherman Chann Zhi Shen},
       year={2025},
       eprint={2502.18101},
       archivePrefix={arXiv},
       primaryClass={cs.CV},
-      url={https://arxiv.org/abs/2502.18101}, 
+      url={https://arxiv.org/abs/2502.18101},
 }
 ```
 
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+## Glossary
 
 [More Information Needed]
 
-## More Information [optional]
+## More Information
 
 [More Information Needed]
 
-## Model Card Authors [optional]
+## Model Card Authors
 
 [More Information Needed]