update model card

README.md

@@ -1,3 +1,251 @@
----
-license: apache-2.0
----
+---
+license: apache-2.0
+library_name: diffusers
+tags:
+  - text-to-image
+  - image-to-image
+  - image-editing
+  - diffusers
+  - lora
+  - peft
+  - reinforcement-learning
+  - rubric-policy-optimization
+  - auto-rubric
+base_model:
+  - black-forest-labs/FLUX.1-dev
+  - Qwen/Qwen-Image-Edit
+---
+
+# ARR-RPO
+
+[Project Page](#) | [Code](#) | [Paper](#) | [Model Weights](https://huggingface.co/OpenEnvisionLab/ARR-RPO)
+
+## Model Description
+
+ARR-RPO provides two LoRA adapters trained with **Auto-Rubric as Reward (ARR)** and **Rubric Policy Optimization (RPO)** for visual generation:
+
+- **`ARR-FLUX.1-dev/`**: a LoRA adapter for FLUX.1-dev text-to-image generation.
+- **`ARR-Qwen-Image-Edit/`**: a LoRA adapter for Qwen-Image-Edit instruction-guided image editing.
+
+ARR-RPO uses a frozen VLM judge conditioned on explicit auto-generated rubrics. During RPO training, two candidate outputs are sampled for the same prompt or edit instruction, the ARR judge selects the preferred output, and the preferred/dispreferred candidates receive binary rewards. The goal is to improve prompt faithfulness, visual quality, compositional alignment, and edit fidelity without training a separate scalar reward model.
+
+## Model Details
+
+| Adapter | Base model | Task | LoRA rank | LoRA alpha | Framework |
+| --- | --- | --- | --- | --- | --- |
+| `ARR-FLUX.1-dev` | `black-forest-labs/FLUX.1-dev` | Text-to-image | 16 | 32 | Diffusers + PEFT |
+| `ARR-Qwen-Image-Edit` | `Qwen/Qwen-Image-Edit` | Image editing | 32 | 64 | Diffusers + PEFT |
+
+### Adapter Files
+
+```text
+ARR-RPO/
+  ARR-FLUX.1-dev/
+    adapter_config.json
+    adapter_model.safetensors
+  ARR-Qwen-Image-Edit/
+    adapter_config.json
+    adapter_model.safetensors
+```
+
+### FLUX Adapter Targets
+
+The FLUX LoRA adapter is configured for `FluxTransformer2DModel` and targets attention and feed-forward modules, including:
+
+```text
+attn.to_q, attn.to_k, attn.to_v, attn.to_out.0,
+attn.add_q_proj, attn.add_k_proj, attn.add_v_proj, attn.to_add_out,
+ff.net.0.proj, ff.net.2, ff_context.net.0.proj, ff_context.net.2
+```
+
+### Qwen-Image-Edit Adapter Targets
+
+The Qwen-Image-Edit LoRA adapter is configured for `QwenImageTransformer2DModel` and targets attention projection modules, including:
+
+```text
+attn.to_q, attn.to_k, attn.to_v, attn.to_out.0,
+attn.add_q_proj, attn.add_k_proj, attn.add_v_proj, attn.to_add_out
+```
+
+## Intended Use
+
+These adapters are intended for research and development on:
+
+- improving text-to-image generation with rubric-guided preference rewards;
+- improving instruction-guided image editing while preserving source-image content;
+- studying Auto-Rubric as an interpretable alternative to scalar reward models;
+- reproducing and extending ARR-RPO experiments.
+
+They are not intended for safety-critical, medical, legal, or identity-sensitive decision-making. Generated or edited images should be reviewed before use in downstream products.
+
+## How ARR-RPO Works
+
+ARR-RPO separates reward construction into explicit criteria and binary preference decisions:
+
+```text
+visual preference examples
+  -> auto-generated rubrics
+  -> verified and structured rubric set
+  -> frozen VLM judge
+  -> pairwise preference decision
+  -> RPO binary reward
+```
+
+For pairwise RPO, the preferred candidate receives `+1.0` and the dispreferred candidate receives `-0.1`.
+
+## Using The Models
+
+Install a recent Diffusers/PEFT environment that supports the corresponding base model.
+
+### FLUX.1-dev LoRA
+
+```python
+import torch
+from diffusers import FluxPipeline
+
+base_model = "black-forest-labs/FLUX.1-dev"
+adapter_repo = "OpenEnvisionLab/ARR-RPO"
+
+pipe = FluxPipeline.from_pretrained(
+    base_model,
+    torch_dtype=torch.bfloat16,
+)
+pipe.load_lora_weights(
+    adapter_repo,
+    subfolder="ARR-FLUX.1-dev",
+)
+pipe.to("cuda")
+
+image = pipe(
+    "A cinematic portrait of a ceramic robot chef in a warm kitchen.",
+    guidance_scale=3.5,
+    num_inference_steps=30,
+).images[0]
+image.save("arr_flux_example.png")
+```
+
+### Qwen-Image-Edit LoRA
+
+```python
+import torch
+from PIL import Image
+from diffusers import QwenImageEditPipeline
+
+base_model = "Qwen/Qwen-Image-Edit"
+adapter_repo = "OpenEnvisionLab/ARR-RPO"
+
+pipe = QwenImageEditPipeline.from_pretrained(
+    base_model,
+    torch_dtype=torch.bfloat16,
+)
+pipe.load_lora_weights(
+    adapter_repo,
+    subfolder="ARR-Qwen-Image-Edit",
+)
+pipe.to("cuda")
+
+source = Image.open("source.png").convert("RGB")
+image = pipe(
+    image=source,
+    prompt="Replace the sky with a sunset while preserving the building.",
+    num_inference_steps=30,
+).images[0]
+image.save("arr_qwen_edit_example.png")
+```
+
+If your Diffusers version uses a different Qwen-Image-Edit pipeline class or call signature, keep the same adapter subfolder and follow the base model's official loading example.
+
+## Effective Prompting
+
+### FLUX Text-to-Image
+
+The FLUX adapter works best with prompts that clearly specify:
+
+- required objects and attributes;
+- object counts;
+- spatial relationships;
+- style or medium;
+- constraints that should not be ignored.
+
+Example:
+
+```text
+A high-resolution product photo of two matte blue ceramic cups on a wooden table,
+with the smaller cup to the left of the larger cup, soft window lighting.
+```
+
+### Qwen-Image-Edit
+
+The Qwen-Image-Edit adapter works best with edit instructions that clearly separate the requested change from content that should remain unchanged.
+
+Example:
+
+```text
+Change the shirt color to dark green while preserving the person's face, pose,
+background, lighting, and all other clothing details.
+```
+
+## Training Details
+
+ARR-RPO was trained with LoRA and pairwise online preference optimization.
+
+| Hyperparameter | FLUX.1-dev | Qwen-Image-Edit |
+| --- | --- | --- |
+| Training method | RPO with ARR reward | RPO with ARR reward |
+| Candidates per prompt | 2 | 2 |
+| Positive reward | `1.0` | `1.0` |
+| Negative reward | `0.1` | `0.1` |
+| Learning rate | `5e-5` | `1e-5` |
+| PPO clip range | `0.2` | `0.2` |
+| KL coefficient | `0.01` | `0.02` |
+| Sampling steps during training | 8 | 10 |
+| Optimizer | AdamW | AdamW |
+| Gradient clipping | `1.0` | `1.0` |
+| LoRA rank | 16 | 32 |
+
+The reward judge is a frozen VLM conditioned on auto-generated visual rubrics. No trainable scalar reward model is required.
+
+## Evaluation Summary
+
+ARR-RPO is designed to improve alignment with multi-dimensional visual preferences. In the associated experiments, ARR-RPO improves over the corresponding unaligned base models on text-to-image and image-editing benchmarks, with gains attributed to explicit rubric-conditioned reward signals rather than opaque scalar regression.
+
+Recommended evaluation axes include:
+
+- text-to-image prompt adherence and compositional correctness;
+- image-edit instruction fulfillment;
+- source-image preservation for editing;
+- artifact control and visual coherence;
+- pairwise human or VLM preference accuracy;
+- position-bias checks by swapping candidate order.
+
+## Limitations
+
+- These are LoRA adapters and require the corresponding base model weights.
+- Output quality still depends on the base model, prompt quality, scheduler, seed, and inference settings.
+- The ARR reward signal depends on the chosen VLM judge and rubric quality.
+- Image editing may still alter unrelated source-image regions, especially under ambiguous instructions.
+- The model card does not guarantee safety filtering; users should apply appropriate content and policy filters for deployment.
+
+## License
+
+The model card metadata declares `apache-2.0`. Users must also comply with the licenses and terms of the base models:
+
+- `black-forest-labs/FLUX.1-dev`
+- `Qwen/Qwen-Image-Edit`
+
+## Citation
+
+If you use these adapters, please cite the ARR-RPO project:
+
+```bibtex
+@misc{visionautorubric2026,
+  title        = {Auto-Rubric as Reward: From Implicit Preference to Explicit Generative Criteria},
+  author       = {Anonymous},
+  year         = {2026},
+  note         = {arXiv coming soon}
+}
+```
+
+## Contact
+
+For questions, issues, or updates, please use the project repository or Hugging Face community tab.