Ailee52/PatentSBERTa_finetuned_green_advanced

Green Patent Detection: Advanced Agentic Workflow(LangGraph) with QLoRA

PatentSBERTa - Final Model

Model Description

• This Model is the Final Assignment Model for Green Patent Detection, fine-tuned on top of original model (Ailee52/PatentSBERTa_finetuned_green).
• This model use Multi-Agent System (MAS), locally fine-tuned Mistral-7B with QLoRa adaptet the create the debate LLM (advocal, skeptic, and judge) to debate 100 high risk patent claims, then using Human-in-the-loop (HITL) to review and produce gold dataset for final fine-tuning.

Pipeline Overview

patent_50k_green.parquet ↓

Uncertainty Sampling (top 100 high-risk claims) ↓

(Mistral-7B + QLoRA adapter) ↓

Multi-Agent Debate (LangGraph) Locally Run Debate LLM

[Advocate] → [Skeptic] → [Judge] ↓

Targeted HITL Review (2/100 claims required human final decision)

Gold Dataset (100 labeled claims) ↓

Fine-tune PatentSBERTa (Ailee/PatentSBERTa_finetuned_green) (Silver 30k + Gold 100) ↓

Final Model Upload on HF

Dataset

• Dataset https://huggingface.co/datasets/Ailee52/PatentSBERTa_finetuned_green_final_advanced-dataset
• Green patent 100 claims MAS results + Gold Labels: mas_result_gold.csv

Evaluation Results

• eval_silver (10k): 0.8091
• eval_gold(100): 0.2727
• The F1 score on silver evaluation of 10,000 claims demonstrates improvement from previous fine-tuned model. The gold evaluation set from 100 uncertain claims are getting lower, reflecting the intention selection of most uncertain claims for HITL review, not model failure.

HITL Summary

• Total claims reviewed by MAS: 100
• Claims required human final decision: 2
• Human agreement rate 98%

Disagreement Report

• From 100 high-risk patent claims that processed by Multi-Agent System (MAS), there were 2 claims that judge agent expressed uncertainly in its rationale. Key words using to find uncertain from jude rationale are "unclear", "uncertain", "debateable", "unsure", "however", and "uncertainly", it indicates that the advocate and skeptical arguements reached a deadlock. So, there are 98% autonomous labeling rate.
• The 2 Claims are:

=== doc_id: 9206804 === TEXT:

1. A compressor for vehicle including a housing, the compressor comprising: a compression portion located in the housing; a rotary shaft connected with the compression portion at one end; a motor driving the compression portion through the rotary shaft; a motor accommodation chamber accommodating the motor; a suction port formed in the housing and sucking fluid into the motor accommodation chamber; and a first shaft portion and a second shaft portion formed at both ends of the rotary shaft; wherein the first shaft portion is formed between the compression portion and the motor, and is supported by a first bearing, wherein the second shaft portion is formed between the motor and the housing, and is supported by a second bearing, wherein a coil spring is wound around the second shaft portion so as to rotate with the second shaft portion, wherein the rotary shaft forms a spring supporting portion and the diameter of which is greater than that of the second shaft portion, wherein the second bearing includes an inner race, wherein the coil spring is installed between the inner race of the second bearing and the spring support portion of the rotary shaft, wherein the second shaft portion is relatively movable with respect to the inner race of the second bearing in the axial direction, and wherein the second shaft portion and the second bearing are disposed in the motor accommodation chamber and the suction port is opened to the motor accommodation chamber so as to face the second shaft portion.

ADVOCATE: This patent for a compressor for vehicles could potentially contribute to environmental benefits by improving the efficiency of the vehicle's air conditioning system. By reducing the energy consumption of the compressor, it could lead to lower greenhouse gas emissions, thereby contributing to the reduction of overall carbon footprint. Additionally, the use of a coil spring to support the second shaft portion may improve the durability and longevity of the compressor, thereby reducing the need for frequent replacements and contributing to resource efficiency. This aligns with the CPC Y02 category for green technology, specifically Y02T for "compression machinery for refrigeration or air conditioning."

SKEPTIC: This claim may not be environmentally friendly due to the use of a motor, which consumes energy and contributes to greenhouse gas emissions. Additionally, the use of a coil spring and the need for a suction port suggest that the compressor may not be optimized for energy efficiency, which could lead to increased energy consumption and higher carbon emissions. The claim does not seem to align with the CPC Y02 criteria for "methods or apparatus for the production of energy from renewable resources or for its storage or for the management of energy demand."

JUDGE RATIONALE: While the compressor may contribute to energy efficiency, the use of a motor and the need for a suction port suggest that it may not be optimized for maximum energy efficiency. However, the potential for energy savings and reduced carbon emissions make it a green technology under CPC Y02. CURRENT is_green_tech: 1

HUMAN FINAL DECISION: 0 NON GREEN

============================================================

=== doc_id: 9122206 === TEXT:

1. A liquid toner composition comprising: (a) a carrier liquid; (b) particles comprising (c) a charge director.

ADVOCATE: This patent for a liquid toner composition could potentially contribute to environmental sustainability by reducing emissions and promoting resource efficiency. The liquid toner composition, which includes a charge director, may help improve the efficiency of printing processes, thereby reducing the amount of toner waste and energy consumption. Additionally, the use of a liquid toner instead of traditional powder toner could lead to less packaging waste and potentially easier recycling, contributing to a more sustainable printing process overall.

SKEPTIC: While the claim does not inherently indicate environmental harm, it is important to consider the source and production methods of the carrier liquid and charge director particles. If these components are derived from non-renewable resources or are produced using energy-intensive processes, the environmental impact could be significant. Additionally, the lack of specificity regarding the charge director's composition may suggest a lack of innovation in terms of using more sustainable materials. This claim may not fully meet the CPC Y02 criteria of "providing a solution to an environmental problem" or "promoting the conservation of natural resources."

JUDGE RATIONALE: While the claim does not guarantee environmental benefits, it has the potential to contribute to a more sustainable printing process. However, the environmental impact of the carrier liquid and charge director particles should be further evaluated. CURRENT is_green_tech: 1

HUMAN FINAL DECISION: 0 NON GREEN

============================================================

Training details

• LLM for MAS: Mistral-7B-Instruct-v0.3 (https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3) + QLoRa
• MAS Framework: LangGraph
• Gold dataset: 100 claims
• Silver dataset: 30,000 claims
• epochs: 3
• lr: 2e-5
• batch size: 16
• max length: 256

Multi-Agent System (LangGraph+QLoRa)

including 3 debates LLM:

• Advocate: Briefly explain why this patent could be environmentally beneficial. Focus on sustainability, emissions reduction, resource efficiency, or alignment with CPC Y02 green technology. Respond in 2-3 sentences
• Skeptic: Identify reasons why this claim may NOT be environmentally friendly or may involve greenwashing. Consider potential negative environmental impacts, lack of innovation, or misalignment with CPC Y02 criteria. Respond in 2-3 sentences
• Judge: You are a Judge. Consider both arguments and decide whether the patent qualifies as green technology under CPC Y02. Return ONLY valid JSON (no markdown, no extra text)

F1 Comparative analysis on eval_silver (Model Baseline, Simple LLM fintuned, MAS fintuned, and Advanced finetuned)

| 1. Baseline | Frozen Embedding (No Fine Tuning) | F1 = 0.77 |

| 2. Assignment 2 Model | Silver + Gold (Simple Generic LLM+HITL) | F1 = 0.8009 |

| 3. Assignment 3 Model | Silver + Gold (Multi-Agent Fine-Tuning+HITL) | F1 = 0.8066 |

| 4. Assignment 4 Model | Silver + Gold (QLoRa MAS+HITL) | F1 = 0.8091 |

• Conclusion: This comparative shows the improvement of F1 score of each model's pipeline. Rising from Baseline of 0.77 to 0.8091 of the final model. The integration of QLoRa finetuned Mistral-7B as the MAS brain, combined with HITL (targeted the uncertain judge rationale), produced the highest performing. This validate that agentic debate LLMs with domain-adapted (which specialized or trained on the related data) generated superior training signals compared to generic LLM labeling alone.

Disclaimer

• This project was developed for academic purposes only. The classification results are intended for research and educational use, and should not be interpreted as legal advice or professional patent evaluation. The Human-in-the-Loop (HITL) annotations were performed by students as part of a coursework assignment and do not represent expert legal judgment. The model may contain biases and errors inherited from both automated labeling (silver labels) and LLM-assisted human review.

Usage of Generative AI

• This project utilized Generative AI tools including ChatGPT, Claude, and Grammarly to assist in code implementation, some debuggings, and grammatical corrections thoughout the development process. All the parameter selections, agent configurations, and final judgements, including HITL label decisions were made solely by the author based on personal consideration and understanding of the project.

Video Link for Explaination: https://aaudk-my.sharepoint.com/:v:/g/personal/sm42zm_student_aau_dk/IQDNh3ERCxQERagvi0tuLZXyAbD8_nr6gQ-yrxuGxLv81U0?e=SCfNl9