|
--- |
|
library_name: peft |
|
license: mit |
|
language: |
|
- en |
|
pipeline_tag: text-generation |
|
--- |
|
|
|
## BioGPT-Large-Natural-Products-RE-Diversity-1000-synt-v1.1 |
|
|
|
Natural products represent a large pool of bioactive compounds of high interest in drug-discovery. However, these relationships are sparsely distributed across organisms and a growing part |
|
of the literature remains unannotated. This volume necessitates the development of a machine assistant to boost the completion of existing resources. Framing the task as an |
|
end-to-end Relation Extraction (RE), we propose a BioGPT model fined-tuned on synthetic data. See details about this procedure in the join [article](https://arxiv.org/abs/2311.06364). |
|
|
|
The model is a derived from [microsoft/BioGPT-Large](hhttps://huggingface.co/microsoft/BioGPT-Large) and was trained on a synthetic dataset generated with [Vicuna-13b-v1.5](https://huggingface.co/lmsys/vicuna-13b-v1.5), derived from LLaMA2. |
|
|
|
The dataset is available on zenodo. |
|
|
|
Dataset: [](https://doi.org/10.5281/zenodo.10231085) |
|
|
|
You can use the model directly as: |
|
|
|
## Example |
|
|
|
```python |
|
import torch |
|
from transformers import AutoTokenizer, AutoModelForCausalLM, AutoTokenizer |
|
from peft import PeftModel |
|
|
|
model_hf = "microsoft/BioGPT-Large" |
|
lora_adapters = "mdelmas/BioGPT-Large-Natural-Products-RE-Extended-synt-v1.0" |
|
|
|
# Load model and plug adapters using peft |
|
model = AutoModelForCausalLM.from_pretrained(model_hf) |
|
model = PeftModel.from_pretrained(model, lora_adapters) |
|
model = model.merge_and_unload() |
|
tokenizer = AutoTokenizer.from_pretrained(model_hf) |
|
|
|
# Example from PubMed article 24048364 |
|
title_text = "Producers and important dietary sources of ochratoxin A and citrinin." |
|
abstract_text = "Ochratoxin A (OTA) is a very important mycotoxin, and its research is focused right now on the new findings of OTA, like being a complete carcinogen, information about OTA producers and new exposure sources of OTA. Citrinin (CIT) is another important mycotoxin, too, and its research turns towards nephrotoxicity. Both additive and synergistic effects have been described in combination with OTA. OTA is produced in foodstuffs by Aspergillus Section Circumdati (Aspergillus ochraceus, A. westerdijkiae, A. steynii) and Aspergillus Section Nigri (Aspergillus carbonarius, A. foetidus, A. lacticoffeatus, A. niger, A. sclerotioniger, A. tubingensis), mostly in subtropical and tropical areas. OTA is produced in foodstuffs by Penicillium verrucosum and P. nordicum, notably in temperate and colder zones. CIT is produced in foodstuffs by Monascus species (Monascus purpureus, M. ruber) and Penicillium species (Penicillium citrinum, P. expansum, P. radicicola, P. verrucosum). OTA was frequently found in foodstuffs of both plant origin (e.g., cereal products, coffee, vegetable, liquorice, raisins, wine) and animal origin (e.g., pork/poultry). CIT was also found in foodstuffs of vegetable origin (e.g., cereals, pomaceous fruits, black olive, roasted nuts, spices), food supplements based on rice fermented with red microfungi Monascus purpureus and in foodstuffs of animal origin (e.g., cheese)." |
|
text = title_text + " " + abstract_text |
|
|
|
# Tokenization |
|
input_text = text + tokenizer.eos_token + tokenizer.bos_token |
|
input_tokens = tokenizer(input_text, return_tensors='pt') |
|
|
|
# Decoding parameters |
|
EVAL_GENERATION_ARGS = {"max_length": 1024, |
|
"do_sample": False, |
|
"forced_eos_token_id": tokenizer.eos_token_id, |
|
"num_beams": 3, |
|
"early_stopping": "never", |
|
"length_penalty": 1.5, |
|
"temperature": 0} |
|
|
|
# Generation |
|
with torch.no_grad(): |
|
beam_output = model.generate(**input_tokens, **EVAL_GENERATION_ARGS) |
|
output = tokenizer.decode(beam_output[0][len(input_tokens["input_ids"][0]):], skip_special_tokens=True) |
|
|
|
# Parse and print |
|
rels = output.strip().split("; ") |
|
for rel in rels: |
|
print("- " + rel) |
|
``` |
|
|
|
## Citation |
|
|
|
If you find this model useful, please cite: |
|
|
|
```latex |
|
|
|
``` |
|
|
