zjunlp/llama2-molinst-biotext-7b

This repo contains a low-rank adapter for LLaMA2-7b-chat, trained on the 🥼 biomolecule text instructions from the 🧪 Mol-Instructions dataset.

Instructions for running it can be found at https://github.com/zjunlp/Mol-Instructions.

Please refer to our paper for more details.

🥼 Tasks

Chemical entity recognition

• Find and list all the instances of the chemical entities in the following content: "Both the control and caramiphen groups with double cannulas had significantly shorter latencies to seizure onset than the corresponding groups with single cannula."

  caramiphen
  

Chemical-disease interaction extraction

• You are provided with a set of clinical trial summaries. Extract the chemical-disease relations from the summaries and present your findings in the format of (Subject, Object): "Eating disorders and the associated behavioural problems and drug abuse are uncommon in pregnancy. When they do occur they are often unrecognized because of denial but when significant may pose a risk to both the mother and her fetus. This case illustrates a number of problems that may be encountered in women with eating disorders in pregnancy, including prolonged and recurrent metabolic disturbances and diuretic abuse. In particular it illustrates the derangements of thyroid function seen in pregnant women with eating disorders and reminds us that when a cause for thyrotoxicosis remains obscure, thyroxine abuse should be considered and explored."

  (thyroxine, thyrotoxicosis) 
  

Chemical-protein interaction extraction

• Your task is to extract the subject and object entities from the provided text and establish the corresponding relation triples. Please format your answers as (Subject, Relation, Object): "Opioid receptors are the membrane proteins that mediate the pain-relieving effect of opioid drugs, such as morphine and fentanyl as well as endogenous opioid peptides enkephalins and endorphins. Using cDNAs for the mu and the kappa opioid receptors, we mapped the chromosomal locations of their genes in mouse. Multilocus cross analysis located the mu receptor gene Oprm on Chr 10 and the kappa receptor gene Oprk1 on Chr 1. Both genes are near centromere, with no markers more centromeric. These data indicate that the two opioid receptors are different gene products, ruling out the possibility that they may be differential splicing products from the same gene."

  (morphine, regulator, Opioid receptors), (fentanyl, regulator, Opioid receptors)
  

Multiple-choice question

• Which of the following conditions does not show multifactorial inheritance? What of the following is the right choice? (A) Pyloric stenosis (B) Schizophrenia (C) Spina bifida (neural tube defects) (D) Marfan syndrome

  The final answer is (D).
  

True or False question

• Does feeding tube insertion and its timing improve survival?

  No, neither insertion of PEG tubes nor timing of insertion affect survival.
  

Open question

• What virus causes chronic disease?

  Hepatitis C virus (HCV).
  

📝 Demo

As illustrated in our repository, we provide an example to perform generation.

>> python generate.py \
    --CLI True \
    --protein False\
    --load_8bit \
    --base_model $BASE_MODEL_PATH \
    --lora_weights $FINETUNED_MODEL_PATH \

Please download Llama-2-7b-chat to obtain the pre-training weights of LlamA-2-7b-chat, refine the --base_model to point towards the location where the model weights are saved.

For model fine-tuned on biomolecular text instructions, set $FINETUNED_MODEL_PATH to 'zjunlp/llama2-molinst-molecule-7b'.

🚨 Limitations

The current state of the model, obtained via instruction tuning, is a preliminary demonstration. Its capacity to handle real-world, production-grade tasks remains limited.

📚 References

If you use our repository, please cite the following related paper:

@inproceedings{fang2023mol,
  author       = {Yin Fang and
                  Xiaozhuan Liang and
                  Ningyu Zhang and
                  Kangwei Liu and
                  Rui Huang and
                  Zhuo Chen and
                  Xiaohui Fan and
                  Huajun Chen},
  title        = {Mol-Instructions: {A} Large-Scale Biomolecular Instruction Dataset
                  for Large Language Models},
  booktitle    = {{ICLR}},
  publisher    = {OpenReview.net},
  year         = {2024},
  url          = {https://openreview.net/pdf?id=Tlsdsb6l9n}
}

🫱🏻‍🫲🏾 Acknowledgements

We appreciate LLaMA-2, LLaMA, Huggingface Transformers Llama, Alpaca, Alpaca-LoRA, Chatbot Service and many other related works for their open-source contributions.