ECOFRI commited on
Commit
7e8c90a
1 Parent(s): 556d97e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +0 -72
README.md CHANGED
@@ -13,80 +13,8 @@ CXR LLaVA is an innovative open-source, multimodal large language model specific
13
  - **Demo Website**: Experience the model in action at [Radiologist App](https://radiologist.app/cxr-llava).
14
 
15
 
16
-
17
-
18
  |Version| Input CXR resolution | Channels | Vision Encoder | Base LLM | Weight
19
  |--|--|--|--|--|--|
20
  | v1.0 | 512x512 | RGB|RN50|LLAMA2-13B-CHAT|Deprecated
21
  |v2.0 (Latest)|512x512|Grayscale|ViT-L/16|LLAMA2-7B-CHAT|[Link](https://huggingface.co/ECOFRI/CXR-LLAVA-v2)
22
 
23
-
24
- ## Usage Guide
25
- ### Importing Packages
26
-
27
- from transformers import AutoModel
28
- from PIL import Image
29
- ### Prepare CXR
30
-
31
- - Ensure you have an CXR image file ready, such as 'img.jpg'.
32
- - Use the following code to load the image
33
-
34
- cxr_image = Image.open(os.path.join(os.path.dirname(__file__), "IMG", "img.jpg"))
35
-
36
- ### Load model
37
- Loading the CXR-LLAVA model is straightforward and can be done in one line of code.
38
-
39
-
40
- model = AutoModel.from_pretrained("ECOFRI/CXR-LLAVA-v2", trust_remote_code=True)
41
- model = model.to("cuda")
42
-
43
-
44
- ### Generating Radiologic Reports
45
-
46
- To write a radiologic report of a chest radiograph:
47
-
48
-
49
-
50
- response = model.write_radiologic_report(cxr_image)
51
-
52
- > The radiologic report reveals a large consolidation in the right upper lobe of the lungs. There is no evidence of pleural effusion or pneumothorax. The cardiac and mediastinal contours are normal.
53
-
54
-
55
- ### Differential Diagnosis
56
- For differential diagnosis:
57
-
58
-
59
- model.write_differential_diagnosis(cxr_image)
60
-
61
- > Possible differential diagnoses for this patient include pneumonia,tuberculosis, lung abscess, or a neoplastic process such as lung cancer.
62
-
63
- ### Question Answering
64
- To ask a question:
65
-
66
- question = "What is true meaning of consolidation?"
67
- response = model.ask_question(question=question, image=cxr_image)
68
-
69
- > Consolidation refers to the filling of the airspaces in the lungs with fluid, pus, blood, cells or other substances, resulting in a region of lung tissue that has become dense and solid rather than containing air.
70
-
71
- ## Custom Prompt
72
- For custom interactions:
73
-
74
- img = Image.open(os.path.join(os.path.dirname(__file__), "IMG", "img.jpg"))
75
- chat = [
76
- {"role": "system",
77
- "content": "You are a helpful radiologist. Try to interpret chest x ray image and answer to the question that user provides."},
78
- {"role": "user",
79
- "content": "<image>\nWrite a radiologic report on the given chest radiograph, including information about atelectasis, cardiomegaly, consolidation, pulmonary edema, pleural effusion, and pneumothorax.\n"}
80
- ]
81
- response = model.generate_cxr_repsonse(chat=chat,pil_image=img, temperature=0, top_p=1)
82
-
83
-
84
-
85
- ## License Information
86
-
87
- CXR LLaVA is available under a Creative Commons NonCommercial License.
88
-
89
- Users must obtain the LLAMA-2 license prior to use. More details can be found [here](https://ai.meta.com/resources/models-and-libraries/llama-downloads/).
90
-
91
-
92
- Lastly, we extend our heartfelt thanks to all the contributors of the [LLaVA project](https://llava-vl.github.io/).
 
13
  - **Demo Website**: Experience the model in action at [Radiologist App](https://radiologist.app/cxr-llava).
14
 
15
 
 
 
16
  |Version| Input CXR resolution | Channels | Vision Encoder | Base LLM | Weight
17
  |--|--|--|--|--|--|
18
  | v1.0 | 512x512 | RGB|RN50|LLAMA2-13B-CHAT|Deprecated
19
  |v2.0 (Latest)|512x512|Grayscale|ViT-L/16|LLAMA2-7B-CHAT|[Link](https://huggingface.co/ECOFRI/CXR-LLAVA-v2)
20