KishoreKommi commited on
Commit
eb2f76b
Β·
1 Parent(s): e99f2c1

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +9 -134
README.md CHANGED
@@ -1,134 +1,9 @@
1
- # privateGPT
2
- Ask questions to your documents without an internet connection, using the power of LLMs. 100% private, no data leaves your execution environment at any point. You can ingest documents and ask questions without an internet connection!
3
-
4
- Built with [LangChain](https://github.com/hwchase17/langchain), [GPT4All](https://github.com/nomic-ai/gpt4all), [LlamaCpp](https://github.com/ggerganov/llama.cpp), [Chroma](https://www.trychroma.com/) and [SentenceTransformers](https://www.sbert.net/).
5
-
6
- <img width="902" alt="demo" src="https://user-images.githubusercontent.com/721666/236942256-985801c9-25b9-48ef-80be-3acbb4575164.png">
7
-
8
- # Environment Setup
9
- In order to set your environment up to run the code here, first install all requirements:
10
-
11
- ```shell
12
- pip3 install -r requirements.txt
13
- ```
14
-
15
- Then, download the LLM model and place it in a directory of your choice:
16
- - LLM: default to [ggml-gpt4all-j-v1.3-groovy.bin](https://gpt4all.io/models/ggml-gpt4all-j-v1.3-groovy.bin). If you prefer a different GPT4All-J compatible model, just download it and reference it in your `.env` file.
17
-
18
- Rename `example.env` to `.env` and edit the variables appropriately.
19
- ```
20
- MODEL_TYPE: supports LlamaCpp or GPT4All
21
- PERSIST_DIRECTORY: is the folder you want your vectorstore in
22
- MODEL_PATH: Path to your GPT4All or LlamaCpp supported LLM
23
- MODEL_N_CTX: Maximum token limit for the LLM model
24
- EMBEDDINGS_MODEL_NAME: SentenceTransformers embeddings model name (see https://www.sbert.net/docs/pretrained_models.html)
25
- TARGET_SOURCE_CHUNKS: The amount of chunks (sources) that will be used to answer a question
26
- ```
27
-
28
- Note: because of the way `langchain` loads the `SentenceTransformers` embeddings, the first time you run the script it will require internet connection to download the embeddings model itself.
29
-
30
- ## Test dataset
31
- This repo uses a [state of the union transcript](https://github.com/imartinez/privateGPT/blob/main/source_documents/state_of_the_union.txt) as an example.
32
-
33
- ## Instructions for ingesting your own dataset
34
-
35
- Put any and all your files into the `source_documents` directory
36
-
37
- The supported extensions are:
38
-
39
- - `.csv`: CSV,
40
- - `.docx`: Word Document,
41
- - `.doc`: Word Document,
42
- - `.enex`: EverNote,
43
- - `.eml`: Email,
44
- - `.epub`: EPub,
45
- - `.html`: HTML File,
46
- - `.md`: Markdown,
47
- - `.msg`: Outlook Message,
48
- - `.odt`: Open Document Text,
49
- - `.pdf`: Portable Document Format (PDF),
50
- - `.pptx` : PowerPoint Document,
51
- - `.ppt` : PowerPoint Document,
52
- - `.txt`: Text file (UTF-8),
53
-
54
- Run the following command to ingest all the data.
55
-
56
- ```shell
57
- python ingest.py
58
- ```
59
-
60
- Output should look like this:
61
-
62
- ```shell
63
- Creating new vectorstore
64
- Loading documents from source_documents
65
- Loading new documents: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 1/1 [00:01<00:00, 1.73s/it]
66
- Loaded 1 new documents from source_documents
67
- Split into 90 chunks of text (max. 500 tokens each)
68
- Creating embeddings. May take some minutes...
69
- Using embedded DuckDB with persistence: data will be stored in: db
70
- Ingestion complete! You can now run privateGPT.py to query your documents
71
- ```
72
-
73
- It will create a `db` folder containing the local vectorstore. Will take 20-30 seconds per document, depending on the size of the document.
74
- You can ingest as many documents as you want, and all will be accumulated in the local embeddings database.
75
- If you want to start from an empty database, delete the `db` folder.
76
-
77
- Note: during the ingest process no data leaves your local environment. You could ingest without an internet connection, except for the first time you run the ingest script, when the embeddings model is downloaded.
78
-
79
- ## Ask questions to your documents, locally!
80
- In order to ask a question, run a command like:
81
-
82
- ```shell
83
- python privateGPT.py
84
- ```
85
-
86
- And wait for the script to require your input.
87
-
88
- ```plaintext
89
- > Enter a query:
90
- ```
91
-
92
- Hit enter. You'll need to wait 20-30 seconds (depending on your machine) while the LLM model consumes the prompt and prepares the answer. Once done, it will print the answer and the 4 sources it used as context from your documents; you can then ask another question without re-running the script, just wait for the prompt again.
93
-
94
- Note: you could turn off your internet connection, and the script inference would still work. No data gets out of your local environment.
95
-
96
- Type `exit` to finish the script.
97
-
98
-
99
- ### CLI
100
- The script also supports optional command-line arguments to modify its behavior. You can see a full list of these arguments by running the command ```python privateGPT.py --help``` in your terminal.
101
-
102
-
103
- # How does it work?
104
- Selecting the right local models and the power of `LangChain` you can run the entire pipeline locally, without any data leaving your environment, and with reasonable performance.
105
-
106
- - `ingest.py` uses `LangChain` tools to parse the document and create embeddings locally using `HuggingFaceEmbeddings` (`SentenceTransformers`). It then stores the result in a local vector database using `Chroma` vector store.
107
- - `privateGPT.py` uses a local LLM based on `GPT4All-J` or `LlamaCpp` to understand questions and create answers. The context for the answers is extracted from the local vector store using a similarity search to locate the right piece of context from the docs.
108
- - `GPT4All-J` wrapper was introduced in LangChain 0.0.162.
109
-
110
- # System Requirements
111
-
112
- ## Python Version
113
- To use this software, you must have Python 3.10 or later installed. Earlier versions of Python will not compile.
114
-
115
- ## C++ Compiler
116
- If you encounter an error while building a wheel during the `pip install` process, you may need to install a C++ compiler on your computer.
117
-
118
- ### For Windows 10/11
119
- To install a C++ compiler on Windows 10/11, follow these steps:
120
-
121
- 1. Install Visual Studio 2022.
122
- 2. Make sure the following components are selected:
123
- * Universal Windows Platform development
124
- * C++ CMake tools for Windows
125
- 3. Download the MinGW installer from the [MinGW website](https://sourceforge.net/projects/mingw/).
126
- 4. Run the installer and select the `gcc` component.
127
-
128
- ## Mac Running Intel
129
- When running a Mac with Intel hardware (not M1), you may run into _clang: error: the clang compiler does not support '-march=native'_ during pip install.
130
-
131
- If so set your archflags during pip install. eg: _ARCHFLAGS="-arch x86_64" pip3 install -r requirements.txt_
132
-
133
- # Disclaimer
134
- This is a test project to validate the feasibility of a fully private solution for question answering using LLMs and Vector embeddings. It is not production ready, and it is not meant to be used in production. The models selection is not optimized for performance, but for privacy; but it is possible to use different models and vectorstores to improve performance.
 
1
+ title: Policegpt1.1
2
+ emoji: πŸ“ˆ
3
+ colorFrom: blue
4
+ colorTo: purple
5
+ sdk: gradio
6
+ sdk_version: 3.33.1
7
+ app_file: app.py
8
+ pinned: false
9
+ license: apache-2.0