Spaces:
Running
Running
Upload CSIT321Project-Info.txt
Browse files- CSIT321Project-Info.txt +129 -0
CSIT321Project-Info.txt
ADDED
@@ -0,0 +1,129 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
**CSIT321 Project FYPS1-21**
|
2 |
+
## Our Refineverse Plugin URL on Hugging Face's Spaces
|
3 |
+
https://huggingface.co/spaces/Nik97/Refineverse
|
4 |
+
|
5 |
+
## Our Source code Repository:
|
6 |
+
https://github.com/Nikosgo/FYP-S1-21-Refineverse.git
|
7 |
+
|
8 |
+
## Document Collaboration:
|
9 |
+
PRD: https://docs.google.com/document/d/1KzWv19fPhlArzDB4qHAZJvnFYp4dYmLc6hcRYNzQYpo/edit
|
10 |
+
PDD: https://docs.google.com/document/d/1iaCLy5Lp8uQCk9S3xiU3oQ-k3_SXKCMH_BroqT468MI/edit#
|
11 |
+
Progress Report: https://docs.google.com/document/d/1Oz7se17k6ZgRiOpJKM1JsYvRBPzr0PIQNATsm_kHGqw/edit?usp=sharing
|
12 |
+
User Manual: https://docs.google.com/document/d/1Xo8E8Xlls_Kg31RyLLfvCPqADh-9agRHGi-efjHB91k/edit?usp=sharing
|
13 |
+
|
14 |
+
|
15 |
+
---
|
16 |
+
|
17 |
+
## IMPORTANT! Rules to follow during development:
|
18 |
+
1. Only Commit to your own branch/workspace under Source control.(Message..to commit on "your name")
|
19 |
+
2. Please DO NOT push or commit to "Main" branch (very Important)
|
20 |
+
3. Before doing anything with code, always select Pull From "Main" under Source Control
|
21 |
+
4. Make sure you are in your own branch/workspace before doing anything.
|
22 |
+
|
23 |
+
---
|
24 |
+
|
25 |
+
## SET UP
|
26 |
+
1. Install Vscode
|
27 |
+
2. Install XAMPP https://www.apachefriends.org/download.html
|
28 |
+
|
29 |
+
---
|
30 |
+
|
31 |
+
## In Vscode, before cloning repository:
|
32 |
+
1. Make sure to install Git https://git-scm.com/downloads (Ignore if Git already installed)
|
33 |
+
2. Install Git Extension Pack, Extension Pack for Java, Jira and Bitbucket(Atlassian Labs).
|
34 |
+
3. Install PHP Extensions.
|
35 |
+
|
36 |
+
## In Vscode, cloning repository:
|
37 |
+
1. In New Window, Select "Clone Git Repository..."
|
38 |
+
2. Copy & paste https://github.com/Nikosgo/FYP-S1-21-Refineverse.git in github url option.
|
39 |
+
3. Click and select "Clone From github". Do not select clone from URL!
|
40 |
+
4. Select local repository location as C:\xampp\htdocs (Xampp)
|
41 |
+
5. Prompt to login in to Atlassian Account, Do so.
|
42 |
+
|
43 |
+
## Setting Git Credentials and Access:
|
44 |
+
1. Select and Open New Terminal under Terminal Tab.
|
45 |
+
2. Type "cd" command to go root directory of your Git repository.
|
46 |
+
3. Type and run git config --global user.name "your Github username"
|
47 |
+
4. Type and run git config --global user.email "your Github email"
|
48 |
+
5. Type and run git config --global credential.helper wincred to store your Github credentials
|
49 |
+
6. Type and run git config --list to verify verify that your Git configuration is set up correctly
|
50 |
+
|
51 |
+
---
|
52 |
+
|
53 |
+
## Selecting Your Branch/Workspace
|
54 |
+
1. In left Side panel of screen, select Source Control
|
55 |
+
2. Select the 3 dots next to refresh icon and all the other icons.
|
56 |
+
3. Select Checkout to origin/yourname
|
57 |
+
|
58 |
+
---
|
59 |
+
|
60 |
+
## To be done each time before any coding to get most updated version of source codes
|
61 |
+
1. In left Side panel of screen, select Source Control
|
62 |
+
2. Select the 3 dots next to refresh icon and all the other icons.
|
63 |
+
3. Under "Pull,Push" select "Pull From..."
|
64 |
+
4. Choose "origin/main"
|
65 |
+
|
66 |
+
---
|
67 |
+
|
68 |
+
## How to push commit to your codes?
|
69 |
+
1. Under File tab, make sure "Autosave" is checked to automatically save your codes
|
70 |
+
2. Under Source Control, ensure message box shows "Message (Ctrl+Enter to commit on "yourname")
|
71 |
+
3. Below blue commit button, under Changes select the "+" sign to stage changes.
|
72 |
+
4. Write a message for the change and Select blue Commit Button.
|
73 |
+
5. Select Sync Changes to update local repository.
|
74 |
+
|
75 |
+
---
|
76 |
+
|
77 |
+
## Setting up a Python Virtual Environment (in VS Code)
|
78 |
+
To run/debug our project's NLP-related Python code, you must first set up a Python virtual environment.
|
79 |
+
Alternatively, you can install the libraries locally on your machine (not recommended).
|
80 |
+
**Important!** At the very least you must have Python installed locally on your machine and the Python extension enabled in VS Code.
|
81 |
+
The library installation commands listed here are meant for Windows machines. Do find the approriate commands for Linux/Mac yourself.
|
82 |
+
|
83 |
+
When running the virtual environment, you might run into an error intially when running the activate script due to Window's PowerShell. (Ignore if not using a Windows machine)
|
84 |
+
You can rectify this by changing the current execution policy for running scripts.
|
85 |
+
1. Open up Window's PowerShell (not command prompt!) **as an administrator**.
|
86 |
+
2. Use Get-ExecutionPolicy to check the current execution policy.
|
87 |
+
3. If it is "Restricted", set it to either "RemoteSigned" or "Unrestricted" using "Set-ExecutionPolicy RemoteSigned" or "Set-ExecutionPolicy Unrestricted".
|
88 |
+
4. You can change it back after you are done with the virtual environment using "Set-ExecutionPolicy Restricted".
|
89 |
+
|
90 |
+
Now you can open up VS Code to create the virutal environment.
|
91 |
+
1. In VS Code, use the shortcut "Ctrl+Shift+P" to open up the command palette.
|
92 |
+
2. Search for "Python: Create Environment..." and click it.
|
93 |
+
3. When asked to pick an enviroment type, choose "Venv".
|
94 |
+
4. When asked for a Python installation to use you should only have one option listed which is the Python version installed on your machine locally. Choose that option.
|
95 |
+
5. A notification will appear to show the progress of the environment creation. After creation, a folder will appear in your workplace automatically, and the Python intepreter should be changed automatically.
|
96 |
+
6. Verify if your Python has been changed to the new environment by looking at the bottom-right of your VS Code. It should say something like "Python 3.11.3 ('venv':venv)". If not, click on it and change it to the correct environment.
|
97 |
+
7. Now you can start installing the neccessary libraries by opening a new terminal window in VS Code. Navigate to the Terminal tab and choose "New Terminal". The new environment should be reflected in the terminal.
|
98 |
+
8. Install Pytorch (CPU): Pytorch (CPU): pip3 install torch torchvision torchaudio
|
99 |
+
9. Install Transformers: pip install transformers
|
100 |
+
11. These libraries are the bare minimum for running NLP code using HuggingFace's transformers. You can now try running some test code to verify.
|
101 |
+
|
102 |
+
To note:
|
103 |
+
By default, the virtual enviroment will not be detected as a new change/commit due to the automatically generated .gitignore file.
|
104 |
+
Please **do not** attempt to change this and upload your environment to GitHub as it may cause errors!
|
105 |
+
|
106 |
+
---
|
107 |
+
|
108 |
+
## Managing your Virtual Environment
|
109 |
+
As the path of the Python installation used to build the environment is user-specific, it cannot be shared.
|
110 |
+
Thus, the virtual environment must be maintained and installed properly by yourself.
|
111 |
+
Feel free to rename your virtual environment anything you want.
|
112 |
+
|
113 |
+
If you wish to install more libraries to test/debug new features (such as Flask or Beautiful Soup) go ahead.
|
114 |
+
However, do communicate with the team if a new library is being used to enable team members to run your code as well.
|
115 |
+
The final required Python libraries for our project will be included in a requirements.txt file.
|
116 |
+
If for some reason you encounter issues such as library conflicts or errors, feel free to remove your environment and follow the steps to create a new one again.
|
117 |
+
|
118 |
+
---
|
119 |
+
|
120 |
+
## Installing your libraries
|
121 |
+
1. Ensure that you have already setup and activated your virtual environment.
|
122 |
+
2. In the VsCode terminal, type the command "pip install -r requirements.txt" and click enter to allow the system to automatically install all necessary libraries that have been defined in the requirements.txt file in order to run the application.
|
123 |
+
|
124 |
+
---
|
125 |
+
|
126 |
+
## Creating or Saving a requirements.txt for library installation
|
127 |
+
1. Ensure that you have already setup and activated your virtual environment.
|
128 |
+
2. Delete any existing requirements.txt file as the command does not overwrite the text file.
|
129 |
+
2. In the VsCode terminal, type the command "pip freeze > requirements.txt" and click enter to save all libraries that have already been installed into the requirements.txt file.
|