GitHub repository: KishalanPather/IBPE-tokenizer
Primary language: Python
Created: 2024-12-24T23:47:51Z
Description: This was a capstone project for the 2024 final semester computer science course CSC3003S at UCT. The program handles tokenizing, which is the first set step LLM's models like ChatGPT perform to derive meaning from what was typed in. I designed and coded the frontend, as well as set up the servers that communicate with a mongoDB database.