Package: techdocs
InspirationTechdocsreally started as a passion project due to our indolent nature towards documenting our own codebases. We are currently working as core backend-microservice developers for an open-source organization -Heritiansat our University.Here is our story about howTechdocswas born:Currently, we are the only devs working on the project but now that we have recruited 20+ new students it is inevitable to document our code to increase code readability.At the time of recruitment, our codebase already consisted of 6+ directories and 4k+ lines of code.For the new recruits, understanding our codebase ASAP is imperative.Documenting the code manually is not a possibility because, well, we are lazy and we hate to document.As we both are very well-versed with LLMs our thought was, why don't we automate this boring documentation thing once and for all? And there we have it.OUR LAZINESS IS THE SOLE MOTIVATION FOR TECHDOCSWhat it doesTechdocs is aPyPI packagecapable of automatically generating documentation for Python codebases viaCLI. Techdocs generate the docstrings in place by analyzing the structure, logic, and functionality of code by leveragingAbstract Syntax Tree (AST)for.pyfiles. It leverages the power and efficiency ofLarge Language Models (LLMs)along with carefully crafted API design to accomplish the task.How we built itTechdocs can be broadly classified into 3 major components:Backend/API:
This API is where all the magic happens. It acts as a brain for our product. Developed usingFastAPI, our API is catered to achieve minimum user-perceived latency. Our API hostsWizardLM-70B- a fine-tuned version of theLlama-2model for downstream tasks like Code Understanding, and Code Generation.Deployment: The API is linked with Huggingface spaces via a simple.ymlworkflow. It then uses the power ofDOCKERto containerize the API module, hosting it onHugginface's servers. This way, our public endpoints are exposed and our CLI tool can use them however the user intends.LLM Tools Used: The API uses theWizardLM-70Bmodel served onClarifai's servers. Integration of Prompt Engineering, user inputs and model inference are done with the help ofLangchain.Deployed API Link:https://caffeinecrew-techdocs.hf.space/docsCLI/PyPI Package:
This is the main interface with which users can interact with our product. The tool is a PyPI package which makes it easier to install and use in any Python project. To prepare the package tools likebuild,twine,setuptools, andargparsehave been used. Here are a few things you can do with the CLI tool.$techdocs

>👋Hithere!Welcometotechdocs.Herearesomecoolthingsyoucando:💫tryoutademowithournewGUI🚀andexplorehowtousetheCLI:➡️https://techdocs.streamlit.app/💫signupheretogetanAPIkey👤:$techdocssignup-u<username>-p<password>-e<email>💫usethelinebelowtoissueanewAPIkey🗝️:$techdocsapikey-u<username>-p<password>💫usetheCLItogeneratedocumentationforyourproject📚:$techdocsgenerate-k<api_key>-u<username>-p<password>-d<directory>PyPI package:https://pypi.org/project/techdocs/Web App:Note that the web app is only intended as a demo. To unleash the full power ofTechdocs, please refer to the CLI usage.We used Streamlit to create a user-friendly web app that allows users to upload their code block and get the documentation in a proper format. We have mentioned clear instructions on the instructions page. The home page of our app has all the details of our Techdocs projects along with its Architecture.Challenges we ran intoAPI Deployment Issues:Our very first API design consisted of keeping the entireWizardLM-70Bmodel locally on Huggingface's servers. However, the size of the model exceeded the free tier available there. Furthermore, inferencing from a model available on a local disk was extremely time-consuming.  To get docstrings of a single function it would take a few minutes.To tackle these mentioned issues, we shifted to Clarifai's API for our LLM. As the model is constantly kept on a dedicated server, the inferencing time wasreduced by 90%from the original approach. Our inference time wasreduced from a few minutes to under 10 seconds.This step, however, added an external dependency for our API deployment. Before going for Hugginface Spaces, the original plan was to deploy the API on Vercel Cloud. However, we were not able to deploy it on Vercel due to this newly added dependency - the 'Clarifai' package, as it exceeded the free tier resources that Vercel grants.To solve this issue, we decided to sync our GitHub repo to Huggingface Spaces via a.ymlworkflow and then deploy the API there by using ourDOCKERFILE. Without the docker support from Huggingface Spaces, we could not have deployed our API anywhere! Docker was our saviour.CLI development:Unlike the infamousclickpackage for building CLIs, the package we usedargparse, does not provide a built-in way to create subcommands. Adding subcommands by adding sub-parsers and binding appropriate function definitions was a nightmare. But we aced it eventually!Accomplishments that we're proud ofFirstly, all the documentation you see in this project is created by usingTechdocs. No code documentation is done manually :)We have developed a PyPI package so contributors all around the world can contribute and make this product better. We strongly believe, in the power of open-source projects!What we learnedThe use of Langchain was crucial here. This open-source tool has so much potential and is very handy in developing LLM products. It is cool!DOCKER images are way better than using other deployment platforms.This was the very first project where we actually used docker to deploy our product.Designing UI is a heck of a task.What's next for TechdocsCurrently, Techdocs can only generate documentation for Python programming language. Our plan is to expand our product to a spectrum of popular programming languages to help developers across the globe with documentation tasks.We plan to hire experts from different programming languages so that we can extend the support and build native packages for those languages. Being final-year undergrad students, we are looking for decent funding so we can achieve this task.As of now, our product suffers majorly in terms of privacy and integrity. As we have mentioned earlier, we are using a third-partyClarifaipackage for hosting our LLM. This creates a big question about user code integrity and privacy.To cater to this issue, we are constantly exploring new quantized and light versions of possible LLMs that we can employ for this task. This will eliminate the need for third-party APIs.We plan to integrate such lightweight models into our PyPI package itself so that none of the user's code will be sent anywhere, and all the documentation generation will take place on the user's device itself.Techstack 🧑‍💻