Beckett Dillon PRO

Severian

AI & ML interests

I make music, teach machines, study nature and build things (www.beckettdillon.com)

Organizations

Severian's activity

posted an update 4 days ago
view post
Post
2408
GraphRAG-Ollama-UI

I've been working on a local version of Microsoft's GraphRAG that uses Ollama for everything. It's got a new interactive UI built with Gradio that makes it easier to manage data, run queries, and visualize results. It's not fully featured or set up to harness the entire GraphRAG library yet but it allows you to run all the standard commands for Indexing/Processing and chatting with your graph. Some key features:

Uses local models via Ollama for LLM and embeddings

3D graph visualization of the knowledge graph using Plotly

File management through the UI (upload, view, edit, delete)

Settings management in the interface

Real-time logging for debugging

https://github.com/severian42/GraphRAG-Ollama-UI
  • 1 reply
ยท
posted an update 24 days ago
view post
Post
1020
Mixture of Agents now in MLC/LMStudio/Ollama

I've been a bit obsessed with the recent MoA paper and its implementation. I've noticed a HUGE upgrade in the final output and it seems to really be a great way to harness the power of a team of different LLMs. The downside is that it can be a bit slow to generate responses with the bigger models (but worth it if you want to wait). I wanted to get faster results so I made an MLC version and it actually works out great! Much quicker and the responses definitely are better than compared to just running one.

I'm going to keep working on seeing how it can be further integrated (API endpoints, RAG, synthetic data generation, etc) and will share the stuff that I can get to work decently enough :)

https://github.com/severian42/MoA-MLC-Chat

https://github.com/severian42/MoA-Ollama-Chat

https://github.com/severian42/MoA-LMStudio-Chat
posted an update about 2 months ago
view post
Post
1591
Jamba GGUF!

Finally, thanks to the awesome work of the brilliant mind of Github user compilade (https://github.com/compilade) Jamba is now beginning to be supported in llama.cpp (just CPU inference at the moment). So far there are a few different versions I have been able to convert, mainly the Jamba-Bagel, Jamba-Claude, 900M Jamba-Small and a 1B Jamba

Severian/jamba-gguf-665884eb2ceef24c1a0547e0
replied to davanstrien's post 2 months ago
posted an update 2 months ago
view post
Post
1486
Craft Your Own Expert LLM - Using 100% Open-Source/Private/Free/Awesome Tools

Hey everyone! After seeing a lot of people's interest in crafting their own datasets and then training their own models, I took it upon myself to try and build a stack to help ease that process. I'm excited to share a major project I've been developingโ€”the Vodalus Expert LLM Forge.

https://github.com/severian42/Vodalus-Expert-LLM-Forge

This is a 100% locally LLM-powered tool designed to facilitate high-quality dataset generation. It utilizes free open-source tools so you can keep everything private and within your control.

Why Open Source?

I decided to open source the Vodalus Expert LLM Forge to empower individuals and organizations everywhere to generate their own high-quality data. By making these tools freely available, I hope this community can start crafting their own models with little to no money and/or experience, helping to improve data quality and innovation across the board. While I'm releasing this tool for free, I've also completed an extensive tutorial/course with lots of videos and instructions that guide you through each step of maximizing the potential of this stack. This course is available for purchase at ko-fi.com/s/076479f834 and is designed to enhance your experience and results with the Vodalus Expert LLM Forge.

Whatโ€™s included in the Vodalus Expert LLM Forge?

- Data Generation: Harness RAG (through AnythingLLM if you are set up properly) and Wikipedia to create datasets via local language models.

- Model Training & Fine-Tuning: Tutorials and Jupyter notebooks to customize models to your specific needs.

- Quantization: Optimize models for performance with our quantization guides.

If this project aids your work, please consider supporting it through a donation at my ko-fi.com/severian42. Your support helps sustain my further LLM developments and experiments, always with a focus on using those efforts to give back to the LLM community.
posted an update 2 months ago
view post
Post
1614
Vodalus Expert LLM Forge - Dataset Crafting and Efficient Fine-Tuning Using Only Free Open-Source Tools

Hey everyone! After my last post getting a sense of people's interest in crafting their own datasets, I'm excited to share a major project I've been developingโ€”the Vodalus Expert LLM Forge.

https://github.com/severian42/Vodalus-Expert-LLM-Forge

This is a 100% locally LLM-powered tool designed to facilitate high-quality dataset generation. It utilizes free open-source tools so you can keep everything private and within your control. After considerable thought and debate (this project is the culmination of my few years of learning/experimenting), I've decided to open-source the entire stack. My hope is to elevate the standard of datasets and democratize access to advanced data-handling tools. There shouldn't be so much mystery to this part of the process.

Why Open Source?
My hope is to empower individuals everywhere to generate their own high-quality data. By making these tools freely available, I hope this community can start crafting their own models with little to no money and/or experience, helping to improve data quality and innovation across the board. While I'm releasing this tool for free, I'm also near completion on an extensive tutorial/course that guides you through each step of maximizing the potential of this stack. This course will be available for purchase soon and is designed to enhance your experience and results with the Vodalus Forge; more details soon

If this project aids your work, please consider supporting it through a donation on my https://ko-fi.com/N4N4XZ2TZ. Your support helps sustain my further LLM developments and experiments, always with a focus on using those efforts to give back to this community
posted an update 3 months ago
view post
Post
3516
Create and Train Your Own Expert LLM: Generating Synthetic, Fact-Based Datasets with LMStudio/Ollama and then fine-tuning with MLX and Unsloth

Hey everyone!

I know there are tons of videos and tutorials out there already but I've noticed a lot of questions popping up in community posts about using synthetic datasets for creative projects and how to transform personal content into more factual material. In my own work doing enterprise-level SFT and crafting my open-source models, I've enhanced a Python framework originally shared by the creator of the Tess models. This improved stack utilizes local language models and also integrates the Wikipedia dataset to ensure that the content generated is as accurate and reliable as possible.

I've been thinking of putting together a comprehensive, step-by-step course/guide on creating your own Expert Language Model. From dataset preparation and training to deployment on Hugging Face and even using something like AnythingLLM for user interaction. I'll walk you through each phase, clarifying complex concepts and troubleshooting common pitfalls.

Let me know if this interests you!

Most of the datasets and models I've made have been using these scripts and my approach
ยท
posted an update 3 months ago