AUGMXNT

Enterprise

https://augmxnt.com/

augmxnt

AUGMXNT

Activity Feed Request to join this org

AI & ML interests

None defined yet.

Recent Activity

leonardlin updated a model 12 days ago

augmxnt/shisa-gamma-7b-v1

leonardlin updated a model 12 days ago

augmxnt/shisa-gamma-7b-v1

leonardlin updated a dataset 9 months ago

augmxnt/deccp

View all activity

augmxnt's activity

leonardlin

updated a model 12 days ago

augmxnt/shisa-gamma-7b-v1

Text Generation • Updated 12 days ago • 17k • 18

leonardlin

posted an update 9 months ago

Post

2042

My weekened project ended up being doing some testing between torchtune, axolotl, and unsloth. I *think* it's a 1:1 comparison of what LoRA fine-tuning performance looks like between the different hardware I have in my dev boxes (4090, 3090, 7900 XTX, W7900) with a few other interesting tidbits.

Tonight I wrote up a WandB report (the panel editor is super broken in Firefox 😔) that sums up some of the more interesting bits from the results: https://wandb.ai/augmxnt/train-bench/reports/torchtune-vs-axolotl-vs-unsloth-Trainer-Comparison--Vmlldzo4MzU3NTAx

1 reply

leonardlin

updated a dataset 9 months ago

augmxnt/deccp

Viewer • Updated Jun 13, 2024 • 132 • 178 • 18

leonardlin

updated a model 9 months ago

augmxnt/Qwen2-7B-Instruct-deccp

Text Generation • Updated Jun 11, 2024 • 21 • 20

leonardlin

posted an update 10 months ago

Post

2502

Maybe of interest, I just finished a long writeup of my weekend project exploring Qwen 2 7B Instruct's Chinese censorship: https://huggingface.co/blog/leonardlin/chinese-llm-censorship-analysis

I also have an accompanying model and dataset (and codebase) for those curious to poke around:

* augmxnt/Qwen2-7B-Instruct-deccp

* augmxnt/deccp

leonardlin

posted an update 10 months ago

Post

1942

Interesting, I've just seen the my first HF spam on one of my new model uploads: shisa-ai/shisa-v1-llama3-70b - someone has an SEO spam page as a HF space attached to the model!?! Wild. Who do I report this to?

4 replies

leonardlin

posted an update 10 months ago

Post

1613

For those with an interest in JA language models, this Llama 3 70B test ablation looks like it is the current strongest publicly released, commercially usable, open model available. A lot of caveats I know, but it also matches gpt-3.5-turbo-0125's JA performance, which is worth noting, and is tuned *exclusively* with the old shisa-v1 dataset (so it's chart position will be very short lived).

shisa-ai/shisa-v1-llama3-70b

augmxnt/ultra-orca-boros-en-ja-v1

2 replies

leonardlin

posted an update 10 months ago

Post

1948

With slurm figured out and ablations humming along, I though I'd update and post my understanding of the legal status of training data in Japan. It is in general, much clearer in the US: https://huggingface.co/blog/leonardlin/ai-training-data-in-japan

leonardlin

posted an update 10 months ago

Post

1372

llm-jp-eval is currently one of the most widely used benchmarks for Japanese LLMs and is half of WandB's comprehensive Nejumi LLM Leaderboard scoring. I was seeing some weirdness in results I was getting and ended up in a bit of a rabbit hole. Here's my article on evaling llm-jp-eval: https://huggingface.co/blog/leonardlin/llm-jp-eval-eval

I've setup a fork of Lightblue's Shaberi testing framework which uses LLM-as-a-Judge style benchmarks as something probably more representative of real world LLM strength in Japanese. Here's how the new base model ablations are looking:

leonardlin

posted an update 10 months ago

Post

1255

I've been doing some evals and tuning, and this chat template repo maintained by @chujiezheng is great: https://github.com/chujiezheng/chat_templates

Here's also a simple script for checking what the output looks like:

from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("augmxnt/shisa-7b-v1")
messages = [
    {'role': 'user', 'content': 'This is the first user input.'},
    {'role': 'assistant', 'content': 'This is the first assistant response.'},
    {'role': 'user', 'content': 'This is the second user input.'},
]

print()
print('Chat Template:')
print(tokenizer.chat_template)
print()
print('---')
print()

print(tokenizer.apply_chat_template(messages, tokenize=False))