slicexai/elm-v0.1_news_content_generation

SliceX AI™ ELM (Efficient Language Models)

ELM (which stands for Efficient Language Models) is the first version in the series of cutting-edge language models from SliceX AI that is designed to achieve the best in class performance in terms of quality, throughput & memory.

ELM is designed to be a modular and customizable family of neural networks that are highly efficient and performant. Today we are sharing the first version in this series: ELM-v0.1 models (named Rambutan).

Model: ELM introduces a new type of (de)-composable LLM model architecture along with the algorithmic optimizations required to learn (training) and run (inference) these models. At a high level, we train a single ELM model in a self-supervised manner (during pre-training phase) but once trained the ELM model can be sliced in many ways to fit different user/task needs. The optimizations can be applied to the model either during the pre-training and/or fine-tuning stage.

Fast Inference with Customization: Once trained, the ELM model architecture permits flexible inference strategies at runtime depending on the deployment needs. For instance, the ELM model can be decomposed into smaller slices, i.e., smaller (or larger) models can be extracted from the original model to create multiple inference endpoints. Alternatively, the original (single) ELM model can be loaded as is for inference and different slices within the model can be queried directly to power faster inference. This provides an additional level of flexibility for users to make compute/memory tradeoffs depending on their application and runtime needs.

• Blog: Medium

• Github: https://github.com/slicex-ai/elm

• Demo (try it out): https://huggingface.co/spaces/slicexai/elm-demo-v1

• HuggingFace (access ELM Model cards, code & app from HF): https://huggingface.co/slicexai

ELM-v0.1 Model Release

This repository contains code to run our ELM models. The current ELM model elm-v0.1 (named Rambutan) was pre-trained (an intermediate checkpoint was used) and then instruction fine-tuned for downstream tasks.

ELM models (in the models folder) in this repository come in three sizes (elm-1.0, elm-0.75 and elm-0.25). All these different slices are extracted from the same ELM finetuned checkpoint for inference and supports the following use-case.

• news_content_generation (tldr_news dataset)

NOTE: ELM-v0.1 release is an early version finetuned from an intermediate pretrained checkpoint & without any KV caching, decoding optimizations, or quantization applied.

Setup ELM

Download ELM repo

sudo apt-get install git-lfs 
git lfs install
git clone https://huggingface.co/slicexai/elm-v0.1_news_content_generation

For Macbook, replace sudo apt-get install git-lfs with brew install git-lfs

Installation

cd elm-v0.1_news_content_generation
pip install -r requirements.txt

(Optional) Installing git-lfs without sudo,

wget https://github.com/git-lfs/git-lfs/releases/download/v3.2.0/git-lfs-linux-amd64-v3.2.0.tar.gz
tar -xzf git-lfs-linux-amd64-v3.2.0.tar.gz
PATH=$PATH:/<absolute-path>/git-lfs-3.2.0/
git lfs install

How to use: Run ELM on a sample task

python run.py <elm-model-directory>
- python run.py elm-1.0_news_content_generation
- python run.py elm-0.75_news_content_generation
- python run.py elm-0.25_news_content_generation

Prompts for the specific tasks can be found in the corresponding checkpoint directory. See an example below from models/elm-0.75_news_content_generation/example_prompts.json.

{
    "inputs": ["Scientists Invent 'Invisible' Metamaterial With Bonus Reflect Mode"],
    "template": "[INST]The following headline is the headline of a news report. Please write the content of the news passage based on only this headline.\n\nHeadline: {input} \n\nContent:[/INST]"
}

Running the above command returns the following response

{
    "prompt": "[INST]The following headline is the headline of a news report. Please write the content of the news passage based on only this headline.\n\nHeadline: Scientists Invent 'Invisible' Metamaterial With Bonus Reflect Mode \n\nContent:[/INST]",
    "response": "A team of scientists have created an invisible material that can make objects disappear. It is made of a special material that creates a layer of nanoscale dots that allow light to enter from the material, directing it to a layer of gas that allows light to enter from the material. The material is able to levitate and roll off its surface without leaving the material. This technology could have many future applications in battery technology, microelectronics, and more. A video demonstrating the material is available in the article."
}