slicexai/elm-v0.1_news_summarization

SliceX AI™ ELM (Efficient Language Models)

ELM (which stands for Efficient Language Models) is the first version in the series of cutting-edge language models from SliceX AI that is designed to achieve the best in class performance in terms of quality, throughput & memory.

ELM is designed to be a modular and customizable family of neural networks that are highly efficient and performant. Today we are sharing the first version in this series: ELM-v0.1 models (named Rambutan).

Model: ELM introduces a new type of (de)-composable LLM model architecture along with the algorithmic optimizations required to learn (training) and run (inference) these models. At a high level, we train a single ELM model in a self-supervised manner (during pre-training phase) but once trained the ELM model can be sliced in many ways to fit different user/task needs. The optimizations can be applied to the model either during the pre-training and/or fine-tuning stage.

Fast Inference with Customization: Once trained, the ELM model architecture permits flexible inference strategies at runtime depending on the deployment needs. For instance, the ELM model can be decomposed into smaller slices, i.e., smaller (or larger) models can be extracted from the original model to create multiple inference endpoints. Alternatively, the original (single) ELM model can be loaded as is for inference and different slices within the model can be queried directly to power faster inference. This provides an additional level of flexibility for users to make compute/memory tradeoffs depending on their application and runtime needs.

• Blog: Medium

• Github: https://github.com/slicex-ai/elm

• Demo (try it out): https://huggingface.co/spaces/slicexai/elm-demo-v1

• HuggingFace (access ELM Model cards, code & app from HF): https://huggingface.co/slicexai

ELM-v0.1 Model Release

This repository contains code to run our ELM models. The current ELM model elm-v0.1 (named Rambutan) was pre-trained (an intermediate checkpoint was used) and then instruction fine-tuned for downstream tasks.

ELM models (in the models folder) in this repository come in three sizes (elm-1.0, elm-0.75 and elm-0.25). All these different slices are extracted from the same ELM finetuned checkpoint for inference and supports the following use-case.

• news_summarization (cnn dailymail dataset)

NOTE: ELM-v0.1 release is an early version finetuned from an intermediate pretrained checkpoint & without any KV caching, decoding optimizations, or quantization applied.

Setup ELM

Download ELM repo

sudo apt-get install git-lfs 
git lfs install
git clone https://huggingface.co/slicexai/elm-v0.1_news_summarization

For Macbook, replace sudo apt-get install git-lfs with brew install git-lfs

Installation

cd elm-v0.1_news_summarization
pip install -r requirements.txt

(Optional) Installing git-lfs without sudo,

wget https://github.com/git-lfs/git-lfs/releases/download/v3.2.0/git-lfs-linux-amd64-v3.2.0.tar.gz
tar -xzf git-lfs-linux-amd64-v3.2.0.tar.gz
PATH=$PATH:/<absolute-path>/git-lfs-3.2.0/
git lfs install

How to use: Run ELM on a sample task

python run.py <elm-model-directory>
- python run.py elm-1.0_news_summarization
- python run.py elm-0.75_news_summarization

Prompts for the specific tasks can be found in the corresponding checkpoint directory. See an example below from models/elm-0.75_news_summarization/example_prompts.json.

{
    "inputs": ["He is a World Cup winner, Spanish football legend, and one of the most recognisable faces in the classy Barcelona CF outfit. And now tourists are being offered the chance to walk in the footsteps of Andr\u00e9s Iniesta after he listed his beautiful Spanish vineyard on Airbnb. The world class midfielder took to Twitter to advise that his the 'Bodega Iniesta' vineyard he owns in Castilla-La Mancha can be rented out. Spain and Barcelona midfielder\u00a0Andr\u00e9s Iniesta is renting out his vineyard on Airbnb . Andr\u00e9s Iniesta posted on Twitter to spread the news that his charming vineyard can be rented out . And it's a snip of a price to tread paths made by one of the beautiful game's best players at \u20ac125 (\u00a390) a night. There is one bathroom and one bedroom at the charming little property, with other facilities including a kitchen, an indoor fireplace, Wi-Fi and parking provided. The residence is aimed for couples as the bedroom consists of one double bed. Decorated with a warm touch, guests can enjoy the fireplace with a glass of wine, relax on the couch or stroll among the vines. The vineyard stay comes with a guided tour of the area so you can get a real feel for the place . The interior of the property is simple yet subtle, ensuring the guests has all the amenities to get by . The house kitchen is fully equipped for people staying to use and enjoy. Breakfast food is provided for the duration of the stay, as well as towels and an extra set of sheets. Guests will also be advised of the surrounding area so they can explore for themselves. Also offered is a tour of the vineyard where guests can discover the secrets of wine-making. 'Airbnb gives you access to the most special places in the world',  Jeroen Merchiers, Regional Manager North, East and South of Europe told MailOnline Travel. The highlight of a stay at\u00a0Andr\u00e9s Iniesta's vineyard is undoubtedly what's outside rather than in . Guests can educate themselves in the art of wine-making, to hopefully produce their own special brand . 'Airbnb guests look for unique experiences. 'And we're pleased to announce Andr\u00e9s Iniesta is joining our community, unlocking a once in a lifetime experience for football and wine enthusiasts.' Some of the rules when staying in the property include being 'gentle with the vines,' smoking is prohibited inside, and the guests are responsible for goods during their stay. The property can be booked here. Iniesta has lit up the world of football for many years with his sublime skills, and now you can see a little more about his life outside the Beautiful Game . The 'Bodega Iniesta' vineyard he owns in Castilla-La Mancha can be rented out ."],
    "template": "[INST]You are given a news article below. Please summarize the article, including only its highlights.\n\n### Article: {input}\n\n### Summary:[/INST]"
}

Running the above command returns the following response

{
    "prompt": "[INST]You are given a news article below. Please summarize the article, including only its highlights.\n\n### Article: He is a World Cup winner, Spanish football legend, and one of the most recognisable faces in the classy Barcelona CF outfit. And now tourists are being offered the chance to walk in the footsteps of Andr\u00e9s Iniesta after he listed his beautiful Spanish vineyard on Airbnb. The world class midfielder took to Twitter to advise that his the 'Bodega Iniesta' vineyard he owns in Castilla-La Mancha can be rented out. Spain and Barcelona midfielder\u00a0Andr\u00e9s Iniesta is renting out his vineyard on Airbnb . Andr\u00e9s Iniesta posted on Twitter to spread the news that his charming vineyard can be rented out . And it's a snip of a price to tread paths made by one of the beautiful game's best players at \u20ac125 (\u00a390) a night. There is one bathroom and one bedroom at the charming little property, with other facilities including a kitchen, an indoor fireplace, Wi-Fi and parking provided. The residence is aimed for couples as the bedroom consists of one double bed. Decorated with a warm touch, guests can enjoy the fireplace with a glass of wine, relax on the couch or stroll among the vines. The vineyard stay comes with a guided tour of the area so you can get a real feel for the place . The interior of the property is simple yet subtle, ensuring the guests has all the amenities to get by . The house kitchen is fully equipped for people staying to use and enjoy. Breakfast food is provided for the duration of the stay, as well as towels and an extra set of sheets. Guests will also be advised of the surrounding area so they can explore for themselves. Also offered is a tour of the vineyard where guests can discover the secrets of wine-making. 'Airbnb gives you access to the most special places in the world',  Jeroen Merchiers, Regional Manager North, East and South of Europe told MailOnline Travel. The highlight of a stay at\u00a0Andr\u00e9s Iniesta's vineyard is undoubtedly what's outside rather than in . Guests can educate themselves in the art of wine-making, to hopefully produce their own special brand . 'Airbnb guests look for unique experiences. 'And we're pleased to announce Andr\u00e9s Iniesta is joining our community, unlocking a once in a lifetime experience for football and wine enthusiasts.' Some of the rules when staying in the property include being 'gentle with the vines,' smoking is prohibited inside, and the guests are responsible for goods during their stay. The property can be booked here. Iniesta has lit up the world of football for many years with his sublime skills, and now you can see a little more about his life outside the Beautiful Game . The 'Bodega Iniesta' vineyard he owns in Castilla-La Mancha can be rented out .\n\n### Summary:[/INST]",
    "response": "Andr\u00e9s Iniesta is renting out his vineyard on Airbnb and is looking to become a leading wine collector. The property is located in Castilla-La Mancha, a region of Spain. The property is designed to be a one-bedroom residence with a kitchen, an indoor fireplace, Wi-Fi and parking provided. The property is aimed for couples as the bedroom consists of one double bed. Andr\u00e9s Iniesta is currently renting out his vineyard on Airbnb. The property is available for those who want to get a taste of the region's wines. Andr\u00e9s Iniesta is hoping to become a leading wine collector."
}