nicolay-r (Nicolay Rusnachenko)

reacted to m-ric's post with 🔥 1 day ago

Post

1194

> Oasis: First Real-Time Video Game Without a Game Engine! 🎮

DecartAI & Etched just released Oasis - a fully AI-generated video game running at 20 FPS (frames per second). The model takes keyboard inputs and generates everything - physics, rules, graphics - on the fly, without any game engine.

⚡️ What makes this special? Current text-to-video models (Mochi-1, Sora, Kling) generate about 1 frame every 10-20 seconds (that's the kind of device I had to play LoL back in the day, thus my low rankings). Oasis is 200 times faster, making it the first playable AI-generated game.

⚙️ Under the hood, it uses a vision transformer to encode space and a diffusion model to generate frames. The secret sauce is "dynamic noising" - a technique that keeps the video stable between frames.

Key insights:
⚡️ Generates 20 FPS, vs 0.2 FPS for other DIT-based video models
‣ The specialized hardware Sohu developed by Etched allows to handle 10x more player than H100

🎮 Features real game mechanics
‣ Movement, jumping, item management
‣ Physics and lighting
‣ Procedurally generated worlds

⚠️ Current limitations
‣ Blurry graphics at a distance
‣ Objects sometimes change appearance
‣ Memory issues in long sessions

Try it yourself, the playable demo is impressive! 👉 https://oasis.decart.ai/welcome
Code 👉 https://github.com/etched-ai/open-oasis
Read it in full 👉 https://oasis-model.github.io/

posted an update 1 day ago

Post

820

📢 If you're aimed at processing complex dependencies in spreadsheet data with LLM Chain-of-Thought technique, then this update might be valuable for you 💎

The updated 📦 bulk-chain-0.24.1 which is aimed at iterative processing of CSV/JSONL data with no-string dependencies from third party LLM frameworks is out 🎉

📦: https://pypi.org/project/bulk-chain/0.24.1/
🌟: https://github.com/nicolay-r/bulk-chain
📘: https://github.com/nicolay-r/bulk-chain/issues/26

The key feature of bulk-chain is SQLite caching that saves your time ⏰️ and money 💵 by guarantee no-data-lost, which is important once using the remote LLM providers such as OpenAI, ReplicateIO, OpenRouter, etc.

🔧 This release has the following updates:
✅ Improved stability for various header conditions and the related support from SQLite
✅ Manual setup for ID column / assigning the ID
✅ Make CSV-related setups dynamic, that refers to the related Python 📦 csv package.

Quick start on GoogleColab:
📙: https://colab.research.google.com/github/nicolay-r/bulk-chain/blob/master/bulk_chain_tutorial.ipynb

Below is an example of the three simple steps in pictures:
1. ⬇️ Package installation
2. ✍️ Declaring schema
3. 🚀 Launching inference for your data with Replicate and 🤖 meta-llama/Llama-3.1-405B

posted an update 16 days ago

Post

653

📢 Excited to share that our studies 📄 "Large Language Models in Targeted Sentiment Analysis for Russian" has recently become in 📘 Springer Lobachevskii Journal of Mathematics 🥳✨ ...

📘 https://link.springer.com/article/10.1134/S1995080224603758

In this studies we provide such a diverse look and experiments over various 🤖 LLM models 🤖 scaled from 7B in two different modes: ❄️ zero-shot and 🔥 fine-tuned (Flan-T5 only) using Three-Hop reasoning technique.
We showcase the importance of performing:
💚 text translation into English
💚 application on Chain-of-Thought for Implicit Sentiment Analysis

More:
📄 Arxiv: https://arxiv.org/abs/2404.12342
🧑‍💻️ Code: https://github.com/nicolay-r/Reasoning-for-Sentiment-Analysis-Framework
🤗 Models: Large Language Models in Targeted Sentiment Analysis (2404.12342)
🎥 Video @NLPSummit : https://www.youtube.com/watch?v=qawLJsRHzB4

THOR: https://github.com/scofield7419/THOR-ISA

posted an update 19 days ago

Post

632

📢 We are giving extra two weeks before switching to the final stage of RuOpinionNE-2024.
⏰ The final stage starts since 1-st of November 2024.
We have already first baseline submission by 👨‍💻 RefalMachine that showcase F1 = 0.17 based on Qwen2 model series.

For those who wish to attend:
📊 Codalab: https://codalab.lisn.upsaclay.fr/competitions/20244
🗒 Task: https://codalab.lisn.upsaclay.fr/competitions/20244#learn_the_details-overview
🔔 Updates: https://t.me/RuOpinionNE2024

🙋 Questions: https://nicolay-r.github.io/
🧪 Past experiments: https://github.com/nicolay-r/RuSentNE-LLM-Benchmark

reacted to fdaudens's post with 🧠 25 days ago

Post

2999

The Nobel Prize background for Hopfield and Hinton's work on neural networks is pure gold. It's a masterclass in explaining AI basics.

Key takeaways from the conclusion:
- ML applications are expanding rapidly. We're still figuring out which will stick.
- Ethical discussions are crucial as the tech develops.
- Physics 🤝 AI: A two-way street of innovation.

Some mind-blowing AI applications in physics:
- Discovering the Higgs particle
- Cleaning up gravitational wave data
- Hunting exoplanets
- Predicting molecular structures
- Designing better solar cells

We're just scratching the surface. The interplay between AI and physics is reshaping both fields.

Bonus: The illustrations accompanying the background document are really neat. (Credit: Johan Jarnestad/The Royal Swedish Academy of Sciences)

#AI #MachineLearning #Physics #Ethics #Innovation

1 reply

·

reacted to fdaudens's post with 🔥 25 days ago

Post

2999

The Nobel Prize background for Hopfield and Hinton's work on neural networks is pure gold. It's a masterclass in explaining AI basics.

Key takeaways from the conclusion:
- ML applications are expanding rapidly. We're still figuring out which will stick.
- Ethical discussions are crucial as the tech develops.
- Physics 🤝 AI: A two-way street of innovation.

Some mind-blowing AI applications in physics:
- Discovering the Higgs particle
- Cleaning up gravitational wave data
- Hunting exoplanets
- Predicting molecular structures
- Designing better solar cells

We're just scratching the surface. The interplay between AI and physics is reshaping both fields.

Bonus: The illustrations accompanying the background document are really neat. (Credit: Johan Jarnestad/The Royal Swedish Academy of Sciences)

#AI #MachineLearning #Physics #Ethics #Innovation

1 reply

·

reacted to Jaward's post with 👀 25 days ago

Post

1109

Lightweight implementation of newly introduced “Differential Transformer”:
Proposes differential attention mechanism which computes attention scores as a difference between two separate softmax attention maps thereby reducing noise in attention blocks. [[[Differential nanoGPT]]] :)

Code: https://github.com/Jaykef/ai-algorithms/blob/main/DIFF_Transformer.ipynb
YT Video: https://youtu.be/9V4mJA5y7dg

reacted to fantaxy's post with 😎 25 days ago

Post

3274

NSFW Erotic Novel AI Generation
-NSFW Text (Data) Generator for Detecting 'NSFW' Text: Multilingual Experience

The multilingual NSFW text (data) auto-generator is a tool designed to automatically generate and analyze adult content in various languages. This service uses AI-based text generation to produce various types of NSFW content, which can then be used as training data to build effective filtering models. It supports multiple languages, including English, and allows users to input the desired language through the system prompt in the on-screen options to generate content in the specified language. Users can create datasets from the generated data, train machine learning models, and improve the accuracy of text analysis systems. Furthermore, content generation can be customized according to user specifications, allowing for the creation of tailored data. This maximizes the performance of NSFW text detection models.

Web: https://fantaxy-erotica.hf.space
API: https://replicate.com/aitechtree/nsfw-novel-generation

Usage Warnings and Notices: This tool is intended for research and development purposes only, and the generated NSFW content must adhere to appropriate legal and ethical guidelines. Proper monitoring is required to prevent the misuse of inappropriate content, and legal responsibility lies with the user. Users must comply with local laws and regulations when using the data, and the service provider is not liable for any issues arising from the misuse of the data.

2 replies

·

reacted to reach-vb's post with 👍 25 days ago

Post

2011

On-device AI framework ecosystem is blooming these days:

1. llama.cpp - All things Whisper, LLMs & VLMs - run across Metal, CUDA and other backends (AMD/ NPU etc)
https://github.com/ggerganov/llama.cpp

2. MLC - Deploy LLMs across platforms especially WebGPU (fastest WebGPU LLM implementation out there)
https://github.com/mlc-ai/web-llm

3. MLX - Arguably the fastest general purpose framework (Mac only) - Supports all major Image Generation (Flux, SDXL, etc), Transcription (Whisper), LLMs
https://github.com/ml-explore/mlx-examples

4. Candle - Cross-platform general purpose framework written in Rust - wide coverage across model categories
https://github.com/huggingface/candle

Honorable mentions:

1. Transformers.js - Javascript (WebGPU) implementation built on top of ONNXruntimeweb
https://github.com/xenova/transformers.js

2. Mistral rs - Rust implementation for LLMs & VLMs, built on top of Candle
https://github.com/EricLBuehler/mistral.rs

3. Ratchet - Cross platform, rust based WebGPU framework built for battle-tested deployments
https://github.com/huggingface/ratchet

4. Zml - Cross platform, Zig based ML framework
https://github.com/zml/zml

Looking forward to how the ecosystem would look 1 year from now - Quite bullish on the top 4 atm - but open source ecosystem changes quite a bit! 🤗

Also, which frameworks did I miss?

1 reply

·

posted an update 25 days ago

Post

996

📢 Two weeks ago I got a chance to share the most recent reasoning 🧠 capabilities of Large Language models in Sentiment Analysis NLPSummit-2024.

For those who missed and still wish to find out the advances of GenAI in that field, the recording is now available:
https://www.youtube.com/watch?v=qawLJsRHzB4

You will be aware of:
☑️ how well LLMs reasoning can be used for reasoning in sentiment analysis as in Zero-shot-Learning,
☑️ how to improve reasoning by applying and leaving step-by-step chains (Chain-of-Thought)
☑️ how to prepare the most advanced model in sentiment analysis using Chain-of-Thought.

Links:
📜 Paper: Large Language Models in Targeted Sentiment Analysis (2404.12342)
⭐ Code: https://github.com/nicolay-r/Reasoning-for-Sentiment-Analysis-Framework

reacted to qnguyen3's post with 🔥 26 days ago

Post

3605

nanoLLaVA-1.5 is here! Same size (1B), better performance 🔥🔥🔥
And it is much more powerful than v1.0
Try it out now on HF Spaces: qnguyen3/nanoLLaVA
Model: qnguyen3/nanoLLaVA-1.5

3 replies

·

reacted to mmhamdy's post with 👀 26 days ago

Post

1811

🔗 Evaluating Long Context #1: Long Range Arena (LRA)

Accurately evaluating how well language models handle long contexts is crucial, but it's also quite challenging to do well. In this series of posts, we're going to examine the various benchmarks that were proposed to assess long context understanding, starting with Long Range Arens (LRA)

Introduced in 2020, Long Range Arens (LRA) is one of the earliest benchmarks designed to tackle the challenge of long context evaluation.

📌 Key Features of LRA

1️⃣ Diverse Tasks: The LRA benchmark consists of a suite of tasks designed to evaluate model performance on long sequences ranging from 1,000 to 16,000 tokens. These tasks encompass different data types and modalities: Text, Natural and Synthetic Images, and Mathematical Expressions.

2️⃣ Synthetic and Real-world Tasks: LRA is comprised of both synthetic probing tasks and real-world tasks.

3️⃣ Open-Source and Extensible: Implemented in Python using Jax and Flax, the LRA benchmark code is publicly available, making it easy to extend.

📌 Tasks

1️⃣ Long ListOps

2️⃣ Byte-level Text Classification and Document Retrieval

3️⃣ Image Classification

4️⃣ Pathfinder and Pathfinder-X (Long-range spatial dependency)

👨‍💻 Long Range Arena (LRA) Github Repository: https://github.com/google-research/long-range-arena

📄 Long Range Arena (LRA) paper: Long Range Arena: A Benchmark for Efficient Transformers (2011.04006)

reacted to merve's post with 🔥 26 days ago

Post

3680

Meta AI vision has been cooking @facebook
They shipped multiple models and demos for their papers at @ECCV 🤗

Here's a compilation of my top picks:
- Sapiens is family of foundation models for human-centric depth estimation, segmentation and more, all models have open weights and demos 👏

All models have their demos and even torchscript checkpoints!
A collection of models and demos: facebook/sapiens-66d22047daa6402d565cb2fc
- VFusion3D is state-of-the-art consistent 3D generation model from images

Model: facebook/vfusion3d
Demo: facebook/VFusion3D

- CoTracker is the state-of-the-art point (pixel) tracking model

Demo: facebook/cotracker
Model: facebook/cotracker

posted an update 28 days ago

Post

638

📢 This year I made decent amout of experiments on LLM reasoning capabilities in author opinion extraction.
However, they did not go further with:
↗️ annoation of other sources of opinion causes: entities, out-of-context object (None).
📏 evaluation of factual statements that support the extracted sentiment.

To address these limitations, so far we launch 🚀 RuOpinionNE-2024 competition on the Codalab platform:
📊 https://codalab.lisn.upsaclay.fr/competitions/20244

The competition is aimed at extraction of opinion tuples (see attached images) from texts written in Russian.
It proceeds the past RuSentNE-2023 codalab competition findings:
🔎 Past year competition: https://www.dialog-21.ru/media/5896/golubevaplusetal118.pdf
🔎 LLM reasoning 🧠: https://arxiv.org/abs/2404.12342

For those who interested to adopt Generative AI, the complete information about competition is below:
📊 RuOpinionNE-2024: https://codalab.lisn.upsaclay.fr/competitions/20244
🗒 Task description: https://codalab.lisn.upsaclay.fr/competitions/20244#learn_the_details-overview
🔔 To follow updates: https://t.me/RuOpinionNE2024
⏰ Stages Deadlines (might be extended)
📦 Submission details (bottom of the competition page)

🙋 For questions you can contact @nicolay-r : https://nicolay-r.github.io/
🧪 Most recent findings on LLM application: https://github.com/nicolay-r/RuSentNE-LLM-Benchmark

reacted to Tonic's post with 👀 28 days ago

Post

2713

🙋🏻‍♂️hey there folks ,

did you know that https://huggingface.co/lmms-lab released a new version of 🌋🌋Llava on thursday ? Now it has 🎥video understanding !
check it out 👇🏻

collection : lmms-lab/llava-video-661e86f5e8dabc3ff793c944
demo : Tonic/Llava-Video

reacted to datatab's post with 👀 28 days ago

Post

2430

Explore the Serbian LLM Evaluation Dataset 🚀

datatab/serbian-llm-benchmark

Excited to share the launch of the Serbian LLM Evaluation Dataset—a specialized resource for rigorously testing Serbian Language Models. Perfect for developers, researchers, and tech enthusiasts eager to push the boundaries of AI in language understanding.

Features Include:
Extensive question sets across multiple domains such as science, history, and more.
Integration with Lighteval for streamlined, efficient evaluation.
Diverse challenge levels to thoroughly assess model performance in Serbian.
Start enhancing your language models today! Dive into a universe of data crafted to refine and perfect AI interactions in Serbian.

2 replies

·

reacted to nroggendorff's post with 👀 28 days ago

Post

1511

pretty much all of the values in the llama training post are placeholders, so if you dont get a desireable result tweak and tweak and tweak. it took months to get smallama to do anything

reacted to clem's post with 👀 28 days ago

Post

2026

What are we thinking about MovieGen from Meta? Are the researchers on Hugging Face to be able to ask them questions?

The paper is here: https://ai.meta.com/static-resource/movie-gen-research-paper

posted an update about 1 month ago

Post

735

📢 The fast application of named entity recognition (NER) model towards vast amout of texts usually serves two major pitfalls:
🔴 Limitation of the input window size
🔴 Drastically slows down the downstream pipeline of the whole application

⭐ https://github.com/nicolay-r/bulk-ner

To address these problems, bulk-ner represent a no-string framework with the handy wrapping over any dynamically linked NER-ml model by providing:
☑️ Native long-input contexts handling.
☑️ Native support of batching (assuming that ML-model engine has the related support too)

To quick start, sharing the wrapper over DeepPavlov NER models.
With the application of such models you can play and bulk your data here:
📙 https://colab.research.google.com/github/nicolay-r/ner-service/blob/main/NER_annotation_service.ipynb
(You have to have your data in CSV / JSONL format)

Lastly, it is powered by AREkit pipelines, and therefore could be a part of the relation extraction and complex information retrieval systems:
💻 https://github.com/nicolay-r/AREkit
📄 https://openreview.net/forum?id=nRybAsJMUt

reacted to merve's post with 👍 about 1 month ago

Post

3941

If you feel like you missed out for ECCV 2024, there's an app to browse the papers, rank for popularity, filter for open models, datasets and demos 📝

Get started at ECCV/ECCV2024-papers ✨

Nicolay Rusnachenko

AI & ML interests

Organizations

nicolay-r's activity