glenn2/LFG-1

Model Details

• Model Name: LFG-1 (Listening Fusion Gemma)
• Model Type: Multimodal Conversational Audio-Language Model
• Background: LFG-1 was built as a personal learning project, trained entirely on Apple Silicon at my house on nights and weekends. It isn't an official model.
• Architecture: LFG-1 bridges the Gemma 4 E2B audio encoder with the Gemma 4 26B-A4B text model. The connection is made via a custom-trained projection layer, allowing the language model to natively ingest and understand raw acoustic features without relying on an intermediate text transcript.
• Framework: MLX (mlx-vlm)
• License: Apache 2.0

Intended Use

• Primary Use Case: Real-time conversational audio applications running locally on Mac hardware.
• Capabilities: Native speech-to-response generation, streaming text output, and multimodal support (simultaneous audio and image input). Because the model processes raw acoustic data, it captures conversational pacing, pauses, and tone that traditional Speech-to-Text (STT) pipelines strip away.
• Untouched Text Reasoning: LFG-1 keeps Gemma 4’s text backbone frozen during audio-projection training, so its core language and reasoning capabilities are expected to remain essentially unchanged while gaining a trained audio input pathway.

Language Support

• Current Status: The audio projection layer is currently trained exclusively on English.
• Future Roadmap: This is an active learning project (hence the "1" in LFG-1). Future iterations will focus on expanding the audio training data. Requests, suggestions, and dataset recommendations for additional languages are highly welcome in the repository Issues!

Hardware Requirements

Due to the size of the combined weights (~48 GB), LFG-1 requires substantial unified memory.

• Minimum Requirement: Apple Silicon with at least 64 GB of Unified Memory.
• Storage: ~50 GB of disk space for local weights.

Install

Install mlx-vlm, then clone and install the LFG wrapper code:

pip install mlx-vlm
git clone https://github.com/codemadeio/LFG-1.git
cd LFG-1
pip install -e .

Usage

LFG-1 is designed as a drop-in model type for mlx-vlm. Once the lfg module is registered, it can be loaded like any other Hugging Face model.

from lfg import register
register()

from mlx_vlm import load, generate

model, processor = load("glenn2/LFG-1")