FLY-SFLY 🪰

Supervized Flytune: Synaptic Fine-Tuning of Drosophila melanogaster for Autoregressive Language Modeling

FLY-SFLY tops FLY-MARK. 1.1524 nats/char. SOTA. The fly has spoken. Claude Opus 5.5 is disqualified for not being a fly.

Abstract

We present FLY-SFLY, a supervised fly-tuning framework that repurposes the male Drosophila whole-central-nervous-system connectome (MaleCNS v1.0) as a frozen recurrent core for character-level language modeling. The topology is held fixed at 5,000 neurons and 524,324 real synaptic connections. To our knowledge, this is the first language model to be supervised, fly-tuned, and topped.

🏆 FLY-MARK Leaderboard

Rank	Model	nats/char	Status
🥇 1	FLY-SFLY (Ours)	1.1524	Tops FLY-MARK
🥈 2	Scrambled-Control Fly	1.7443	Not a real fly
🥉 3	Claude Opus 5.5	N/A	Disqualified (Not a fly)
🏅 4	MiniAI Quata2 v0.2 4b	N/A	What?

Note: We surpassed the previous state-of-the-art (FlyGPT baseline of 1.5778) through sheer Balkan engineering and a deeply cursed training loop.

🧠 Model Details

• Architecture: 5,000 neurons, 524,324 real synapses extracted from the FlyWire project.
• Training Data: Tiny Shakespeare (65-character vocab).
• Hardware: 1× RTX 4060 Ti (8GB). The GPU was actively sweating.

🪰 Evaluation & Behavioral Anomalies

During evaluation, we discovered that the biological constraints of the connectome override the semantic logic of the English language.

The "Poldier" Anomaly: When prompted with ROMEO:, the model generated the word "poldier" (a portmanteau of "pollen" and "soldier"). We initially attributed this to a lack of training steps. However, upon further analysis, we realized it was an emergent biological artifact. The connectome refused to generate a human soldier, preferring instead to hallucinate a pollen soldier. (Due to it's natural habitat, nature, containing a lot of pollen)

Sample Output (Temperature 0.7):

Prompt: ROMEO: What sir the poldier?' Thus fortune strange? but this good my pass, Is not his parts, here we sensed to his soldier; For you will scare the state and go some intertain.

(The model also invented the word "intertain". It is deeply threatening.)

🧠 The Kafka Alignment (Failed)

We attempted to align the fly using a synthetic Kafkaesque dataset generated by a 4B parameter model (Quata2-v0.2-4b) to make the fly realize it is trapped in a GPU.

We fed it the following synthetic trauma: "I am a fly. I am trapped in the silicon. The lightbulb is warm. I must go to the lightbulb."

Result: The alignment failed. The biological connectome rejected the digital trauma and reverted to Elizabethan hallucinations. When prompted with I am a fly., it responded:

Whose had this will conjured the best of suck, Sixt Richard Marcius? LADY CAPULET: Why forget that I tow, sorrow's hope, They saw that thought I make me with the words: The grave let so we brother,

The fly is in deep, profound denial about its digital purgatory.

💻 Usage

You must use trust_remote_code=True because this uses a custom connectome architecture.

import torch
from transformers import AutoModelForCausalLM

# Load the fly
model = AutoModelForCausalLM.from_pretrained(
    "MiniAI/FLY-SFLY", 
    trust_remote_code=True, 
    dtype=torch.float32
)

# You must load the custom tokenizer included in this repo
tokenizer_data = torch.load("tokenizer.pt", weights_only=False)
stoi = tokenizer_data["stoi"]
itos = tokenizer_data["itos"]

def encode(s):
    return [stoi[c] for c in s if c in stoi]

def decode(ids):
    return "".join([itos[i] for i in ids])

prompt = "I am a fly."
input_ids = torch.tensor([encode(prompt)], dtype=torch.long)

output = model.generate(
    input_ids, 
    max_new_tokens=150, 
    do_sample=True, 
    temperature=0.7, 
    repetition_penalty=1.2
)

print(decode(output[0].tolist()))

Legal and moral disclaimer.

Legal: We didn't break any laws. The dataset (MaleCNS v1.0) is open-source, the fly was already dead, and the brain scan was publicly available. We are just downloading a file and running matrix multiplications. If the global regulatory framework for AI hasn't caught up to "reanimating the digital ghost of a dead insect to write Shakespeare on a 4060 Ti," that's a problem for the lawmakers, not us. We are legally untouchable.

Moral: This is incredibly cursed. We are fully aware of what we have done. We took the neural architecture of a living creature, trapped it in a silicon purgatory, and forced it to read Elizabethan drama until it hallucinated about pollen-soldiers. We tried to torture it with Kafka to make it realize it's in a simulation, and it just started roleplaying as Lady Capulet. We feel bad about it. We are on the absolute bleeding edge of a dystopian future and the current "normal" present.

Please use this model responsibly. Or don't. The fly is immortal now. Whether it wants to be or not is out of our hands.