Call for collaborators: scaling RSIL from 50M to 1B+ parameters
Call for High-Compute Research Collaborators
We are seeking collaborators with access to substantial GPU
compute to independently reproduce and scale Resonant Spectral
Interaction Ladders (RSIL).
RSIL is an experimental sequence-model architecture investigating
translation-equivariant higher-order resonant interactions in the
spectral domain.
The repository contains a reproducible benchmark comparing:
- standard Transformer attention
- Fourier token mixing
- RSIL degree 1
- RSIL degree 2
- RSIL degree 3
- RSIL non-KAN controls
Research Question
The central question is not simply whether a larger RSIL model can
be trained.
We want to determine whether increasing resonant spectral interaction
degree provides reproducible improvements under controlled parameter,
data, and compute budgets.
We are particularly interested in determining whether effects observed
in the current small-scale experiments survive:
- independent random seeds,
- larger datasets,
- larger models,
- longer contexts,
- matched wall-clock budgets,
- optimized spectral kernels.
Target Scales
We propose the following progression:
10M
β 50M
β 100M
β 300M
β 1B+ parameters.
Large experiments should only proceed when results at the preceding
scale justify the additional compute.
Compute Collaborators
We are particularly interested in researchers with access to:
- NVIDIA A100
- NVIDIA H100/H200
- NVIDIA B200/GB200
- AMD MI300X
- DGX systems
- multi-node GPU clusters
- equivalent academic or commercial compute infrastructure
Priority Contributions
Useful contributions include:
- independent reproduction
- multi-seed training
- 50Mβ1B+ parameter scaling
- large-corpus pretraining
- FSDP and DeepSpeed scaling
- optimized FFT kernels
- optimized resonant-interaction kernels
- Triton/CUDA implementations
- long-context experiments
- wall-clock and memory profiling
- parameter/FLOP matching
- non-KAN controls
- alternative spectral parameterizations
Experimental Controls
Scaling experiments should compare, wherever practical:
Transformer
vs.
Fourier Mixer
vs.
RSIL-D1
vs.
RSIL-D2
vs.
RSIL-D3
vs.
RSIL non-KAN.
Experiments should report:
- parameter count
- training tokens
- optimizer and learning-rate schedule
- training loss
- held-out validation loss
- perplexity
- GPU type and count
- peak GPU memory
- tokens/second
- wall-clock training time
- estimated FLOPs
- random seed
Negative Results
Negative and null results are explicitly valuable.
If higher interaction degree fails to improve validation performance,
fails to scale efficiently, or is dominated by simpler baselines,
those results should be reported.
The purpose of this collaboration is to test the hypothesis, not to
preserve a predetermined conclusion.
How to Collaborate
Please open a Hugging Face Discussion describing:
- your available hardware,
- approximate GPU-hours available,
- model scale you would like to test,
- dataset/token budget,
- proposed experiment,
- whether you can contribute optimized kernels or distributed
training expertise.
Small experimental changes can be submitted through Hugging Face
Pull Requests.
Researchers making substantial experimental, theoretical, dataset,
or engineering contributions will be credited in the repository and
associated research outputs according to their contribution.