Call for collaborators: scaling RSIL from 50M to 1B+ parameters

#2
by kiruluta - opened

Call for High-Compute Research Collaborators

We are seeking collaborators with access to substantial GPU
compute to independently reproduce and scale Resonant Spectral
Interaction Ladders (RSIL).

RSIL is an experimental sequence-model architecture investigating
translation-equivariant higher-order resonant interactions in the
spectral domain.

The repository contains a reproducible benchmark comparing:

  • standard Transformer attention
  • Fourier token mixing
  • RSIL degree 1
  • RSIL degree 2
  • RSIL degree 3
  • RSIL non-KAN controls

Research Question

The central question is not simply whether a larger RSIL model can
be trained.

We want to determine whether increasing resonant spectral interaction
degree provides reproducible improvements under controlled parameter,
data, and compute budgets.

We are particularly interested in determining whether effects observed
in the current small-scale experiments survive:

  1. independent random seeds,
  2. larger datasets,
  3. larger models,
  4. longer contexts,
  5. matched wall-clock budgets,
  6. optimized spectral kernels.

Target Scales

We propose the following progression:

10M
β†’ 50M
β†’ 100M
β†’ 300M
β†’ 1B+ parameters.

Large experiments should only proceed when results at the preceding
scale justify the additional compute.

Compute Collaborators

We are particularly interested in researchers with access to:

  • NVIDIA A100
  • NVIDIA H100/H200
  • NVIDIA B200/GB200
  • AMD MI300X
  • DGX systems
  • multi-node GPU clusters
  • equivalent academic or commercial compute infrastructure

Priority Contributions

Useful contributions include:

  • independent reproduction
  • multi-seed training
  • 50M–1B+ parameter scaling
  • large-corpus pretraining
  • FSDP and DeepSpeed scaling
  • optimized FFT kernels
  • optimized resonant-interaction kernels
  • Triton/CUDA implementations
  • long-context experiments
  • wall-clock and memory profiling
  • parameter/FLOP matching
  • non-KAN controls
  • alternative spectral parameterizations

Experimental Controls

Scaling experiments should compare, wherever practical:

Transformer
vs.
Fourier Mixer
vs.
RSIL-D1
vs.
RSIL-D2
vs.
RSIL-D3
vs.
RSIL non-KAN.

Experiments should report:

  • parameter count
  • training tokens
  • optimizer and learning-rate schedule
  • training loss
  • held-out validation loss
  • perplexity
  • GPU type and count
  • peak GPU memory
  • tokens/second
  • wall-clock training time
  • estimated FLOPs
  • random seed

Negative Results

Negative and null results are explicitly valuable.

If higher interaction degree fails to improve validation performance,
fails to scale efficiently, or is dominated by simpler baselines,
those results should be reported.

The purpose of this collaboration is to test the hypothesis, not to
preserve a predetermined conclusion.

How to Collaborate

Please open a Hugging Face Discussion describing:

  1. your available hardware,
  2. approximate GPU-hours available,
  3. model scale you would like to test,
  4. dataset/token budget,
  5. proposed experiment,
  6. whether you can contribute optimized kernels or distributed
    training expertise.

Small experimental changes can be submitted through Hugging Face
Pull Requests.

Researchers making substantial experimental, theoretical, dataset,
or engineering contributions will be credited in the repository and
associated research outputs according to their contribution.

Sign up or log in to comment