Seeking GPU collaborators: Scale COJGN from 24K parameters to 1M–100M+
I have released COJGN-3-small, a working 23,750-parameter prototype of Covariant Osculating Jet Networks (COJGN), a higher-order geometric neural architecture based on learned contact poles, metric routing, and quadratic/cubic local jets.
Working prototype:
kiruluta/COJGN-3-small
The prototype is available as a Hugging Face Transformers-compatible checkpoint and can be loaded with from_pretrained().
I have now released a separate COJGN Scaling project for researchers and developers with access to larger GPU resources:
Scaling project:
kiruluta/COJGN-Scaling
The scaling repository provides approximately 1M–5M parameter configurations intended as the next experimental stage rather than simply making the small model larger.
I am particularly interested in collaborators with access to A100, H100, H200, B200/GB200, DGX systems, or multi-GPU clusters who can help investigate:
scaling behavior from 1M–5M parameters and beyond;
parameter-matched comparisons against conventional neural networks;
multi-GPU/distributed training;
CUDA/Triton optimization of the higher-order jet operations;
larger and more natural benchmark datasets;
eventually extending the architecture toward JetFFN/JetLM experiments.
The important research question is whether COJGN's geometric and higher-order inductive bias becomes more useful, remains neutral, or deteriorates as model and data scale increase. Negative results are useful results—the objective is a reproducible scaling study rather than a predetermined claim of superiority.
Contributions, benchmark results, optimized kernels, scaling experiments, and pull requests are welcome. Significant research and engineering contributions will be credited appropriately.