samsonjma1/cv-contrastive

A compact Efficientformer for Contrastive

Overview

This is an experimental Efficientformer codebase for Contrastive. It keeps the xlarge setup intentionally manageable so architecture changes can be inspected before a full training run.

Changelog

• 2026-10-17 — initial upload
• 2026-11-04 — config and docs sync

Default experiment recipe

Defaults: adam optimizer, step schedule. Treat these as starting points; a real run should sweep learning rate and batch size before drawing conclusions.

Limitations

The initialization checkpoint has not been trained or audited for robustness, fairness, or domain transfer. The implementation should be treated as an experimental starting point. Results from a future trained checkpoint must be documented separately from the defaults shipped here.

Quick check

Run python pipeline.py --help for the built-in smoke test. Because this is a custom implementation, generic automatic loading APIs require an explicit adapter before use.

Repository status

• The Python file contains the model and runnable example or training entry point.
• config.json records the generated architecture settings.
• training_args.json records the default experiment recipe.
• model.safetensors is a valid initialization checkpoint for smoke tests; it is not presented as a trained benchmark checkpoint.
• No benchmark score is claimed in this repository.

Architecture

• Architecture: Efficientformer
• Scale: xlarge
• Attention: multi query
• Fusion: cross attention
• Activation: approx gelu
• Normalization: layernorm

Files

File	Purpose
pipeline.py	primary artifact
README.md	this documentation
config.json	architecture configuration
training_args.json	default experiment settings
model.safetensors	initialization checkpoint

License

Distributed under the unknown license. External datasets keep their own terms — check them separately.