Text Generation
Transformers
Safetensors
qwen2
coder
code
agent
conversational
text-generation-inference
Instructions to use AdminReal/NexusCoder with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use AdminReal/NexusCoder with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="AdminReal/NexusCoder") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("AdminReal/NexusCoder") model = AutoModelForCausalLM.from_pretrained("AdminReal/NexusCoder", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use AdminReal/NexusCoder with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "AdminReal/NexusCoder" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "AdminReal/NexusCoder", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/AdminReal/NexusCoder
- SGLang
How to use AdminReal/NexusCoder with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "AdminReal/NexusCoder" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "AdminReal/NexusCoder", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "AdminReal/NexusCoder" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "AdminReal/NexusCoder", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use AdminReal/NexusCoder with Docker Model Runner:
docker model run hf.co/AdminReal/NexusCoder
| [build-system] | |
| requires = ["setuptools>=68.0", "wheel"] | |
| build-backend = "setuptools.build_meta" | |
| [project] | |
| name = "nexus-coder" | |
| version = "0.4.0" | |
| description = "Nexus Coder v0.4 - CyberForge edition. MoE 423B/39B + 3M context + CyberGym training. Created by Hieu Louis." | |
| readme = "README.md" | |
| license = {file = "LICENSE"} | |
| authors = [ | |
| {name = "Hieu Louis", email = "mhieuhonda@users.noreply.github.com"} | |
| ] | |
| requires-python = "==3.12.13" | |
| dependencies = [ | |
| "torch>=2.0.0", | |
| "numpy>=1.24.0", | |
| "tqdm>=4.65.0", | |
| "pyyaml>=6.0", | |
| "datasets>=2.14.0", | |
| "requests>=2.31.0", | |
| "cryptography>=41.0.0", | |
| ] | |
| keywords = ["ai", "llm", "moe", "mixture-of-experts", "transformer", "nexus", "agent", "skills", "tools", "cyberforge", "cybergym"] | |
| classifiers = [ | |
| "Development Status :: 4 - Beta", | |
| "Intended Audience :: Developers", | |
| "Intended Audience :: Science/Research", | |
| "License :: Other/Proprietary License", | |
| "Programming Language :: Python :: 3.12", | |
| "Topic :: Scientific/Engineering :: Artificial Intelligence", | |
| ] | |
| [project.optional-dependencies] | |
| dev = ["pytest>=7.0", "black", "flake8", "ruff"] | |
| gpu = ["flash-attn>=2.0.0", "bitsandbytes>=0.41.0", "triton>=2.0.0"] | |
| data = ["datasets>=2.14.0", "datasketch>=1.6.0", "langdetect>=1.0.9"] | |
| tools = ["ruff>=0.1.0", "black>=23.0.0", "isort>=5.12.0", "sqlparse>=0.4.4", "jsbeautifier>=1.14.0"] | |
| crypto = ["cryptography>=41.0.0", "pyjwt>=2.8.0"] | |
| database = [ | |
| "sqlalchemy>=2.0.0", | |
| "psycopg2-binary>=2.9.0", | |
| "pymysql>=1.1.0", | |
| "redis>=5.0.0", | |
| "pymongo>=4.5.0", | |
| "elasticsearch>=8.0.0", | |
| "kafka-python>=2.0.2", | |
| "pika>=1.3.0", | |
| ] | |
| web = [ | |
| "aiohttp>=3.9.0", | |
| "websockets>=12.0", | |
| "grpcio>=1.59.0", | |
| "beautifulsoup4>=4.12.0", | |
| "lxml>=4.9.0", | |
| ] | |
| devops = ["paramiko>=3.4.0", "kubernetes>=28.1.0", "docker>=7.0.0"] | |
| media = ["Pillow>=10.0.0", "reportlab>=4.0.0", "markdown>=3.5.0"] | |
| ml = ["scikit-learn>=1.3.0", "scipy>=1.11.0", "transformers>=4.35.0", "accelerate>=0.24.0", "peft>=0.6.0", "lm-eval>=0.3.0"] | |
| distributed = ["deepspeed>=0.12.0", "accelerate>=0.24.0", "flash-attn>=2.0.0"] | |
| all = [ | |
| "flash-attn>=2.0.0", | |
| "bitsandbytes>=0.41.0", | |
| "triton>=2.0.0", | |
| "datasets>=2.14.0", | |
| "datasketch>=1.6.0", | |
| "langdetect>=1.0.9", | |
| "ruff>=0.1.0", | |
| "black>=23.0.0", | |
| "isort>=5.12.0", | |
| "sqlparse>=0.4.4", | |
| "jsbeautifier>=1.14.0", | |
| "cryptography>=41.0.0", | |
| "pyjwt>=2.8.0", | |
| "aiohttp>=3.9.0", | |
| "websockets>=12.0", | |
| "grpcio>=1.59.0", | |
| "beautifulsoup4>=4.12.0", | |
| "lxml>=4.9.0", | |
| "sqlalchemy>=2.0.0", | |
| "psycopg2-binary>=2.9.0", | |
| "pymysql>=1.1.0", | |
| "redis>=5.0.0", | |
| "pymongo>=4.5.0", | |
| "elasticsearch>=8.0.0", | |
| "kafka-python>=2.0.2", | |
| "pika>=1.3.0", | |
| "paramiko>=3.4.0", | |
| "kubernetes>=28.1.0", | |
| "docker>=7.0.0", | |
| "Pillow>=10.0.0", | |
| "reportlab>=4.0.0", | |
| "markdown>=3.5.0", | |
| "scikit-learn>=1.3.0", | |
| "scipy>=1.11.0", | |
| "transformers>=4.35.0", | |
| "accelerate>=0.24.0", | |
| "peft>=0.6.0", | |
| ] | |
| [project.urls] | |
| Homepage = "https://github.com/mhieuhonda/NexusCoder" | |
| Repository = "https://github.com/mhieuhonda/NexusCoder" | |
| Issues = "https://github.com/mhieuhonda/NexusCoder/issues" | |
| Changelog = "https://github.com/mhieuhonda/NexusCoder/blob/main/CHANGELOG.md" | |
| [tool.setuptools.packages.find] | |
| include = ["nexus*"] | |