vLLM inference: adapter_vllm/ (decision head as lm_head LoRA), model card section "Inference with vLLM", OpenAI client, measured speed

b16c3a6 verified about 9 hours ago

208 Bytes

	# shared runtime env for training/eval on the shared-CPU B200 box
	export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
	export TOKENIZERS_PARALLELISM=false
	export OMP_NUM_THREADS=4
	export MKL_NUM_THREADS=4