> For the complete documentation index, see [llms.txt](https://docs.nvcl.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.nvcl.ai/overview/what-it-is.md). # What it is NVCL AI is an inference server for open language models, built from the ground up to get more out of every GPU. {% hint style="info" %} NVCL AI is in beta and free for personal use, with a free NVCL AI account: install it yourself with one command, see [Get NVCL AI](/overview/request-access.md). NVCL AI, its installer and these docs are updated all the time, so expect changes. {% endhint %} It is our own engine, written in C++ with its own CUDA kernels (Metal kernels on a Mac) and its own compressed weight formats; it is not a fork of vLLM or SGLang. It runs on Linux servers with NVIDIA GPUs and on Macs with Apple Silicon; the models and GPUs are listed in [Supported models](/reference/models.md) and [Hardware](/reference/hardware.md). Clients of the OpenAI Chat Completions API, such as the OpenAI SDK, work unchanged. Agent tools built for the OpenAI Responses API or the Anthropic Messages API connect directly, without a proxy. ## More output from every GPU than vLLM On one NVIDIA H200, NVCL AI's total output was higher than vLLM's at every load we measured, in each quality class. NVCL AI ran on its automatic settings with speculative decoding (a quick guess of the next few tokens that the model then checks in one pass). vLLM 0.32 ran with dynamic depth: vLLM changes its draft depth with the number of running requests, so one setting serves every load. Total output is all users' answer tokens per second together: it measures capacity and cost per token, not the speed one user sees. * **Full-precision class: 1.34-2.52x the total output of vLLM BF16 with dynamic depth** (one and two users measured with a separate, publicly available draft model; re-measurement with the current default, a draft head (MTP), is planned). Our Standard tier against vLLM serving the full-precision 16-bit model, 1 to 32 users. * **FP8 class: 1.16-1.85x the total output of vLLM FP8 with dynamic depth.** Our opt-in Throughput tier against vLLM serving the official 8-bit model, 1 to 768 users; from 32 users 1.16-1.27x, that is 16-27% more output from the same GPU and a 14-21% cheaper token. * **The same work finishes sooner.** At 512 users the same 1,536 requests finish 1.23x sooner on the Throughput tier (749 s against 923 s for vLLM FP8 with dynamic depth). Quality of the Throughput tier: FP8 class, by our full 1,015-task run. Qwen3.8-27B, mixed 4K-token prompts, answers up to 2,048 tokens, measured by our team on 24-25 October 2026. Every vLLM and SGLang setting we ran: [One H200](/benchmarks/speed-h200.md); method: [How we benchmark](/concepts/benchmarking.md). ## Our own quantization, lossless quality Smart quantization with lossless quality: fewer bits than FP8, the quality of full precision. Our default tier, Standard, stores each weight in about 7.13 bits, fewer than FP8's 8, so the model is about 2.2x smaller than BF16. On Qwen3.8-27B on H200 it shows no measurable quality loss on our 1,015-task suite: 732 / 732 / 730 over three runs against 726 for BF16 on vLLM 0.31 and SGLang 0.5.21 (the differences stay within its run-to-run noise; [Quality](/benchmarks/quality.md)). Need the original bit for bit? Switch to Exact. Our formats run only on NVCL AI. Check them on your own evals: run Standard and Exact side by side. See [Weight formats](/reference/formats.md). ## Big models on fewer GPUs DeepSeek-V4.1-Flash runs on two H200 plus server memory, on DeepSeek's published weights, unchanged, where vLLM's published recipe needs four; GLM-5.3, a 750B model, runs on two cards plus server memory, where the usual setups use eight (its request options: [Per model](/reference/api.md#per-model)). In plain words: [Performance at a glance](/overview/performance.md#big-models-on-fewer-gpus). ## Who it is for * **Inference providers** who want more output and a cheaper token from every GPU. * **Teams running coding agents** on their own GPUs: on Qwen3.8-27B, when an agent comes back with a conversation that is still cached, its history is not read again from the start ([Why it is fast](/concepts/why-fast.md#less-waiting-on-a-busy-server)). * **Teams running big open models in-house** on few GPUs: see [Big models on fewer GPUs](#big-models-on-fewer-gpus). ## Licence NVCL AI is proprietary. It is free for personal use; for commercial use, contact details coming soon. Until then, you can request a trial licence on [Get NVCL AI](/overview/request-access.md#commercial-use-a-trial-licence). See [Licensing](/overview/licensing.md). ## Where to go next **Not an engineer?** Start with the Overview pages: * [Performance at a glance](/overview/performance.md): where NVCL AI gets more out of every GPU than vLLM and SGLang, in plain words. * [Get NVCL AI](/overview/request-access.md): free for personal use, installed with one command. **Engineer?** Go to Concepts and Reference: * [Why it is fast](/concepts/why-fast.md) and [Why quality holds](/concepts/quality.md). * [How we benchmark](/concepts/benchmarking.md): the rules behind every comparison. * [Benchmarks](/benchmarks/benchmarks.md): detailed tables with their conditions. * [Get started](/get-started/requirements.md): requirements, install and the first request.