> For the complete documentation index, see [llms.txt](https://docs.nvcl.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.nvcl.ai/get-started/install.md).

# Install

Install the `nvcl-ai` command and a model with one line.

{% stepper %}
{% step %}

## Run the install line

On the target machine:

```bash
curl -fsSL https://get.nvcl.ai/install.sh | sh
```

It puts the `nvcl-ai` command in `~/.local/bin`, checks it against its SHA-256 fingerprint and opens the `nvcl-ai` menu in the terminal, over SSH too. With options (see [Commands](#commands)), or without a terminal, it runs `nvcl-ai install` instead. It needs no administrator rights; only a driver install, or an install of missing system tools, asks for them, after you say yes (see [Administrator rights](#administrator-rights)). Without `curl`, `wget -qO- https://get.nvcl.ai/install.sh | sh` does the same. If `~/.local/bin` is not on your `PATH`, the script prints the line to add.
{% endstep %}

{% step %}

## Choose a model and sign in

The menu opens on Home: this machine (operating system, GPUs, NVIDIA driver, RAM, free disk; on a Mac, unified memory) and whether it meets the requirements. Choose **Install a ready model**, then a model and a format: those that fit this machine are marked, and Qwen3.8-27B in the Standard format is recommended (Small on a Mac; see [Formats](#formats)). One screen then asks the rest, each with a default: the port (8000), who can connect (only this machine, or the network with an API key that `nvcl-ai` generates), where model files go and, on Linux with systemd, start at boot; with two suitable GPUs, also the [GPU layout](#gpu-layout). **Install** starts it. The first time, it asks you to sign in to your free NVCL AI account before it installs anything (see [Sign in](#sign-in)).
{% endstep %}

{% step %}

## Wait until it is ready

`nvcl-ai` installs the engine built for this machine's GPU (or Mac), checks that it runs here, downloads the model, starts it and checks that it answers. Every download is checked against the SHA-256 fingerprints in our catalog; a download that stops resumes when you install the same format again, and a piece that lags behind is fetched again over a second connection. Then the model's page opens: `c` chats with it in the terminal, and `e` shows the address, the API key when the network can connect, and examples to copy (see [A model's page](#a-models-page) and [First request](/get-started/first-request.md)). `nvcl-ai install` prints the same address, key and `curl` commands at the end.

GLM-5.3 on two B200 can stall on its first start right after the download. If loading stops making progress, run `nvcl-ai stop`, reboot the machine, then run `nvcl-ai start`: nothing is downloaded again.
{% endstep %}
{% endstepper %}

## The menu

The install line opens the menu; later, `nvcl-ai` with no arguments opens it again (`nvcl-ai tui` does too). Each action in it runs the same `nvcl-ai` command you could type, with what you chose on its screens. To put only the command on the machine, without opening the menu:

```bash
curl -fsSL https://get.nvcl.ai/install.sh | NVCL_AI_NO_INSTALL=1 sh
```

<figure><img src="https://3133985975-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FDZCnCH3aMU9wYLt2Z4wR%2Fuploads%2Fgit-blob-d0bebad9e1acbba06aecc89e1a7f45a1c81c0433%2Fnvcl-01-home.png?alt=media" alt="The nvcl-ai home screen: this machine (Ubuntu 24.04 on x86-64, an AMD EPYC CPU with 251 GB of RAM, one NVIDIA H200 NVL GPU with 140 GB, 120 GB of it in use, NVIDIA driver 580, CUDA 13.0), the requirements NVIDIA driver, supported GPU and glibc, all checked, Qwen3.8 27B standard running on port 8000 under Your models, and Install a ready model, Update nvcl-ai, Remove NVCL AI and Quit"><figcaption><p>Home: the machine, its requirements and your models.</p></figcaption></figure>

* **Home** also lists your models, each running, loading or stopped, and your NVCL AI account (see [Sign in](#sign-in)). When the NVIDIA driver is missing or too old, Home offers **Install the NVIDIA driver** (see [Administrator rights](#administrator-rights)). `r` checks the machine and the download site again; `q` quits.
* **Install a ready model** opens the catalog. A model that does not fit this machine says why, such as too few GPUs or too little GPU memory. Choosing a model lists its formats, each with its size, what is left to download and the GPU memory it needs.
* **Install** shows each step until the model answers; Esc stops a download.

<figure><img src="https://3133985975-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FDZCnCH3aMU9wYLt2Z4wR%2Fuploads%2Fgit-blob-e231fd68e31de0001fffa6747718ae7502126301%2Fnvcl-03-format-selection.png?alt=media" alt="Choosing a format of Qwen3.8 27B: exact 59 GB, standard 32 GB (recommended and selected), compact 31 GB and throughput 32 GB, each with a check mark; for standard, BF16 quality, about 2.2x smaller, 32 GB on disk and 32 GB to download, 40 GB or more of GPU memory, and Fits this machine. Enter installs it"><figcaption><p>Formats, with what each needs and whether it fits.</p></figcaption></figure>

### A model's page

Choose a model on Home to open its page. Its keys:

| Key | What it does                                                                                                                                                                                                                                                                                              |
| --- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `c` | A chat with the model in the terminal. Each answer ends with its numbers: the time, the time to the first token, the tokens in and out, and the decode speed (tokens/s after the first token). Esc stops an answer (write "continue" to go on); Ctrl+T turns thinking on or off, Ctrl+N starts a new chat |
| `e` | The API endpoint: the address, the model name, the API key and examples to copy (see [First request](/get-started/first-request.md))                                                                                                                                                                      |
| `l` | The server log, followed as it grows                                                                                                                                                                                                                                                                      |
| `s` | Stop, which frees the GPU memory, or Start                                                                                                                                                                                                                                                                |
| `f` | Another installed format, when the model is stopped                                                                                                                                                                                                                                                       |
| `g` | The [GPU layout](#gpu-layout) for the next start, when the model is stopped and there is a choice                                                                                                                                                                                                         |
| `d` | Remove the model's files from disk, after you confirm; a running model stops first                                                                                                                                                                                                                        |

Esc goes back. **Remove NVCL AI** on Home is `nvcl-ai uninstall` (see [Uninstall](#uninstall)). Without a terminal, `nvcl-ai` prints its help instead of the menu.

<figure><img src="https://3133985975-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FDZCnCH3aMU9wYLt2Z4wR%2Fuploads%2Fgit-blob-1a307eadd77eb1b810f101e8d311a51224faf7ad%2Fnvcl-07-model-running.png?alt=media" alt="A running model: Qwen3.8 27B standard at http://127.0.0.1:8000/v1, with Chat, API endpoint, Server log, Stop, Remove and Back, and the keys c chat, e endpoint, l log, s start or stop, f format, g GPUs and d remove"><figcaption><p>A running model.</p></figcaption></figure>

<figure><img src="https://3133985975-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FDZCnCH3aMU9wYLt2Z4wR%2Fuploads%2Fgit-blob-2304b9ac3379a843d11d41e16f11d246979d66d4%2Fnvcl-12-chat.png?alt=media" alt="Chat with Qwen3.8 27B standard, thinking on: asked what 17 x 23 is, the model thinks (137 characters) and answers 391; asked for a haiku about a GPU, it thinks again and writes three lines. Ctrl-T turns thinking on or off, Ctrl-N starts a new chat"><figcaption><p>Chat in the terminal.</p></figcaption></figure>

## Administrator rights

* `nvcl-ai` never uses sudo by default: with a working NVIDIA driver, and curl, systemd and procps in place, there is no sudo prompt at all.
* When the NVIDIA driver is missing or too old, it says what it will install, shows the exact commands and asks first, before it downloads or writes anything: `nvcl-ai install` before the install, `nvcl-ai driver` (**Install the NVIDIA driver** in the menu) on its own. Yes runs them with sudo; when the new driver needs a reboot, it says so and stops with exit code 4: reboot and run the same command again.
* No, or `nvcl-ai install --yes`, prints the commands and stops with exit code 4, without changing anything; run them yourself (a new driver usually needs a reboot), then run the install line again. `nvcl-ai install --install-driver` and `nvcl-ai driver --yes` answer yes.
* It has driver commands for Ubuntu, Debian 12 and 13, and RHEL, Rocky Linux, AlmaLinux, CentOS Stream and Oracle Linux 10. On another system it points to NVIDIA's driver installation guide, and when your user cannot run sudo it prints the commands; either way it stops with exit code 4.
* In a container it never installs a driver: the driver is the host's.
* On Linux, when curl, systemd or procps is missing, `nvcl-ai install` offers to install them with sudo, in a separate question that defaults to no and that `--yes` does not answer.
* A Mac needs no NVIDIA driver, so `nvcl-ai` never asks for administrator rights there.

## Sign in

Downloads need your free NVCL AI account. Starting, stopping and serving an installed model need neither the account nor the network, unless its engine has to be downloaded again.

```bash
nvcl-ai login    # sign in on this machine
nvcl-ai logout   # sign out; installed and running models are not affected
```

* `nvcl-ai login` prints a link and a code such as `ABCD-EFGH`: open the link in any browser, on any device, and approve the code. It opens no browser itself, so it works over SSH. No account yet? Create one at [app.nvcl.ai](https://app.nvcl.ai) (the link opens it too) and confirm your email, then approve the code. `nvcl-ai login` also activates this machine on your account; when it cannot, it says why and ends with exit code 3.
* `nvcl-ai install`, the install line and **Install** in the menu ask you to sign in the same way the first time; in the menu, the link and the code show on the install screen. If you do not approve the code, nothing is installed. `--yes` does not skip the sign-in: approve the code when it is printed, or run `nvcl-ai login` first.
* The session is kept in `~/.nvcl-ai/account.json`, readable only by you. `nvcl-ai status` ends with `Account:` and your email, or `Account: not signed in (nvcl-ai login)`.
* When a session has ended, `nvcl-ai install`, the install line and **Install** in the menu ask you to sign in again, the same way. Other commands that download, such as `nvcl-ai update`, say the session has ended and stop: run `nvcl-ai login`, then the same command again.

## Commands

```
nvcl-ai                          the menu, in a terminal (also nvcl-ai tui)
nvcl-ai install [--yes] [--model ID] [--format F] [--port N] [--public] [--models-dir DIR] [--boot]
                [--install-driver] [--parallel single|tp2|dp2]
                                 install the engine and a model, then start it (asks, with defaults)
nvcl-ai start [MODEL] [--format F] [--port N] [--public|--local] [--parallel L] [--boot]
                                 start an installed model in the background
nvcl-ai stop                     stop it and free the GPU memory
nvcl-ai status                   what runs, where, what is installed, and the account
nvcl-ai logs [-f] [-n N]         the server log
nvcl-ai models                   the models and formats, and which fit this machine
nvcl-ai check                    this machine and what it still needs
nvcl-ai driver [--yes]           install the NVIDIA driver this machine needs, after asking
nvcl-ai update [--check] [--yes] a newer nvcl-ai, then new builds of the installed models' engines
nvcl-ai remove MODEL [--yes]     remove one model's files, after asking
nvcl-ai uninstall [--yes]        remove NVCL AI from this machine, after asking
nvcl-ai login                    sign in to your free NVCL AI account: a link and a code for any browser
nvcl-ai logout                   sign out (installed and running models are not affected)
nvcl-ai serve [MODEL] [--format F] [--port N] [--public] [--log]
                                 the server in the foreground, as a container's command
```

`--yes` takes every default without asking, as does a run without a terminal. One model runs at a time. Exit codes: 0 done, 1 an error, 2 a wrong command or option, 3 this machine could not be activated on your account (`nvcl-ai login`), 4 an NVIDIA driver is needed or was declined, 130 interrupted. Start, stop, status and logs: [Start the server](/get-started/run.md#start-stop-status).

The install line passes options on to `nvcl-ai install`, to install and start a model without questions:

```bash
curl -fsSL https://get.nvcl.ai/install.sh | sh -s -- --yes --model qwen3.8-27b --format standard
```

Without the menu, `nvcl-ai install` (and the install line with options) shows this machine and stops with exit code 1 if it cannot run NVCL AI, for example on a GPU it has no engine for. The first time, it asks you to sign in. Then it asks the same questions in the terminal, each with a default; Enter takes it. When you run `nvcl-ai install` again, the defaults are your last answers.

## Formats

Pick a weight tier when `nvcl-ai install` asks, in the menu, or with `--format`. Without it, `nvcl-ai` installs Standard (Small on a Mac), or another tier that fits when that one does not; a tier you name is never swapped for another. What each tier needs: [Requirements](/get-started/requirements.md).

| `--format`   | Choose it for                                                                                        |
| ------------ | ---------------------------------------------------------------------------------------------------- |
| `standard`   | The default: full-precision quality class, about 2.2x smaller than BF16                              |
| `compact`    | The smallest weights on H100 and H200, about 2.8x smaller than BF16; FP8 class                       |
| `throughput` | Many users at once on Hopper cards (opt-in); FP8 class, by our full 1,015-task run                   |
| `exact`      | The original weights, bit for bit                                                                    |
| `small`      | A Mac with Apple Silicon, where it is the default: 4-5 bits per weight, about 3.3x smaller than BF16 |

These are the tiers of Qwen3.8-27B. Qwen3.8-Flash-Next's only format is also called `standard`, but it is a build of its own. On two B200, install GLM-5.3 without `--format`: `nvcl-ai` takes Compact, a build of its own for two B200.

```bash
nvcl-ai install --model qwen3.8-27b --format compact
```

What each tier is: [Weight formats](/reference/formats.md); which tiers each model has: [Supported models](/reference/models.md).

## GPU layout

Qwen3.8-27B runs on one GPU or on two; the other models run on the layout their package sets.

| `--parallel` | What it is                                                                            | Choose it for                                                |
| ------------ | ------------------------------------------------------------------------------------- | ------------------------------------------------------------ |
| `single`     | One GPU                                                                               | A machine with one suitable GPU                              |
| `tp2`        | TP2: one copy of the model split across two cards. The default with two suitable GPUs | Fewer than about 16 chats at once: every chat uses both GPUs |
| `dp2`        | DP2: a full copy of the model on each card, behind one address                        | About 16 chats at once or more: each chat runs on one GPU    |

<figure><picture><source srcset="https://3133985975-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FDZCnCH3aMU9wYLt2Z4wR%2Fuploads%2Fgit-blob-5e4fefecd5872fbc975e7da7c5ea2dcc65caebcc%2Fdiagram-gpu-layouts-dark.svg?alt=media" media="(prefers-color-scheme: dark)"><img src="https://3133985975-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FDZCnCH3aMU9wYLt2Z4wR%2Fuploads%2Fgit-blob-533c81139ed8915d65f068436408e3afd7f4f67c%2Fdiagram-gpu-layouts-light.svg?alt=media" alt="Diagram of the two layouts on two GPUs. TP2: one address and one copy of the model split across both GPUs, so every chat uses both GPUs; choose it for a few chats at once. DP2: one address with a router that sends each chat to one GPU, and a full copy of the model on each GPU; choose it for many chats at once."></picture><figcaption><p>TP2 splits one copy of the model across both GPUs; DP2 keeps a full copy on each GPU, behind one address.</p></figcaption></figure>

Measured results: [Which layout to use](/benchmarks/two-gpus.md#which-layout-to-use).

With two suitable GPUs, `nvcl-ai install` takes TP2 without asking, and the menu asks; with one, `nvcl-ai` uses one GPU. The layout is kept for later starts, and `nvcl-ai status` shows it. To change it, stop the model and choose another on its page (`g`), or pass `--parallel` to `nvcl-ai install` or `nvcl-ai start`:

```bash
nvcl-ai start qwen3.8-27b --parallel dp2
```

## Model files you already have

Model files can be on the machine before the install, copied or linked from another folder. Put them in a models folder the way `nvcl-ai` lays them out, `<folder>/<model>/cuda/<part>/` (`metal` instead of `cuda` on a Mac; see [Where files go](#where-files-go)), and install with `--models-dir <folder>`, or choose that folder in the menu. `nvcl-ai` checks the files against the SHA-256 fingerprints in our catalog and downloads only what is missing or differs; the free-space check counts only that download.

* A linked file, symbolic or hard, is never written through. One that differs is replaced by a downloaded copy, and the file it pointed to stays as it is.
* Files that match are only read, so a read-only folder works when all of a model's files are there.
* `nvcl-ai remove` and `nvcl-ai uninstall` delete the links, not the files they point to.

## Update

This page describes `nvcl-ai` 0.5.8; `nvcl-ai --version` prints yours.

```bash
nvcl-ai update --check   # is a newer version available?
nvcl-ai update           # install it
```

`nvcl-ai update` (**Update nvcl-ai** in the menu) replaces the command with a newer one, checked against its SHA-256 fingerprint in our list of releases, then installs again the engine of each installed model whose build changed. A new engine needs your [NVCL AI account](#sign-in): when you are not signed in, `nvcl-ai update` stops and says so: run `nvcl-ai login`, then `nvcl-ai update` again. A model that runs stops while its engine is replaced and starts again after you say yes (`--yes` without asking); model files are not downloaded again. The menu starts again in the new version. There are no automatic updates.

## Remove a model

`nvcl-ai remove MODEL` (`d` on the model's page) deletes all of one model's files, in every format, after asking; `--yes` skips the question. A model that runs stops first.

## Uninstall

`nvcl-ai uninstall` (**Remove NVCL AI** on Home) lists what it will remove and asks first: the command, the engine, the model files (also in a folder you chose with `--models-dir`), the API key, the settings and the logs. It stops a running model first and signs you out of your NVCL AI account; an NVIDIA driver it installed stays.

## Where files go

| What                          | Where                                                                                                                                                                                                 |
| ----------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Everything `nvcl-ai` installs | `~/.nvcl-ai` (set another with `NVCL_HOME`)                                                                                                                                                           |
| Engines                       | `~/.nvcl-ai/engines/`                                                                                                                                                                                 |
| Models                        | `~/.nvcl-ai/models/<model>/cuda/` (`metal/` on a Mac), one folder per part (`base/`, `standard/`, `compact/`, ...); `nvcl-ai install --models-dir DIR` puts them under `DIR`, kept for later installs |
| Server log                    | `~/.nvcl-ai/logs/server.log` (`nvcl-ai logs`)                                                                                                                                                         |
| Download log                  | `~/.nvcl-ai/logs/download.log` (each download, part by part, with every retry)                                                                                                                        |
| API key for `--public`        | `~/.nvcl-ai/api-key`, readable only by you                                                                                                                                                            |
| Your NVCL AI sign-in          | `~/.nvcl-ai/account.json`, readable only by you (`nvcl-ai logout` removes it)                                                                                                                         |
| The command                   | `~/.local/bin/nvcl-ai`                                                                                                                                                                                |

Next: [Start the server](/get-started/run.md).