> ## Documentation Index
> Fetch the complete documentation index at: https://docs.stigg.io/llms.txt
> Use this file to discover all available pages before exploring further.

# vLLM

> Wrap vLLM's OpenAI-compatible API to meter and gate calls with Stigg.

vLLM is an open-source server for running models on your own hardware. Its API follows OpenAI's format, so you wrap it with the `OpenAI` client and the SDK meters it with the OpenAI recipe.

## Set up

Point the `OpenAI` client's `baseURL` at vLLM and wrap it:

```typescript theme={null}
import OpenAI from 'openai';

const vllm = stigg.wrap(
  new OpenAI({ baseURL: 'http://localhost:8000/v1', apiKey: process.env.VLLM_API_KEY }),
);

const response = await vllm.chat.completions.create({
  model,
  messages,
  stiggContext: { customerId: 'customer-acme', entitlement: { currencyId: 'credits' } },
});
```

Use the address of your own vLLM server. `http://localhost:8000/v1` is vLLM's default.

## What is reported

vLLM returns a `usage` object in OpenAI's format with the number of tokens each call consumed. The calls go through the `OpenAI` client, so they are reported under OpenAI event names. The wrapper reads the counts and reports them to Stigg as the metrics below.

| Method | Event name |
| - | - |
| `chat.completions.create` | `openai.chat.completions.create` |

**Chat Completions**

| Metric | What it counts |
| - | - |
| `prompt_tokens` | Input tokens, **including** cached tokens |
| `completion_tokens` | Output tokens |
| `cached_tokens` | Input tokens served from the cache, when the vendor reports them |
| `total_tokens` | All input and output tokens, as reported by the vendor |

`total_tokens` includes the other token counts, so price the total or the parts, never both. To tell vLLM apart from other vendors on the same event name, add a dimension such as `provider` to the `stiggContext`.

Every usage field the provider returns is also included, with a `raw.` prefix. See [How usage is reported](/documentation/ai-agents/usage-metering#how-usage-is-reported).

For details on vLLM's API, see vLLM's [OpenAI-compatible server docs](https://docs.vllm.ai/en/latest/serving/openai_compatible_server/).


## Related topics

- [Usage metering](/documentation/ai-agents/usage-metering.md)
- [Bring your own vendor](/documentation/ai-agents/providers/custom.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.