vLLM

vLLM is an open-source server for running models on your own hardware. Its API follows OpenAI’s format, so you wrap it with the OpenAI client and the SDK meters it with the OpenAI recipe.

Set up

Point the OpenAI client’s baseURL at vLLM and wrap it:

import OpenAI from 'openai';

const vllm = stigg.wrap(
  new OpenAI({ baseURL: 'http://localhost:8000/v1', apiKey: process.env.VLLM_API_KEY }),
);

const response = await vllm.chat.completions.create({
  model,
  messages,
  stiggContext: { customerId: 'customer-acme', entitlement: { currencyId: 'credits' } },
});

Use the address of your own vLLM server. http://localhost:8000/v1 is vLLM’s default.

What is reported

vLLM returns a usage object in OpenAI’s format with the number of tokens each call consumed. Calls are reported under the openai.chat.completions.create event name, with the same metrics as OpenAI. Every usage field the provider returns is also included, with a raw. prefix. See How usage is reported. For details on vLLM’s API, see vLLM’s OpenAI-compatible server docs.

Getting started

Developer tools

Governance (Beta)

AI agents and LLMs (Beta)

Modeling your pricing in Stigg

Entitlements

Managing customers, subscriptions and contracts

Getting usage data into Stigg

Snap-in widgets

Importing and exporting data

Going live

Workflow automation

Rolling out pricing changes with confidence

Price localization

Managing your account

High-availability and scale

Security & compliance

Set up

What is reported

​Set up

​What is reported

Set up

What is reported