# Speech
Source: https://docs.zerogpu.ai/api-reference/audio-speech

api-reference/openapi/zerogpu.openapi.json POST /audio/speech
Generate speech from text with chatterbox-nano, with optional voice cloning from a reference clip.

## Limits

| Limit | Value |
| - | - |
| `input` length | 2,000 characters, English only. Requests near the limit can time out with `524`. |
| `voice_sample` duration | 6 to 30 seconds |
| `voice_sample` size | 10 MB |
| `voice_sample` formats | mp3, m4a, wav, flac, ogg, webm, mp4 |
| Unsupported parameters | `instructions`, `exaggeration`, `cfg_weight`, `min_p`, and `speed` other than `1.0` return `400` |

<RequestExample>
  ```bash cURL (voice cloning) theme={null}
  curl --location 'https://api.zerogpu.ai/v1/audio/speech' \
    --header 'x-api-key: YOUR_API_KEY' \
    --form 'model="chatterbox-nano"' \
    --form 'input="Testing model to generate voice. Hey how are you today?"' \
    --form 'response_format="mp3"' \
    --form 'seed="1234"' \
    --form 'voice_sample=@"/path/to/file"' \
    --output speech.mp3
  ```

  ```bash cURL (JSON) theme={null}
  curl --location 'https://api.zerogpu.ai/v1/audio/speech' \
    --header 'content-type: application/json' \
    --header 'Authorization: Bearer YOUR_API_KEY' \
    --data '{
    "model": "chatterbox-nano",
    "input": "Hello from ZeroGPU.",
    "voice": "alloy",
    "response_format": "mp3"
  }' \
    --output speech.mp3
  ```

  ```python Python theme={null}
  from openai import OpenAI

  client = OpenAI(
      base_url="https://api.zerogpu.ai/v1",
      api_key="YOUR_API_KEY",  # sent as Authorization: Bearer
  )

  with client.audio.speech.with_streaming_response.create(
      model="chatterbox-nano",
      voice="alloy",  # OpenAI voice names map to the built-in voice
      input="Hello from ZeroGPU.",
      response_format="mp3",
  ) as response:
      response.stream_to_file("speech.mp3")
  ```

  ```javascript JavaScript theme={null}
  import fs from "node:fs";
  import OpenAI from "openai";

  const client = new OpenAI({
    baseURL: "https://api.zerogpu.ai/v1",
    apiKey: "YOUR_API_KEY", // sent as Authorization: Bearer
  });

  const speech = await client.audio.speech.create({
    model: "chatterbox-nano",
    voice: "alloy", // OpenAI voice names map to the built-in voice
    input: "Hello from ZeroGPU.",
    response_format: "mp3",
  });

  fs.writeFileSync("speech.mp3", Buffer.from(await speech.arrayBuffer()));
  ```
</RequestExample>

<ResponseExample>
  ```text 200 theme={null}
  HTTP/2 200
  content-type: audio/mpeg
  x-audio-duration-seconds: 1.76

  <binary MP3 audio>
  ```
</ResponseExample>


# Transcriptions
Source: https://docs.zerogpu.ai/api-reference/audio-transcriptions

api-reference/openapi/zerogpu.openapi.json POST /audio/transcriptions
Transcribe audio into text with whisper-tiny.

## Limits

| Limit | Value |
| - | - |
| File size | 25 MB |
| Audio duration | 10 minutes |
| Formats | flac, m4a, mp3, mp4, mpeg, mpga, oga, ogg, wav, webm |

To translate speech into English, send the same form to `POST /v1/audio/translations`, without `language`.

<RequestExample>
  ```bash cURL theme={null}
  curl --location 'https://api.zerogpu.ai/v1/audio/transcriptions' \
    --header 'x-api-key: YOUR_API_KEY' \
    --form 'model="whisper-tiny"' \
    --form 'file=@"/path/to/audio.mp3"'
  ```

  ```python Python theme={null}
  from openai import OpenAI

  client = OpenAI(
      base_url="https://api.zerogpu.ai/v1",
      api_key="YOUR_API_KEY",  # sent as Authorization: Bearer
  )

  with open("/path/to/audio.mp3", "rb") as audio:
      transcript = client.audio.transcriptions.create(
          model="whisper-tiny",
          file=audio,
      )

  print(transcript.text)
  ```

  ```javascript JavaScript theme={null}
  import fs from "node:fs";
  import OpenAI from "openai";

  const client = new OpenAI({
    baseURL: "https://api.zerogpu.ai/v1",
    apiKey: "YOUR_API_KEY", // sent as Authorization: Bearer
  });

  const transcript = await client.audio.transcriptions.create({
    model: "whisper-tiny",
    file: fs.createReadStream("/path/to/audio.mp3"),
  });

  console.log(transcript.text);
  ```
</RequestExample>

<ResponseExample>
  ```json json theme={null}
  {
    "text": "the quick brown fox jumps over the lazy dog."
  }
  ```

  ```json verbose_json theme={null}
  {
    "task": "transcribe",
    "language": "english",
    "duration": 1.76,
    "text": "Hello from Zero GPU.",
    "segments": [
      {
        "id": 0,
        "seek": 0,
        "start": 0.0,
        "end": 2.0,
        "text": " Hello from Zero GPU.",
        "tokens": [50364, 2425, 490, 17182, 18407, 13, 50464],
        "temperature": 0.0,
        "avg_logprob": -0.797803,
        "compression_ratio": 0.724138,
        "no_speech_prob": 0.023181
      }
    ]
  }
  ```
</ResponseExample>


# Cancel batch
Source: https://docs.zerogpu.ai/api-reference/cancel-batch

api-reference/openapi/batch.openapi.json POST /v1/batches/{batch_id}/cancel
Interactive playground for POST /v1/batches/{batch_id}/cancel.

Request cancellation while a batch is `in_progress` or `validating`. Terminal batches return `409`.

Poll status on [Retrieve batch](/api-reference/retrieve-batch). Full reference: [Batches API](/docs/batch/objects).


# Chat completions
Source: https://docs.zerogpu.ai/api-reference/chat-completions

api-reference/openapi/zerogpu.openapi.json POST /chat/completions
OpenAI-style chat inference for models that use this route.

Some models are invoked with a **chat-completions** style body (`messages` array) instead of the [Responses](/api-reference/responses) `input` field. The dashboard and [model catalog](/docs/model-catalog) indicate which route applies. Each model page has its own playground with the right examples for that model.

Install the official SDK from [npm](https://www.npmjs.com/package/zerogpu-api) or [PyPI](https://pypi.org/project/zerogpu-api/) (`pip install zerogpu-api`). Source: [zerogpu/SDK](https://github.com/zerogpu/SDK).

Response JSON shape depends on the model; handle errors the same way as [API error codes](/docs/production-patterns#handle-status-codes-explicitly).


# Create batch
Source: https://docs.zerogpu.ai/api-reference/create-batch

api-reference/openapi/batch.openapi.json POST /v1/batches
Interactive playground for POST /v1/batches.

Create a batch job from a JSONL file you already uploaded.

1. Upload input JSONL on [Upload file](/api-reference/upload-file) or via [Quickstart](/docs/batch/getting-started) (`purpose=batch`).
2. Paste the returned `file-…` id into `input_file_id` below.
3. Set `endpoint` to `/v1/chat/completions` (must match every line in your JSONL).

Full reference: [Batches API](/docs/batch/objects).


# Delete file
Source: https://docs.zerogpu.ai/api-reference/delete-file

api-reference/openapi/batch.openapi.json DELETE /v1/files/{file_id}
Interactive playground for DELETE /v1/files/{file_id}.

Soft-delete a file. It is removed from storage immediately and returns `404` on list, retrieve, and download afterward. Deleting an input file does not stop an active batch.

Full reference: [Files API](/docs/batch/objects).


# Download file
Source: https://docs.zerogpu.ai/api-reference/download-file

api-reference/openapi/batch.openapi.json GET /v1/files/{file_id}/content
Interactive playground for GET /v1/files/{file_id}/content.

Download the raw JSONL body for an input file you uploaded or for `output_file_id` / `error_file_id` from a [completed batch](/api-reference/retrieve-batch).

Response is streamed as `application/jsonl`. Full reference: [Files API](/docs/batch/objects).


# Embeddings
Source: https://docs.zerogpu.ai/api-reference/embeddings

api-reference/openapi/zerogpu.openapi.json POST /embeddings
Turn text into vectors for semantic search, RAG, clustering, and deduplication.

Send text as `input` and get back OpenAI's embeddings envelope: an `object` of `list`, a `data` array with one `{ object, index, embedding }` entry per input, the resolved `model`, and `usage`. It is drop-in compatible with OpenAI's Embeddings API, so an existing client only needs its base URL and model id changed. See the [Embeddings](/docs/embeddings) guide for a prefilled example, or a model page ([all-minilm-l6-v2](/api-reference/models/all-minilm-l6-v2), [bge-small-en-v1.5](/api-reference/models/bge-small-en-v1-5)) for an interactive playground.

Embedding models are routable **only** on this endpoint. A `/responses` or `/chat/completions` call with an embedding model returns `400`. Pass `input` as a string, or as an array of strings to embed a batch in one request; `data[i].index` maps each vector back to its input. Both models return 384-dimensional vectors, so they are interchangeable in an existing index.

Embeddings are billed on input tokens only. `usage` carries `prompt_tokens` and `total_tokens`, and there are no output tokens to bill.

Install the official SDK from [npm](https://www.npmjs.com/package/zerogpu-api) or [PyPI](https://pypi.org/project/zerogpu-api/) (`pip install zerogpu-api`). Source: [zerogpu/SDK](https://github.com/zerogpu/SDK). Handle errors the same way as [API error codes](/docs/production-patterns#handle-status-codes-explicitly).


# List batches
Source: https://docs.zerogpu.ai/api-reference/list-batches

api-reference/openapi/batch.openapi.json GET /v1/batches
Interactive playground for GET /v1/batches.

List batch jobs in your project. Use `after` with `last_id` from a previous page to paginate.

Full reference: [Batches API](/docs/batch/objects).


# List files
Source: https://docs.zerogpu.ai/api-reference/list-files

api-reference/openapi/batch.openapi.json GET /v1/files
Interactive playground for GET /v1/files.

List files for your project. Filter with `purpose=batch` (uploads) or `purpose=batch_output` (batch results; check `is_error` on each item for error files).

Use `last_id` from the response as `after` on the next request to page. Full reference: [Files API](/docs/batch/objects).


# all-minilm-l6-v2
Source: https://docs.zerogpu.ai/api-reference/models/all-minilm-l6-v2

api-reference/openapi/playgrounds/all-minilm-l6-v2.openapi.json POST /embeddings
Model details for all-minilm-l6-v2. 384-dimensional sentence embeddings for semantic search and RAG.

<Note>
  This model is routable only on `/v1/embeddings`. A `/responses` or
  `/chat/completions` call with an embedding model returns `400`. Send `input`
  as a string, or as an array of strings to embed a batch in one request. The
  response `model` field reads `all-MiniLM-L6-v2`, the upstream capitalization
  of the id you send.
</Note>

> Sentence-Transformers' all-MiniLM-L6-v2 is the default workhorse of semantic search. It maps a sentence or short paragraph to a 384-dimensional vector, trained with contrastive learning on more than a billion sentence pairs, so cosine distance between two vectors tracks how close the two texts are in meaning. At 22.7M parameters it embeds fast and cheap enough to index a whole corpus and re-embed it whenever your content changes, which is what retrieval-augmented generation, deduplication, clustering, and recommendation pipelines actually need. Inputs are truncated at 256 tokens, so embed chunks rather than whole documents.

Embeddings are priced on input tokens only. There are no output tokens to bill, so a call costs \$0.004 per 1M tokens embedded.

**References:** [Model card](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) • [License](https://www.apache.org/licenses/LICENSE-2.0) • [Terms](https://zerogpu.ai/terms) • [Privacy](https://zerogpu.ai/privacy-policy)


# bge-small-en-v1.5
Source: https://docs.zerogpu.ai/api-reference/models/bge-small-en-v1-5

api-reference/openapi/playgrounds/bge-small-en-v1_5.openapi.json POST /embeddings
Model details for bge-small-en-v1.5. 384-dimensional English retrieval embeddings with a 512-token window.

<Note>
  This model is routable only on `/v1/embeddings`. A `/responses` or
  `/chat/completions` call with an embedding model returns `400`. Send `input`
  as a string, or as an array of strings to embed a batch in one request.
</Note>

> BAAI's BGE-small-en-v1.5 is a retrieval-first English embedding model and one of the strongest performers on the MTEB benchmark for its size. Like [all-minilm-l6-v2](/api-reference/models/all-minilm-l6-v2) it produces 384-dimensional vectors, so it is a drop-in swap in an existing index, but it takes a 512-token window instead of 256 and is tuned specifically for dense retrieval rather than general sentence similarity. At 33M parameters it is the one to reach for when the job is ranking passages against a query: RAG retrieval, search over a document corpus, or reranking candidates before they reach a larger model.

Embeddings are priced on input tokens only. There are no output tokens to bill, so a call costs \$0.004 per 1M tokens embedded.

**References:** [Model card](https://huggingface.co/BAAI/bge-small-en-v1.5) • [License](https://opensource.org/license/mit) • [Terms](https://zerogpu.ai/terms) • [Privacy](https://zerogpu.ai/privacy-policy)


# chatterbox-nano
Source: https://docs.zerogpu.ai/api-reference/models/chatterbox-nano

api-reference/openapi/playgrounds/chatterbox-nano.openapi.json POST /audio/speech
Model details for chatterbox-nano. English text-to-speech with voice cloning.

<Note>
  This model is routable only on `/v1/audio/speech`. To clone a voice, send
  `multipart/form-data` with a `voice_sample` file.
</Note>

> Resemble AI's Chatterbox Nano is a 110M-parameter English text-to-speech model built for fast, low-cost speech generation. It speaks your text in a built-in voice, or clones a speaker from a short reference clip you upload with the request, and inline tags such as `[laugh]`, `[chuckle]`, and `[cough]` add paralinguistic cues. Its compact size keeps generation cheap enough for high-volume work: voice agents, conversational AI, gaming, accessibility, audiobooks, and content creation.

**References:** [Model card](https://huggingface.co/ResembleAI/chatterbox-nano) • [License](https://github.com/resemble-ai/chatterbox/blob/master/LICENSE) • [Terms](https://zerogpu.ai/terms) • [Privacy](https://zerogpu.ai/privacy-policy)

## Limits

| Limit | Value |
| - | - |
| `input` length | 2,000 characters, English only. Requests near the limit can time out with `524`. |
| `voice_sample` duration | 6 to 30 seconds |
| `voice_sample` size | 10 MB |
| `voice_sample` formats | mp3, m4a, wav, flac, ogg, webm, mp4 |
| Unsupported parameters | `instructions`, `exaggeration`, `cfg_weight`, `min_p`, and `speed` other than `1.0` return `400` |

<RequestExample>
  ```bash cURL (voice cloning) theme={null}
  curl --location 'https://api.zerogpu.ai/v1/audio/speech' \
    --header 'x-api-key: YOUR_API_KEY' \
    --form 'model="chatterbox-nano"' \
    --form 'input="Testing model to generate voice. Hey how are you today?"' \
    --form 'response_format="mp3"' \
    --form 'seed="1234"' \
    --form 'voice_sample=@"/path/to/file"' \
    --output speech.mp3
  ```

  ```bash cURL (JSON) theme={null}
  curl --location 'https://api.zerogpu.ai/v1/audio/speech' \
    --header 'content-type: application/json' \
    --header 'Authorization: Bearer YOUR_API_KEY' \
    --data '{
    "model": "chatterbox-nano",
    "input": "Hello from ZeroGPU.",
    "voice": "alloy",
    "response_format": "mp3"
  }' \
    --output speech.mp3
  ```

  ```python Python theme={null}
  from openai import OpenAI

  client = OpenAI(
      base_url="https://api.zerogpu.ai/v1",
      api_key="YOUR_API_KEY",  # sent as Authorization: Bearer
  )

  with client.audio.speech.with_streaming_response.create(
      model="chatterbox-nano",
      voice="alloy",  # OpenAI voice names map to the built-in voice
      input="Hello from ZeroGPU.",
      response_format="mp3",
  ) as response:
      response.stream_to_file("speech.mp3")
  ```

  ```javascript JavaScript theme={null}
  import fs from "node:fs";
  import OpenAI from "openai";

  const client = new OpenAI({
    baseURL: "https://api.zerogpu.ai/v1",
    apiKey: "YOUR_API_KEY", // sent as Authorization: Bearer
  });

  const speech = await client.audio.speech.create({
    model: "chatterbox-nano",
    voice: "alloy", // OpenAI voice names map to the built-in voice
    input: "Hello from ZeroGPU.",
    response_format: "mp3",
  });

  fs.writeFileSync("speech.mp3", Buffer.from(await speech.arrayBuffer()));
  ```
</RequestExample>

<ResponseExample>
  ```text 200 theme={null}
  HTTP/2 200
  content-type: audio/mpeg
  x-audio-duration-seconds: 1.76

  <binary MP3 audio>
  ```
</ResponseExample>


# deberta-v3-small
Source: https://docs.zerogpu.ai/api-reference/models/deberta-v3-small

api-reference/openapi/playgrounds/deberta-v3-small.openapi.json POST /responses
Model details for deberta-v3-small.

> Microsoft's DeBERTa-v3-small is a fast, lightweight zero-shot text classifier for high-volume routing, filtering, and tagging. Hand it any text alongside your own candidate labels and it returns a confidence score for each. No training data and no fixed taxonomy, just the categories you define at request time. Because it scores every label rather than forcing one choice, you get soft probabilities to threshold, rank, or route on however your pipeline needs. Purpose-built and CPU-optimized, it runs faster and cheaper than asking a general-purpose frontier model to classify.

**References:** [Model docs](https://github.com/microsoft/DeBERTa/tree/master/docs) • [Terms](https://github.com/microsoft/DeBERTa/blob/master/SECURITY.md) • [Privacy](https://github.com/microsoft/DeBERTa/blob/master/LICENSE)


# deepseek-v4.1-flash
Source: https://docs.zerogpu.ai/api-reference/models/deepseek-v4-1-flash

api-reference/openapi/playgrounds/deepseek-v4_1-flash.openapi.json POST /responses
Model details for deepseek-v4.1-flash. Reasoning, coding, and agentic workflows with a 1M-token context window.

> DeepSeek's DeepSeek-V4.1-Flash is an open-weight sparse Mixture-of-Experts model and the first built on DeepSeek's Causal Encoder-Decoder (CED) architecture, activating 8B parameters on input and 16B on output, served on ZeroGPU for general text generation. It keeps the 1,048,576-token (1M) context window of the V4 Flash line, which suits large codebases, long documents, extended conversations, and multi-step agent tasks, and supports function calling alongside both fast non-thinking replies and higher-effort reasoning.

**References:** [Model docs](https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash) • [Terms](https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash/blob/main/LICENSE) • [Privacy](https://zerogpu.ai/privacy-policy)


# gliner2.5-base-v1
Source: https://docs.zerogpu.ai/api-reference/models/gliner2-5-base-v1

api-reference/openapi/playgrounds/gliner2_5-base-v1.openapi.json POST /responses
Model details for gliner2.5-base-v1. English schema-based information extraction built on DeBERTa-v3-base.

> Fastino's GLiNER2.5 Base is an English schema-based information extraction model built on DeBERTa-v3-base. A single model covers entity extraction, text classification, structured record extraction, relation extraction, and span-level attributes — you supply a label set or a schema at inference time instead of fine-tuning one model per task. It fits GTM workflows such as parsing inbound leads, extracting company and contact details, classifying sales conversations, identifying relationships between people and companies, structuring CRM notes, and enriching customer or account data.

**References:** [Model docs](https://huggingface.co/fastino/gliner2.5-base-v1) • [Terms](https://github.com/fastino-ai/GLiNER2/blob/main/LICENSE) • [Privacy](https://github.com/fastino-ai/GLiNER2/blob/main/LICENSE)


# gliner2.5-decide-v1
Source: https://docs.zerogpu.ai/api-reference/models/gliner2-5-decide-v1

api-reference/openapi/playgrounds/gliner2_5-decide-v1.openapi.json POST /responses
Model details for gliner2.5-decide-v1. Zero-shot decision and classification with caller-supplied labels.

> Fastino's GLiNER2.5 Decide is a 486M-parameter zero-shot decision and classification model built for operational AI workflows. You supply the labels or decision options at request time to classify intent, route agents and tools, triage support requests, make moderation decisions, detect completion, and determine when a request needs human handoff. It handles single-label, multi-label, and several decision heads in one call, which suits high-volume routing and decision workloads where speed and structured outputs matter.

**References:** [Model docs](https://huggingface.co/fastino/GLiNER2.5-Decide) • [Terms](https://github.com/fastino-ai/GLiNER2/blob/main/LICENSE) • [Privacy](https://github.com/fastino-ai/GLiNER2/blob/main/LICENSE)


# gliner2.5-multi-v1
Source: https://docs.zerogpu.ai/api-reference/models/gliner2-5-multi-v1

api-reference/openapi/playgrounds/gliner2_5-multi-v1.openapi.json POST /responses
Model details for gliner2.5-multi-v1. Multilingual information extraction built on mDeBERTa-v3-base.

> Fastino's GLiNER2.5 Multi is a 287M-parameter multilingual information extraction model built on mDeBERTa-v3-base. It supports zero-shot named entity recognition, text classification, and structured JSON extraction across multiple languages, so it can pull people, organizations, locations, attributes, and other custom entities out of text without task-specific retraining.

**References:** [Model docs](https://huggingface.co/fastino/gliner2.5-multi-v1) • [Terms](https://github.com/fastino-ai/GLiNER2/blob/main/LICENSE) • [Privacy](https://github.com/fastino-ai/GLiNER2/blob/main/LICENSE)


# glm-5.3-flash
Source: https://docs.zerogpu.ai/api-reference/models/glm-5-3-flash

api-reference/openapi/playgrounds/glm-5_3-flash.openapi.json POST /responses
Model details for glm-5.3-flash. Coding and long-horizon agent tasks with a 1M-token context window.

> Z.ai's GLM-5.3-Flash is an efficient open-weight model for coding and long-horizon agent tasks, served on ZeroGPU for general text generation. Its hybrid sparse and linear attention keeps long-context behaviour accurate across a 1,048,576-token (1M) window while reducing compute, and it supports function calling and adjustable reasoning effort.

**References:** [Model docs](https://huggingface.co/zai-org/GLM-5.3-Flash) • [Terms](https://huggingface.co/zai-org/GLM-5.3-Flash/blob/main/LICENSE) • [Privacy](https://zerogpu.ai/privacy-policy)


# gpt-4.1-mini
Source: https://docs.zerogpu.ai/api-reference/models/gpt-4-1-mini

api-reference/openapi/playgrounds/gpt-4_1-mini.openapi.json POST /responses
Model details for gpt-4.1-mini. Instruction following and tool calling with a 1,047,576-token context window.

> OpenAI's GPT-4.1 mini is the fast, cost-efficient model of the GPT-4.1 family, served on ZeroGPU for general text generation. It excels at instruction following and tool calling, supports function calling and structured outputs, and works across a 1,047,576-token (1M) context window with low latency.

**References:** [Terms](https://openai.com/policies/services-agreement/) • [Privacy](https://zerogpu.ai/privacy-policy)


# gpt-5.4-nano
Source: https://docs.zerogpu.ai/api-reference/models/gpt-5-4-nano

api-reference/openapi/playgrounds/gpt-5_4-nano.openapi.json POST /responses
Model details for gpt-5.4-nano. High-volume, latency-sensitive workloads with a 400K-token context window.

> OpenAI's GPT-5.4 nano is the most cost-efficient model in the GPT-5.4 family, served on ZeroGPU for high-volume and latency-sensitive workloads such as classification, extraction, routing, and sub-agent tasks. It supports function calling and structured outputs across a 400,000-token context window.

**References:** [Terms](https://openai.com/policies/services-agreement/) • [Privacy](https://zerogpu.ai/privacy-policy)


# gpt-5.6-luna
Source: https://docs.zerogpu.ai/api-reference/models/gpt-5-6-luna

api-reference/openapi/playgrounds/gpt-5_6-luna.openapi.json POST /responses
Model details for gpt-5.6-luna. Cost-optimized generation with adjustable reasoning effort and a 272K-token context window.

> OpenAI's GPT-5.6 Luna is the cost-optimized model of the GPT-5.6 family, served on ZeroGPU for cost-sensitive, high-volume workloads. It supports adjustable reasoning effort, function calling, and structured outputs across a 272,000-token context window.

**References:** [Terms](https://openai.com/policies/services-agreement/) • [Privacy](https://zerogpu.ai/privacy-policy)


# gpt-oss-120b
Source: https://docs.zerogpu.ai/api-reference/models/gpt-oss-120b

api-reference/openapi/playgrounds/gpt-oss-120b.openapi.json POST /responses
Model details for gpt-oss-120b. Function calling, batch tasks, and reasoning with a 131K context window.

> OpenAI's gpt-oss-120b is an open-weight Mixture-of-Experts model with 117B total parameters (5.1B active per token), served on ZeroGPU for general text generation. It reasons through a problem before answering and returns that reasoning trace alongside the final answer, and it supports function calling, batch tasks, and a 131,072-token context window. When a task needs frontier-level reasoning or tool use beyond what the smaller edge models cover, this is the model.

**References:** [Model docs](https://huggingface.co/openai/gpt-oss-120b) • [Terms](https://zerogpu.ai/terms) • [Privacy](https://zerogpu.ai/privacy-policy)


# LFM2.5-1.2B-Instruct
Source: https://docs.zerogpu.ai/api-reference/models/lfm2-5-1-2b-instruct

api-reference/openapi/playgrounds/LFM2_5-1_2B-Instruct.openapi.json POST /responses
Model details for LFM2.5-1.2B-Instruct.

> Liquid AI's LFM2.5-1.2B-Instruct is a hybrid architecture model purpose-built for on-device deployment, trained on 28 trillion tokens with multi-stage reinforcement learning. It delivers best-in-class instruction following and native tool-calling at the 1B scale — something very few models this size can do reliably. Its hybrid design gives it significantly faster inference with lower memory usage compared to transformer-only models of the same size. If you need a conversational model that can actually run on real devices without a GPU, this is it. Supports 8 languages including English, Chinese, Japanese, and Spanish.

**References:** [Model docs](https://docs.liquid.ai/deployment/on-device/android/ai-agent-usage-guide#text-models) • [Terms](https://www.liquid.ai/terms-conditions) • [Privacy](https://www.liquid.ai/lfm-license)


# LFM2.5-1.2B-Thinking
Source: https://docs.zerogpu.ai/api-reference/models/lfm2-5-1-2b-thinking

api-reference/openapi/playgrounds/LFM2_5-1_2B-Thinking.openapi.json POST /responses
Model details for LFM2.5-1.2B-Thinking.

> Liquid AI's LFM2.5-1.2B-Thinking is a compact reasoning model that works through a problem step by step before it answers. Built by Liquid AI, it generates an explicit chain-of-thought trace so for multi-step math, planning, data extraction, and agentic tool-calling, you get verifiable intermediate reasoning, not just a final guess. At 1.2B parameters it fits under a gigabyte and runs fast on CPU at the edge, rivaling models several times its size on reasoning benchmarks. Reach for a frontier model when the task is knowledge-heavy or code-intensive, and keep this one as the planning brain that decides and orchestrates. When the task needs reasoning, this is the model.

**References:** [Model docs](https://docs.liquid.ai/deployment/on-device/android/ai-agent-usage-guide#text-models) • [Terms](https://www.liquid.ai/terms-conditions) • [Privacy](https://www.liquid.ai/lfm-license)


# llama-3.1-8b-instruct-fast
Source: https://docs.zerogpu.ai/api-reference/models/llama-3-1-8b-instruct-fast

api-reference/openapi/playgrounds/llama-3_1-8b-instruct-fast.openapi.json POST /chat/completions
Model details for llama-3.1-8b-instruct-fast.

> Meta's Llama 3.1 Instruct, tuned for fast, low-cost summarization at scale on the ZeroGPU edge network. Its 128K-token context window takes in entire documents, long transcripts, and full email or chat threads in a single pass. No chunking, no stitching partial summaries back together. Built for the high-volume summarization that runs around your core product: condensing long reports and transcripts, summarizing email and chat threads, and processing articles in bulk. When context length matters and you're summarizing on every request, this is the model that keeps it fast and affordable.

**References:** [Model docs](https://github.com/meta-llama/llama-models/tree/main/models/llama3_1) • [Terms](https://github.com/meta-llama/llama-models/blob/main/models/llama3_1/LICENSE) • [Privacy](https://github.com/meta-llama/llama-models/blob/main/models/llama3_1/USE_POLICY.md)


# llama-guard-4-12b
Source: https://docs.zerogpu.ai/api-reference/models/llama-guard-4-12b

api-reference/openapi/playgrounds/llama-guard-4-12b.openapi.json POST /moderations
Model details for llama-guard-4-12b. Multimodal safety classification for prompts and responses.

> Meta's Llama Guard 4 12B is a multimodal safety classification model for moderating text, images, and mixed text-image inputs. It evaluates both incoming prompts and generated responses, returning a safe or unsafe classification along with the policy categories a violation falls under. It handles multilingual text and multiple images in one call, which makes it a fit for chat moderation, prompt and response filtering, policy enforcement, and agent guardrails. Built on a dense 12B architecture derived from Llama 4 Scout, it is designed as a dedicated safety layer in front of a production model rather than as a general-purpose generator.

**References:** [Model docs](https://huggingface.co/meta-llama/Llama-Guard-4-12B) • [Terms](https://huggingface.co/meta-llama/Llama-Guard-4-12B) • [Privacy](https://huggingface.co/meta-llama/Llama-Guard-4-12B)


# qwen3-30b-a3b-fp8
Source: https://docs.zerogpu.ai/api-reference/models/qwen3-30b-a3b-fp8

api-reference/openapi/playgrounds/qwen3-30b-a3b-fp8.openapi.json POST /chat/completions
Model details for qwen3-30b-a3b-fp8. Multilingual, streaming, batch tasks, reasoning, and function calling with a 32K context window.

<Note>
  This model supports the Chat Completions API only. Send requests to
  `/v1/chat/completions` — the Responses endpoint (`/v1/responses`) is not
  available for this model.
</Note>

> Alibaba's Qwen3-30B-A3B is an open-weight Mixture-of-Experts model with 30.5B total parameters (3.3B active per token), served on ZeroGPU as an FP8 build for efficient inference. It thinks through a problem before answering and returns that reasoning alongside the final text, and it supports function calling, streaming, batch tasks, and 100+ languages with a 32,768-token context window. When you want frontier-style reasoning and tool use in a lighter, multilingual package, this is the model.

**References:** [Model docs](https://huggingface.co/Qwen/Qwen3-30B-A3B-FP8) • [Terms](https://zerogpu.ai/terms) • [Privacy](https://zerogpu.ai/privacy-policy)


# qwen3.6-35b-a3b
Source: https://docs.zerogpu.ai/api-reference/models/qwen3-6-35b-a3b

api-reference/openapi/playgrounds/qwen3_6-35b-a3b.openapi.json POST /responses
Model details for qwen3.6-35b-a3b. Fast open-weight multimodal generation across text, images, and video.

> Alibaba's Qwen3.6-35B-A3B is a fast, open-weight multimodal model for applications that need to understand text, images, and video, served on ZeroGPU as an FP8 build. It suits visual question answering, document and screenshot analysis, video understanding, content extraction, agent workflows, and tool calling, and it supports structured output. Its Mixture-of-Experts architecture keeps inference efficient while it works through large amounts of multimodal content. When the input is images or video rather than text alone, this is the model.

**References:** [Model docs](https://huggingface.co/Qwen/Qwen3.6-35B-A3B) • [Terms](https://huggingface.co/Qwen/Qwen3.6-35B-A3B) • [Privacy](https://huggingface.co/Qwen/Qwen3.6-35B-A3B)


# whisper-tiny
Source: https://docs.zerogpu.ai/api-reference/models/whisper-tiny

api-reference/openapi/playgrounds/whisper-tiny.openapi.json POST /audio/transcriptions
Model details for whisper-tiny. Multilingual speech-to-text.

<Note>
  This model is routable only on `/v1/audio/transcriptions`, and on
  `/v1/audio/translations` to translate speech into English.
</Note>

> OpenAI's Whisper Tiny is the smallest model in the Whisper family of speech recognition models, at 39M parameters. It is the multilingual checkpoint: it transcribes speech in the language it was spoken, detecting that language on its own when you don't name it, and it can translate speech into English. Its size makes it fast and cheap enough for high-volume transcription where throughput and cost matter more than the accuracy of the larger Whisper models: voice agents, meeting notes, subtitles, podcasts, and audio indexing.

**References:** [Model card](https://huggingface.co/openai/whisper-tiny) • [License](https://github.com/openai/whisper/blob/main/LICENSE) • [Terms](https://zerogpu.ai/terms) • [Privacy](https://zerogpu.ai/privacy-policy)

## Limits

| Limit | Value |
| - | - |
| File size | 25 MB |
| Audio duration | 10 minutes |
| Formats | flac, m4a, mp3, mp4, mpeg, mpga, oga, ogg, wav, webm |

<RequestExample>
  ```bash cURL theme={null}
  curl --location 'https://api.zerogpu.ai/v1/audio/transcriptions' \
    --header 'x-api-key: YOUR_API_KEY' \
    --form 'model="whisper-tiny"' \
    --form 'file=@"/path/to/audio.mp3"'
  ```

  ```python Python theme={null}
  from openai import OpenAI

  client = OpenAI(
      base_url="https://api.zerogpu.ai/v1",
      api_key="YOUR_API_KEY",  # sent as Authorization: Bearer
  )

  with open("/path/to/audio.mp3", "rb") as audio:
      transcript = client.audio.transcriptions.create(
          model="whisper-tiny",
          file=audio,
      )

  print(transcript.text)
  ```

  ```javascript JavaScript theme={null}
  import fs from "node:fs";
  import OpenAI from "openai";

  const client = new OpenAI({
    baseURL: "https://api.zerogpu.ai/v1",
    apiKey: "YOUR_API_KEY", // sent as Authorization: Bearer
  });

  const transcript = await client.audio.transcriptions.create({
    model: "whisper-tiny",
    file: fs.createReadStream("/path/to/audio.mp3"),
  });

  console.log(transcript.text);
  ```
</RequestExample>

<ResponseExample>
  ```json json theme={null}
  {
    "text": "the quick brown fox jumps over the lazy dog."
  }
  ```

  ```json verbose_json theme={null}
  {
    "task": "transcribe",
    "language": "english",
    "duration": 1.76,
    "text": "Hello from Zero GPU.",
    "segments": [
      {
        "id": 0,
        "seek": 0,
        "start": 0.0,
        "end": 2.0,
        "text": " Hello from Zero GPU.",
        "tokens": [50364, 2425, 490, 17182, 18407, 13, 50464],
        "temperature": 0.0,
        "avg_logprob": -0.797803,
        "compression_ratio": 0.724138,
        "no_speech_prob": 0.023181
      }
    ]
  }
  ```
</ResponseExample>


# zlm-v1-iab-classify-edge
Source: https://docs.zerogpu.ai/api-reference/models/zlm-v1-iab-classify-edge

api-reference/openapi/playgrounds/zlm-v1-iab-classify-edge.openapi.json POST /responses
Model details for zlm-v1-iab-classify-edge. Supports 50+ languages.

<Note>
  The `instructions` field is not needed for this model. It maps your input
  directly to the IAB Content Taxonomy, so you only need to send input
</Note>

> ZeroGPU's IAB classifier maps any text to the industry-standard IAB Content Taxonomy in a single, fast inference call. Each call returns categories across both the 1.0 and 2.2 taxonomies plus matched audience segments, every result scored for confidence. With support for 50+ languages, it classifies multilingual content without a translation step. At 90M parameters on ONNX, it's built for the high-volume, sub-millisecond classification that ad tech and content platforms demand, right at the edge. When you need clean category and audience signals on every request, this is the model — and when you need the full profile, reach for the enriched variant.

## System One

It also answers [System One](/api-reference/systemone) decisions on `/v1/systemone`: ask which IAB category best matches (`choice`), whether content belongs to a category (`noul`), or how strongly it matches one (`score`).

**References:** [Model docs](https://docs.zerogpu.ai/) • [Terms](https://zerogpu.ai/terms) • [Privacy](https://zerogpu.ai/privacy-policy)


# zlm-v1-iab-domain-classifier
Source: https://docs.zerogpu.ai/api-reference/models/zlm-v1-iab-domain-classifier

api-reference/openapi/playgrounds/zlm-v1-iab-domain-classifier.openapi.json POST /responses
Model details for zlm-v1-iab-domain-classifier.

<Note>
  The `instructions` field is not needed for this model. It maps a raw domain
  name directly to the IAB Content Taxonomy, so you only need to send the domain
  as input
</Note>

> ZeroGPU's Domain IAB Classifier maps a raw domain name straight to the IAB Content Taxonomy in a single, fast inference call, returning content categories, topics, keywords, and user-intent signals. Because it needs only the domain as input, it reduces payload size by up to 10x compared to page-level classification workflows while still surfacing the contextual signals that bidstream enrichment, contextual targeting, and domain-level intelligence pipelines run on. At 149M parameters on ONNX, it's built for the high-volume, sub-millisecond classification that ad tech and content platforms demand, right at the edge.

## System One

It also answers [System One](/api-reference/systemone) decisions on `/v1/systemone`, with the domain or URL as `state`: ask which category best describes the domain (`choice`), whether it belongs to a category (`noul`), or how strongly it matches one (`score`).

**References:** [Model docs](https://docs.zerogpu.ai/) • [Terms](https://zerogpu.ai/terms) • [Privacy](https://zerogpu.ai/privacy-policy)


# zlm-v1-moderation-edge
Source: https://docs.zerogpu.ai/api-reference/models/zlm-v1-moderation-edge

api-reference/openapi/playgrounds/zlm-v1-moderation-edge.openapi.json POST /moderations
Model details for zlm-v1-moderation-edge. OpenAI-compatible moderation, benchmarked against omni-moderation-latest.

<Note>
  This model is routable on `/v1/moderations` and on `/v1/systemone` (see
  [System One](/api-reference/systemone)). Send the text to classify as
  `input` — a string, an array of strings, or an array of
  `{"type": "text", "text": "…"}` content parts. The response is OpenAI's native
  moderations envelope, `{ id, model, results[] }`, with each result carrying a
  `flagged` verdict, per-category `categories` booleans, and calibrated
  `category_scores` — so it drops into any pipeline written against
  `omni-moderation-latest`.
</Note>

> ZeroGPU's moderation model screens text for unsafe, harmful, or policy-sensitive content and returns the complete OpenAI 13-category taxonomy (a `flagged` verdict, per-category booleans, and calibrated `category_scores`), so it drops into any pipeline written against `omni-moderation-latest`. Under the hood it's an 86M-parameter DeBERTa encoder with a shared trunk feeding one binary safe/unsafe head and 13 category heads, with per-category thresholds calibrated on held-out validation data. In [head-to-head benchmarks](https://zerogpu.ai/benchmarks/moderation-edge) against OpenAI omni-moderation it wins the binary safe/unsafe decision (0.899 vs 0.853 F1) and 9 of 13 harm categories, with the largest gains on graphic violence, illicit content, and self-harm, while returning verdicts 1.2–1.8× faster at the median on production-range inputs, because inference is co-located at the edge instead of a round trip to a central API. Moderation sits inline in front of every response your app serves; this is the model that's fast and accurate enough to live there.

## System One

It also answers [System One](/api-reference/systemone) decisions on `/v1/systemone`: ask whether content is unsafe or brand safe (`noul`), which moderation category applies (`choice`), or how severe a violation is (`score`).

**References:** [Moderation benchmark](https://zerogpu.ai/benchmarks/moderation-edge) • [Terms](https://zerogpu.ai/terms) • [Privacy](https://zerogpu.ai/privacy-policy)


# zlm-v1-signal-extract
Source: https://docs.zerogpu.ai/api-reference/models/zlm-v1-signal-extract

api-reference/openapi/playgrounds/zlm-v1-signal-extract.openapi.json POST /responses
Model details for zlm-v1-signal-extract. Topics, keywords, and intent extracted from text in a single call.

> ZeroGPU's signal extractor turns unstructured text into structured signals that downstream systems can act on. One inference call returns topics, keywords, intent, and other contextual attributes as structured output, so content enrichment, contextual intelligence, ad targeting, agent routing, recommendation systems, and analytics pipelines all read from the same pass. At 80M parameters it is built for high-volume workloads where a general-purpose LLM is more machinery than the job needs.

**References:** [Terms](https://zerogpu.ai/terms) • [Privacy](https://zerogpu.ai/privacy-policy)


# zlm-v2-iab-classify-edge-enriched
Source: https://docs.zerogpu.ai/api-reference/models/zlm-v2-iab-classify-edge-enriched

api-reference/openapi/playgrounds/zlm-v2-iab-classify-edge-enriched.openapi.json POST /responses
Model details for zlm-v2-iab-classify-edge-enriched. Supports 50+ languages.

<Note>
  The `instructions` field is not needed for this model. It maps your input
  directly to the IAB Content Taxonomy, so you only need to send input
</Note>

> The enriched variant of ZeroGPU's IAB classifier turns a single inference call into a full content-intelligence profile not just a label, but everything a contextual pipeline needs to act on. Each call returns IAB categories across both the 1.0 and 2.2 taxonomies (down to tier-3), audience and interest segments, topics, keywords, and user-intent classification — all with confidence scores with edge-native, sub-millisecond speeds. With support for 50+ languages, it delivers the same full profile on multilingual content without a translation step. Built for contextual ad targeting, brand-safety scoring, publisher content categorization, and signal pipelines that need full metadata on every request.

## System One

It also answers [System One](/api-reference/systemone) decisions on `/v1/systemone`, including several mixed questions in one request: category, intent and contextual classification (`choice`), brand safety and suitability (`noul`), and commercial intent or category strength (`score`).

**References:** [Model docs](https://docs.zerogpu.ai/) • [Terms](https://zerogpu.ai/terms) • [Privacy](https://zerogpu.ai/privacy-policy)


# Moderations
Source: https://docs.zerogpu.ai/api-reference/moderations

api-reference/openapi/zerogpu.openapi.json POST /moderations
Classify text against OpenAI's safety categories.

Send text as `input` and get back OpenAI's moderations envelope: an `id`, the resolved `model`, and a `results` array with per-category booleans (`categories`) and confidence scores (`category_scores`) across all 13 safety categories. It is drop-in compatible with OpenAI's Moderations API — OpenAI model ids (`omni-moderation-latest`, `text-moderation-stable`) are accepted and mapped. See the [Moderation](/docs/moderation) guide for a prefilled example, or the [model page](/api-reference/models/zlm-v1-moderation-edge) for an interactive playground.

Moderation models are routable **only** on this endpoint — a `/responses` or `/chat/completions` call with a moderation model returns `400`. Pass `input` as a string, an array of strings (one result per element), or an array of `{ "type": "text", "text": "…" }` content parts forming one multi-modal input.

Install the official SDK from [npm](https://www.npmjs.com/package/zerogpu-api) or [PyPI](https://pypi.org/project/zerogpu-api/) (`pip install zerogpu-api`). Source: [zerogpu/SDK](https://github.com/zerogpu/SDK). Handle errors the same way as [API error codes](/docs/production-patterns#handle-status-codes-explicitly).


# Responses
Source: https://docs.zerogpu.ai/api-reference/responses

api-reference/openapi/zerogpu.openapi.json POST /responses
Send input to an AI model and receive a response.

One endpoint for every model. Pass text or a document as `input`; what comes back depends on the model: generated text, a classification, extracted fields, redacted output, and more. The `input` and `output` shapes vary by model, so open a [model page](/api-reference/models) for a prefilled playground. For the `messages`-style route, see [Chat completions](/api-reference/chat-completions).

Install the official SDK from [npm](https://www.npmjs.com/package/zerogpu-api) or [PyPI](https://pypi.org/project/zerogpu-api/) (`pip install zerogpu-api`). Source: [zerogpu/SDK](https://github.com/zerogpu/SDK). Handle errors the same way as [API error codes](/docs/production-patterns#handle-status-codes-explicitly).


# Retrieve batch
Source: https://docs.zerogpu.ai/api-reference/retrieve-batch

api-reference/openapi/batch.openapi.json GET /v1/batches/{batch_id}
Interactive playground for GET /v1/batches/{batch_id}.

Poll a batch by ID. Use this while `status` is `in_progress` or `cancelling`; stop when the status is terminal (`completed`, `failed`, `expired`, or `cancelled`).

When complete, download results with `output_file_id` and `error_file_id` via [Files API](/docs/batch/objects).


# Retrieve file
Source: https://docs.zerogpu.ai/api-reference/retrieve-file

api-reference/openapi/batch.openapi.json GET /v1/files/{file_id}
Interactive playground for GET /v1/files/{file_id}.

Look up metadata for one file (`filename`, `bytes`, `purpose`, `status`) without downloading the body. Use [Download file](/api-reference/download-file) for the JSONL content.

Full reference: [Files API](/docs/batch/objects).


# System One
Source: https://docs.zerogpu.ai/api-reference/systemone

api-reference/openapi/zerogpu.openapi.json POST /systemone
Answer yes/no, multiple-choice and rubric questions about any content with the ZLM classifiers.

Send the content as `state` and ask up to 16 named `questions` about it in one call. Each answer comes back under the question's name, with calibrated probabilities. The request and response follow the [TypeSafe System One](https://api.typesafe.ai/openapi.json) decision schema, so an existing System One client works by changing the base URL and API key.

System One is available on four models, in addition to their existing endpoints:

| Model | Also served on | Good for |
| - | - | - |
| [`zlm-v1-moderation-edge`](/api-reference/models/zlm-v1-moderation-edge) | `/moderations` | Is this unsafe? Which moderation category applies? How severe is it? |
| [`zlm-v1-iab-classify-edge`](/api-reference/models/zlm-v1-iab-classify-edge) | `/chat/completions`, `/responses` | Which IAB category matches? Does it belong to category X? How strongly? |
| [`zlm-v1-iab-domain-classifier`](/api-reference/models/zlm-v1-iab-domain-classifier) | `/chat/completions`, `/responses` | The same questions about a domain. `state` must be a URL or domain. |
| [`zlm-v2-iab-classify-edge-enriched`](/api-reference/models/zlm-v2-iab-classify-edge-enriched) | `/chat/completions`, `/responses` | Mixed category, brand-safety and commercial-intent questions in one request. |

## Question types

* **`noul`** - a yes/no decision. The answer's `noul` is the probability of yes: near 1 favors yes, near 0 favors no. Optional `criteria.true` and `criteria.false` describe what counts as each.
* **`choice`** - pick one option. `criteria` maps each choice name to a description. The answer gives the `choice`, its `confidence` and the `probabilities` of every option.
* **`score`** - rate against an ordered rubric. `criteria` is a list of levels, scored from 0. The answer's `score` is the probability-weighted level and may fall between integers; `legend` maps each level number back to its text.

## How answers are produced

Each model scores the content against its own labels: the 13 OpenAI moderation categories plus an overall unsafe score, or the IAB Content 2.2 and Audience 1.1 taxonomies. A question whose criteria name those labels is answered directly from the model's scores. A category name covers its whole branch, so `Technology and computing` also counts `Laptops`. Brand-safety questions on the IAB models read the Sensitive Topics branch, and commercial-intent questions read the Purchase Intent audiences.

A question whose criteria are not among the model's labels is answered by zero-shot classification instead. You can mix both kinds in one request.

## Limits and billing

* Up to 16 questions per request, and up to 32 criteria per question.
* `state` may be a string, an object or an array. An object or array is read as its JSON.
* You are billed for input tokens only (the state plus the questions); `output_tokens` is always 0.
* Request validation errors return `422` with System One's `{ "detail": [{ "loc", "msg", "type" }] }` body. Other errors use the standard [error format](/docs/production-patterns#handle-status-codes-explicitly).


# Upload file
Source: https://docs.zerogpu.ai/api-reference/upload-file

api-reference/openapi/batch.openapi.json POST /v1/files
Interactive playground for POST /v1/files (multipart JSONL upload).

Upload batch input JSONL before [creating a batch](/api-reference/create-batch).

1. Set `purpose` to `batch`.
2. Attach a `.jsonl` file (max 100 MB). Each line must match the [JSONL format](/docs/batch/jsonl-format).
3. Copy the returned `id` (e.g. `file-…`) into `input_file_id` on the create-batch playground.

Full reference: [Files API](/docs/batch/objects).


# Speech (ZeroClick)
Source: https://docs.zerogpu.ai/api-reference/zeroclick-audio-speech

api-reference/openapi/zeroclick.openapi.json POST /zeroclick/v1/audio/speech
Generate speech with chatterbox-nano, authenticated by a ZeroClick HMAC signature instead of an API key.

This is the [Speech](/api-reference/audio-speech) endpoint behind **ZeroClick**, ZeroGPU's agent-payment layer. Same request/response bodies, limits, and model as `/v1/audio/speech` — only authentication, authorization, and metering differ. The request is always served by the configured model (`chatterbox-nano`). Send a JSON body; voice cloning (a `voice_sample` upload) is not available through ZeroClick.

* **Authentication** — no `x-api-key`. A buying agent calls ZeroClick's hosted pay URL; ZeroClick proxies the request and signs the raw request bytes with an HMAC in the `zc-signature` header. A request is authenticated **only** by a valid signature.
* **Authorization** — a ZeroClick allowance check runs before any billable work. It fails *open* on a transient ZeroClick outage (the signature has already verified), but never for a missing or invalid signature.
* **Metering** — billed per audio second. The allowance check asks for a 1-second floor, since the duration is not known until the clip is decoded or generated. On success, `zc-usage` reports `max(1, ceil(x-audio-duration-seconds))` audio seconds. Stripe metering is bypassed; ZeroClick is the sole biller.

Handle errors the same way as [API error codes](/docs/production-patterns#handle-status-codes-explicitly).


# Transcriptions (ZeroClick)
Source: https://docs.zerogpu.ai/api-reference/zeroclick-audio-transcriptions

api-reference/openapi/zeroclick.openapi.json POST /zeroclick/v1/audio/transcriptions
Transcribe audio with whisper-tiny, authenticated by a ZeroClick HMAC signature instead of an API key.

This is the [Transcriptions](/api-reference/audio-transcriptions) endpoint behind **ZeroClick**, ZeroGPU's agent-payment layer. Same response bodies, limits, and model as `/v1/audio/transcriptions`; the request is JSON, and authentication, authorization, and metering differ. The request is always served by the configured model (`whisper-tiny`).

* **Body** — ZeroClick's pay host forwards JSON only, so send `application/json` with the audio as base64 in `file` (or a `data:` URL, within ZeroClick's \~7.5 MB body cap) or as an https URL in `file_url` (up to 25 MB), plus the usual fields. A JSON body that cannot be converted (bad base64, unreachable URL, neither or both of `file` / `file_url`) gets a `400` or `413` before any inference and is not billed.
* **Authentication** — no `x-api-key`. A buying agent calls ZeroClick's hosted pay URL; ZeroClick proxies the request and signs the raw request bytes with an HMAC in the `zc-signature` header. A request is authenticated **only** by a valid signature.
* **Authorization** — a ZeroClick allowance check runs before any billable work. It fails *open* on a transient ZeroClick outage (the signature has already verified), but never for a missing or invalid signature.
* **Metering** — billed per audio second. The allowance check asks for a 1-second floor, since the duration is not known until the clip is decoded or generated. On success, `zc-usage` reports `max(1, ceil(x-audio-duration-seconds))` audio seconds. Stripe metering is bypassed; ZeroClick is the sole biller.

```json theme={null}
{
  "file_url": "https://example.com/audio/hello.mp3",
  "model": "whisper-tiny",
  "response_format": "json"
}
```

Handle errors the same way as [API error codes](/docs/production-patterns#handle-status-codes-explicitly).


# Translations (ZeroClick)
Source: https://docs.zerogpu.ai/api-reference/zeroclick-audio-translations

api-reference/openapi/zeroclick.openapi.json POST /zeroclick/v1/audio/translations
Translate speech into English text with whisper-tiny, authenticated by a ZeroClick HMAC signature instead of an API key.

This is the audio translations endpoint (see [Transcriptions](/api-reference/audio-transcriptions)) behind **ZeroClick**, ZeroGPU's agent-payment layer. Same response bodies, limits, and model as `/v1/audio/translations`; the request is JSON, and authentication, authorization, and metering differ. The request is always served by the configured model (`whisper-tiny`). Send the same fields as a transcription without `language`; the transcript is always English.

* **Body** — ZeroClick's pay host forwards JSON only, so send `application/json` with the audio as base64 in `file` (or a `data:` URL, within ZeroClick's \~7.5 MB body cap) or as an https URL in `file_url` (up to 25 MB), plus the usual fields. A JSON body that cannot be converted (bad base64, unreachable URL, neither or both of `file` / `file_url`) gets a `400` or `413` before any inference and is not billed.
* **Authentication** — no `x-api-key`. A buying agent calls ZeroClick's hosted pay URL; ZeroClick proxies the request and signs the raw request bytes with an HMAC in the `zc-signature` header. A request is authenticated **only** by a valid signature.
* **Authorization** — a ZeroClick allowance check runs before any billable work. It fails *open* on a transient ZeroClick outage (the signature has already verified), but never for a missing or invalid signature.
* **Metering** — billed per audio second. The allowance check asks for a 1-second floor, since the duration is not known until the clip is decoded or generated. On success, `zc-usage` reports `max(1, ceil(x-audio-duration-seconds))` audio seconds. Stripe metering is bypassed; ZeroClick is the sole biller.

```json theme={null}
{
  "file_url": "https://example.com/audio/hello.mp3",
  "model": "whisper-tiny",
  "response_format": "json"
}
```

Handle errors the same way as [API error codes](/docs/production-patterns#handle-status-codes-explicitly).


# Chat completions (ZeroClick)
Source: https://docs.zerogpu.ai/api-reference/zeroclick-chat-completions

api-reference/openapi/zeroclick.openapi.json POST /zeroclick/v1/chat/completions
OpenAI-style chat inference authenticated by a ZeroClick HMAC signature instead of an API key.

This is the [Chat completions](/api-reference/chat-completions) endpoint behind **ZeroClick**, ZeroGPU's agent-payment layer. Same request/response bodies and same model routing as `/v1/chat/completions`, except for retired model ids (below) — authentication, authorization, and metering differ.

* **Authentication** — no `x-api-key`. A buying agent calls ZeroClick's hosted pay URL; ZeroClick proxies the request and signs the raw request bytes with an HMAC in the `zc-signature` header. A request is authenticated **only** by a valid signature.
* **Authorization** — a ZeroClick allowance check runs before any billable work. It fails *open* on a transient ZeroClick outage (the signature has already verified), but never for a missing or invalid signature.
* **Retired model ids** — `deepseek-v4-flash`, `glm-5.2` and `gliner2-base-v1` are served and metered as their successors (`deepseek-v4.1-flash`, `glm-5.3-flash`, `gliner2.5-base-v1`); the response's `model` is the successor. Send the current id.
* **Metering** — actual token usage is settled with ZeroClick and reported via the `zc-usage` response header. Stripe metering is bypassed; ZeroClick is the sole biller.

Response JSON shape depends on the model; handle errors the same way as [API error codes](/docs/production-patterns#handle-status-codes-explicitly).


# Embeddings (ZeroClick)
Source: https://docs.zerogpu.ai/api-reference/zeroclick-embeddings

api-reference/openapi/zeroclick.openapi.json POST /zeroclick/v1/embeddings
OpenAI-style embeddings authenticated by a ZeroClick HMAC signature instead of an API key.

This is the [Embeddings](/api-reference/embeddings) endpoint behind **ZeroClick**, ZeroGPU's agent-payment layer. Same request/response bodies and same model routing as `/v1/embeddings` — only authentication, authorization, and metering differ.

* **Authentication** — no `x-api-key`. A buying agent calls ZeroClick's hosted pay URL; ZeroClick proxies the request and signs the raw request bytes with an HMAC in the `zc-signature` header. A request is authenticated **only** by a valid signature.
* **Authorization** — a ZeroClick allowance check runs before any billable work. It fails *open* on a transient ZeroClick outage (the signature has already verified), but never for a missing or invalid signature.
* **Metering** — billed on input tokens only. The `prompt_tokens` from the response's `usage` are settled with ZeroClick and reported via the `zc-usage` response header. Stripe metering is bypassed; ZeroClick is the sole biller.

Handle errors the same way as [API error codes](/docs/production-patterns#handle-status-codes-explicitly).


# Moderations (ZeroClick)
Source: https://docs.zerogpu.ai/api-reference/zeroclick-moderations

api-reference/openapi/zeroclick.openapi.json POST /zeroclick/v1/moderations
OpenAI-style content moderation authenticated by a ZeroClick HMAC signature instead of an API key.

This is the [Moderations](/api-reference/moderations) endpoint behind **ZeroClick**, ZeroGPU's agent-payment layer. Same request/response bodies and same model routing as `/v1/moderations` — only authentication, authorization, and metering differ. `model` may be omitted, or given as an OpenAI id such as `omni-moderation-latest`; either way the configured moderation model serves the request.

* **Authentication** — no `x-api-key`. A buying agent calls ZeroClick's hosted pay URL; ZeroClick proxies the request and signs the raw request bytes with an HMAC in the `zc-signature` header. A request is authenticated **only** by a valid signature.
* **Authorization** — a ZeroClick allowance check runs before any billable work. It fails *open* on a transient ZeroClick outage (the signature has already verified), but never for a missing or invalid signature.
* **Metering** — the input and output tokens from the response's `usage` are settled with ZeroClick and reported via the `zc-usage` response header. Stripe metering is bypassed; ZeroClick is the sole biller.

Handle errors the same way as [API error codes](/docs/production-patterns#handle-status-codes-explicitly).


# Responses (ZeroClick)
Source: https://docs.zerogpu.ai/api-reference/zeroclick-responses

api-reference/openapi/zeroclick.openapi.json POST /zeroclick/v1/responses
OpenAI-style Responses inference authenticated by a ZeroClick HMAC signature instead of an API key.

This is the [Responses](/api-reference/responses) endpoint behind **ZeroClick**, ZeroGPU's agent-payment layer. Same request/response bodies and same model routing as `/v1/responses`, except for retired model ids (below) — authentication, authorization, and metering differ.

* **Authentication** — no `x-api-key`. A buying agent calls ZeroClick's hosted pay URL; ZeroClick proxies the request and signs the raw request bytes with an HMAC in the `zc-signature` header. A request is authenticated **only** by a valid signature.
* **Authorization** — a ZeroClick allowance check runs before any billable work. It fails *open* on a transient ZeroClick outage (the signature has already verified), but never for a missing or invalid signature.
* **Retired model ids** — `deepseek-v4-flash`, `glm-5.2` and `gliner2-base-v1` are served and metered as their successors (`deepseek-v4.1-flash`, `glm-5.3-flash`, `gliner2.5-base-v1`); the response's `model` is the successor. Send the current id.
* **Metering** — actual token usage is settled with ZeroClick and reported via the `zc-usage` response header. Stripe metering is bypassed; ZeroClick is the sole biller.

The `input` and `output` shapes vary by model; open a [model page](/api-reference/models) for a prefilled playground. Handle errors the same way as [API error codes](/docs/production-patterns#handle-status-codes-explicitly).


# 🏷️ Tag Customer Reviews Overnight with the ZeroGPU Batch API
Source: https://docs.zerogpu.ai/cookbook/batch-review-tagging

Tag a CSV of customer reviews with sentiment and topics in one overnight job using the ZeroGPU Batch API and LFM2.5-1.2B-Instruct, then recover any failed rows by custom_id.

This notebook shows how to use the ZeroGPU Batch API to tag a CSV of customer reviews in a single asynchronous job. You hand the API a `reviews.csv` with one review per row, and you get back a `tagged.csv` with a `sentiment` label and a short list of `topics` for every row, plus a recoverable list of any rows that failed. By combining the Batch API's OpenAI-compatible JSONL flow with LFM2.5-1.2B-Instruct, this notebook walks you through a practical pattern where thousands of rows are tagged overnight at a fraction of the cost of synchronous calls, keyed back to their source rows by `custom_id`.

For the full reference, see the [Batch API quickstart](/docs/batch/getting-started).

For another end-to-end workflow, see:
**[Screen resumes with LangChain and ZeroGPU](https://zerogpu.ai/cookbook/langchain-resume-screening)**.

In this notebook, you'll explore:

* **ZeroGPU Batch API**: An asynchronous, OpenAI-compatible endpoint that takes thousands of `/v1/chat/completions` requests as a single JSONL file and returns the results within a completion window, at a lower per-request cost than synchronous calls. Here it tags every row of a customer-review CSV in one overnight job.
* **ZeroGPU**: An ultra-fast, compute-efficient inference provider for apps and agents. We run purpose-built small and nano language models across an edge-powered network for the high-volume, purpose-specific tasks your app or agent runs constantly. Plug in our OpenAI-compatible API and you're live - zero GPU infrastructure, serverless, auto-scaling by default.
* **LFM2.5-1.2B-Instruct**: A small, fast instruct model that handles short-form classification well, keeping per-row cost low while still producing open-vocabulary topics and a JSON tag. See the [model card](/api-reference/models/lfm2-5-1-2b-instruct) for context window and playground.

This setup not only demonstrates a practical application of large-scale review tagging, but also provides a flexible framework that can be adapted to other real-world scenarios requiring high-volume, asynchronous classification over tabular data.

## Run this example

Run it in Google Colab and execute cells top to bottom — no setup required.

**[🏷️ Run in Google Colab →](https://colab.research.google.com/drive/18TW62E7m5goXEnNUglRFd-Pd47cFIdyJ?usp=sharing)**

The notebook generates the dataset, builds the JSONL file, and runs the full Batch API workflow automatically.

## 🎥 Watch the Video Guide

Prefer a quick walkthrough? Watch the full demo here:

<iframe title="YouTube video player" />

## 📦 Installation

First, install `requests`, the only dependency for driving the Batch API from Python. You reach ZeroGPU through its OpenAI-compatible REST surface, so no SDK is required:

```bash theme={null}
!pip install requests
```

The cURL examples below also use `jq` to pretty-print JSON responses. For the full lifecycle, see the [Batch API quickstart](/docs/batch/getting-started).

## 🔑 Setting Up API Keys

You'll need to set up your ZeroGPU API key so that every Batch API call authenticates. This ensures the upload, create, poll, and download calls can reach ZeroGPU securely. A Project ID is optional: supply one only if you want to scope the batch to a specific project.

You can go to [here](https://platform.zerogpu.ai/dashboard) to get an API key and Project ID from ZeroGPU. The key starts with `zgpu-api-` and the Project ID (UUID) is on the project settings page.

```python Python theme={null}
import os
from getpass import getpass

# Prompt for the ZeroGPU API key securely
zerogpu_api_key = getpass('Enter your ZeroGPU API key: ')
os.environ["ZEROGPU_API_KEY"] = zerogpu_api_key

# Optional: press Enter to skip and run unscoped
zerogpu_project_id = getpass('Enter your ZeroGPU Project ID (optional): ')
if zerogpu_project_id:
    os.environ["ZEROGPU_PROJECT_ID"] = zerogpu_project_id
```

This cookbook calls the raw REST surface with the `x-api-key` and `x-project-id` headers (`x-project-id` is optional - it scopes requests to a project). If you use an SDK client, authentication is handled for you. The cURL examples read the same values from your shell environment, and you can leave `ZEROGPU_PROJECT_ID` unset:

```bash theme={null}
export ZEROGPU_API_KEY="zgpu-api-..."
export ZEROGPU_PROJECT_ID="your-project-uuid"   # optional
```

```python Python theme={null}
import os, requests

headers = {
    "x-api-key":    os.environ["ZEROGPU_API_KEY"],
    "content-type": "application/json",
}
# x-project-id is optional; send it only when scoping to a project
if os.environ.get("ZEROGPU_PROJECT_ID"):
    headers["x-project-id"] = os.environ["ZEROGPU_PROJECT_ID"]

resp = requests.post(
    "https://api.zerogpu.ai/v1/chat/completions",
    headers=headers,
    json={
        "model":       "LFM2.5-1.2B-Instruct",
        "temperature": 0,
        "messages": [
            {"role": "system", "content": (
                "You are a review tagger. Respond with a single JSON object and "
                'nothing else, in this shape: { "sentiment": "...", "topics": ["..."] }. '
                'sentiment is one of: "positive", "neutral", "negative".'
            )},
            {"role": "user", "content": "Battery lasts two days, love it."},
        ],
    },
).json()

print(resp["choices"][0]["message"]["content"])
```

```
{"sentiment": "positive", "topics": ["battery"]}
```

For a fixed, short taxonomy, a classifier head like [deberta-v3-small](/api-reference/models/deberta-v3-small) for sentiment or [zlm-v1-iab-classify-edge](/api-reference/models/zlm-v1-iab-classify-edge) for IAB topics is cheaper and slots into the same batch flow. Reach for LFM only when you need open-vocabulary topics or a free-form rationale.

🎉 **ZeroGPU tags a review in a single call, returning a clean JSON object you can parse straight into a row, the building block for tagging an entire CSV at once!**

## 🌙 Tag a CSV of Reviews Overnight

This section takes a CSV of customer reviews and produces a tagged copy plus a recoverable list of failures, with the Batch API running every row as one asynchronous job while you sleep.

Your support tool exports `reviews.csv`, one review per row. You want a `sentiment` label and a short list of `topics` for every row, but calling the model synchronously across thousands of rows is slow and costly. Instead you submit them as a single batch, poll until it finishes, and merge the results back by `custom_id`.

The job has five steps: prepare the data, build a JSONL request file, upload it, create the batch, then poll and download the results.

### Step 1: Prepare the input CSV

The dataset is generated directly in the Colab notebook, so you do not need to prepare any files manually.

Each row includes a unique `review_id` and a free-text `review`. One row is intentionally malformed (empty review) to demonstrate how invalid input is handled.

Keep the `review_id` column unique. It is what links a tagged row back to its source, and duplicates are rejected at create time.

### Step 2: Build the JSONL, one request per row

The notebook builds this JSONL file programmatically from the CSV, so you do not need to create it manually.

We pin the same JSON-only system prompt on every line:

```text theme={null}
You are a review tagger. For each customer review, respond with a single JSON
object and nothing else, in this exact shape:

{ "sentiment": "...", "topics": ["..."] }

sentiment is one of: "positive", "neutral", "negative".
topics is a list of 1-3 short lowercase strings (for example: "battery",
"shipping", "support", "build quality", "app", "price"). No prose, no code
fences, no extra keys.
```

A single JSONL line for `r-001` looks like this:

```json theme={null}
{
  "custom_id": "r-001",
  "method":    "POST",
  "url":       "/v1/chat/completions",
  "body": {
    "model":       "LFM2.5-1.2B-Instruct",
    "temperature": 0,
    "messages": [
      { "role": "system", "content": "You are a review tagger. ..." },
      { "role": "user",   "content": "Battery lasts two days, love it." }
    ]
  }
}
```

Row `r-007` has no review text. The builder skips it locally rather than spending a request on a line the API would reject, and records it as a local skip.

### Step 3: Upload the file and create the batch

Upload the JSONL with `purpose=batch`, then create the batch against `/v1/chat/completions`:

<CodeGroup>
  ```bash cURL theme={null}
  # 1. Upload the JSONL
  curl -X POST https://api.zerogpu.ai/v1/files \
    -H "x-api-key: $ZEROGPU_API_KEY" \
    -H "x-project-id: $ZEROGPU_PROJECT_ID" \
    -F purpose=batch \
    -F file=@input.jsonl
  # -> { "id": "file-abc123...", "status": "processed", ... }

  # 2. Create the batch
  curl -X POST https://api.zerogpu.ai/v1/batches \
    -H "x-api-key: $ZEROGPU_API_KEY" \
    -H "x-project-id: $ZEROGPU_PROJECT_ID" \
    -H "content-type: application/json" \
    -d '{
      "input_file_id":     "file-abc123...",
      "endpoint":          "/v1/chat/completions",
      "completion_window": "24h",
      "metadata":          { "job": "review-tagging-2026-06-08" }
    }'
  # -> { "id": "batch_01HZX...", "status": "in_progress", ... }
  ```

  ```python Python theme={null}
  import os, requests

  BASE    = "https://api.zerogpu.ai"
  HEADERS = {"x-api-key": os.environ["ZEROGPU_API_KEY"]}
  # x-project-id is optional; send it only when scoping to a project
  if os.environ.get("ZEROGPU_PROJECT_ID"):
      HEADERS["x-project-id"] = os.environ["ZEROGPU_PROJECT_ID"]

  # 1. Upload
  with open("input.jsonl", "rb") as f:
      up = requests.post(
          f"{BASE}/v1/files",
          headers=HEADERS,
          data={"purpose": "batch"},
          files={"file": ("input.jsonl", f, "application/jsonl")},
      ).json()
  file_id = up["id"]

  # 2. Create
  batch = requests.post(
      f"{BASE}/v1/batches",
      headers={**HEADERS, "content-type": "application/json"},
      json={
          "input_file_id":     file_id,
          "endpoint":          "/v1/chat/completions",
          "completion_window": "24h",
          "metadata":          {"job": "review-tagging-2026-06-08"},
      },
  ).json()
  batch_id = batch["id"]
  ```
</CodeGroup>

Validation runs at create time: the Batch API parses every line before `POST /v1/batches` returns, so a duplicate `custom_id`, a line over 1 MB, or `"stream": true` rejects the whole batch with a 400 that points at the offending line. Fix the JSONL locally and resubmit; nothing is charged for a rejected create.

### Step 4: Poll until the batch finishes

Poll `GET /v1/batches/{id}` until the batch reaches a terminal state. A batch ends in one of four: `completed`, `failed`, `expired`, or `cancelled`. Only `completed` guarantees an `output_file_id`, and even a `completed` batch can carry a populated `error_file_id` when some lines failed. Don't poll faster than every 30 seconds; the status only changes on minute-scale transitions.

```python Python theme={null}
import time

def poll(batch_id: str, interval: int = 30) -> dict:
    while True:
        resp = requests.get(f"{BASE}/v1/batches/{batch_id}", headers=HEADERS)
        resp.raise_for_status()
        batch = resp.json()
        counts = batch.get("request_counts", {})
        print(
            f"{batch['status']}: "
            f"{counts.get('completed', 0)}/{counts.get('total', 0)} done, "
            f"{counts.get('failed', 0)} failed"
        )
        if batch["status"] in ("completed", "failed", "expired", "cancelled"):
            return batch
        time.sleep(interval)
```

```
in_progress: 6/9 done, 0 failed
in_progress: 9/9 done, 0 failed
completed: 9/9 done, 0 failed
```

### Step 5: Download and merge into a tagged CSV

The full script below reads the CSV, builds the JSONL (skipping the empty row), uploads, creates, polls, downloads the output and error files, and merges everything back into `tagged.csv` by `custom_id`. Failed rows land in `failed.csv` for inspection.

````python tag_reviews.py theme={null}
import csv
import json
import os
import time

import requests

BASE    = "https://api.zerogpu.ai"
HEADERS = {"x-api-key": os.environ["ZEROGPU_API_KEY"]}
# x-project-id is optional; send it only when scoping to a project
if os.environ.get("ZEROGPU_PROJECT_ID"):
    HEADERS["x-project-id"] = os.environ["ZEROGPU_PROJECT_ID"]
MODEL = "LFM2.5-1.2B-Instruct"

SYSTEM_PROMPT = (
    "You are a review tagger. For each customer review, respond with a "
    "single JSON object and nothing else, in this exact shape:\n"
    '{ "sentiment": "...", "topics": ["..."] }\n'
    'sentiment is one of: "positive", "neutral", "negative". '
    "topics is a list of 1-3 short lowercase strings (for example: "
    '"battery", "shipping", "support", "build quality", "app", "price"). '
    "No prose, no code fences, no extra keys."
)


def build_jsonl(csv_path: str, jsonl_path: str) -> int:
    written = 0
    seen_ids: set[str] = set()

    with open(csv_path, encoding="utf-8") as src, \
         open(jsonl_path, "w", encoding="utf-8") as dst:

        reader = csv.DictReader(src)

        for row in reader:
            review_id = (row.get("review_id") or "").strip()

            if not review_id:
                print("skip: empty review_id")
                continue

            if review_id in seen_ids:
                print(f"skip {review_id}: duplicate review_id")
                continue

            seen_ids.add(review_id)

            review = (row.get("review") or "").strip()
            if not review:
                print(f"skip {review_id}: empty review text")
                continue

            line = {
                "custom_id": review_id,
                "method": "POST",
                "url": "/v1/chat/completions",
                "body": {
                    "model": MODEL,
                    "temperature": 0,
                    "messages": [
                        {"role": "system", "content": SYSTEM_PROMPT},
                        {"role": "user", "content": review},
                    ],
                },
            }

            dst.write(json.dumps(line) + "\n")
            written += 1

    return written


def upload(jsonl_path: str) -> str:
    with open(jsonl_path, "rb") as f:
        resp = requests.post(
            f"{BASE}/v1/files",
            headers=HEADERS,
            data={"purpose": "batch"},
            files={"file": (os.path.basename(jsonl_path), f, "application/jsonl")},
        )
    resp.raise_for_status()
    return resp.json()["id"]


def create_batch(file_id: str) -> dict:
    resp = requests.post(
        f"{BASE}/v1/batches",
        headers={**HEADERS, "content-type": "application/json"},
        json={
            "input_file_id":     file_id,
            "endpoint":          "/v1/chat/completions",
            "completion_window": "24h",
            "metadata":          {"job": "review-tagging"},
        },
    )
    resp.raise_for_status()
    return resp.json()


def poll(batch_id: str, interval: int = 30) -> dict:
    while True:
        resp = requests.get(f"{BASE}/v1/batches/{batch_id}", headers=HEADERS)
        resp.raise_for_status()
        batch = resp.json()
        counts = batch.get("request_counts", {})
        print(
            f"{batch['status']}: "
            f"{counts.get('completed', 0)}/{counts.get('total', 0)} done, "
            f"{counts.get('failed', 0)} failed"
        )
        if batch["status"] in ("completed", "failed", "expired", "cancelled"):
            return batch
        time.sleep(interval)


def download(file_id: str) -> str:
    resp = requests.get(f"{BASE}/v1/files/{file_id}/content", headers=HEADERS)
    resp.raise_for_status()
    return resp.text


def parse_tag(content: str) -> dict | None:
    """Model is asked for JSON only, but defensively strip code fences."""
    text = content.strip()
    if text.startswith("```"):
        text = text.strip("`")
        if text.startswith("json"):
            text = text[4:]
    try:
        obj = json.loads(text)
    except json.JSONDecodeError:
        return None
    if not isinstance(obj, dict):
        return None
    sentiment = obj.get("sentiment")
    topics    = obj.get("topics")
    if sentiment not in ("positive", "neutral", "negative"):
        return None
    if not isinstance(topics, list):
        return None
    return {"sentiment": sentiment, "topics": [str(t) for t in topics[:3]]}


def main() -> None:
    n = build_jsonl("reviews.csv", "input.jsonl")
    print(f"wrote {n} requests to input.jsonl")

    file_id = upload("input.jsonl")
    print(f"uploaded {file_id}")

    batch = create_batch(file_id)
    print(f"created {batch['id']}")

    batch = poll(batch["id"])
    if batch["status"] != "completed":
        raise SystemExit(f"batch ended with status {batch['status']}")

    # Output order is NOT input order. Key everything by custom_id.
    tagged: dict[str, dict] = {}
    if batch.get("output_file_id"):
        for line in download(batch["output_file_id"]).splitlines():
            if not line.strip():
                continue
            rec = json.loads(line)
            # Batch API returns OpenAI-style responses for /v1/chat/completions
            content = rec["response"]["body"]["choices"][0]["message"]["content"]
            tag = parse_tag(content)
            if tag is not None:
                tagged[rec["custom_id"]] = tag

    failures: list[dict] = []
    if batch.get("error_file_id"):
        for line in download(batch["error_file_id"]).splitlines():
            if not line.strip():
                continue
            rec = json.loads(line)
            failures.append({
                "review_id": rec["custom_id"],
                "code":      (rec.get("error") or {}).get("code", ""),
                "message":   (rec.get("error") or {}).get("message", ""),
            })

    # Merge: walk the CSV in its original order, attach tags by custom_id.
    with open("reviews.csv",  encoding="utf-8") as src, \
         open("tagged.csv", "w", encoding="utf-8", newline="") as dst:
        reader = csv.DictReader(src)
        writer = csv.writer(dst)
        writer.writerow(["review_id", "review", "sentiment", "topics"])
        for row in reader:
            tag = tagged.get(row["review_id"], {})
            writer.writerow([
                row["review_id"],
                row.get("review", ""),
                tag.get("sentiment", ""),
                "|".join(tag.get("topics", [])),
            ])

    if failures:
        with open("failed.csv", "w", encoding="utf-8", newline="") as f:
            writer = csv.DictWriter(f, fieldnames=["review_id", "code", "message"])
            writer.writeheader()
            writer.writerows(failures)
        print(f"{len(failures)} rows failed, see failed.csv")

    print(f"done. tagged {len(tagged)}/{n} reviews -> tagged.csv")


if __name__ == "__main__":
    main()
````

A successful output line looks like this:

```json theme={null}
{
  "id": "batch_req_a1",
  "custom_id": "r-001",
  "response": {
    "status_code": 200,
    "request_id": "req_xyz",
    "body": {
      "choices": [
        { "message": { "role": "assistant",
                       "content": "{\"sentiment\":\"positive\",\"topics\":[\"battery\"]}" } }
      ]
    }
  }
}
```

The script keys results by `custom_id`, then walks the original CSV in order to attach tags. This means the row order of `tagged.csv` is identical to `reviews.csv`, even though the Batch API returned results in arbitrary order, and rows that failed land with empty `sentiment` and `topics` columns instead of being dropped silently.

`reviews.csv` goes in:

```csv theme={null}
review_id,review
r-001,"Battery lasts two days, love it."
r-002,"Screen cracked after one drop. Awful build quality."
r-003,"Customer support replied in 3 minutes, very impressed."
r-004,"Shipping took three weeks. Product is fine but the wait was painful."
r-005,"It's okay. Does the job. Nothing special."
r-006,"App keeps crashing on Android 14 after the latest update."
r-007,
r-008,"Sound is crisp, mic is muddy. Mixed feelings."
r-009,"Best purchase of the year. Replaced my old setup entirely."
r-010,"Refund process was smooth, even if the product wasn't for me."
```

`tagged.csv` comes out:

```csv theme={null}
review_id,review,sentiment,topics
r-001,"Battery lasts two days, love it.",positive,battery
r-002,"Screen cracked after one drop. Awful build quality.",negative,build quality|screen
r-003,"Customer support replied in 3 minutes, very impressed.",positive,support
r-004,"Shipping took three weeks. Product is fine but the wait was painful.",neutral,shipping
r-005,"It's okay. Does the job. Nothing special.",neutral,general
r-006,"App keeps crashing on Android 14 after the latest update.",negative,app|stability
r-007,,,
r-008,"Sound is crisp, mic is muddy. Mixed feelings.",neutral,audio|mic
r-009,"Best purchase of the year. Replaced my old setup entirely.",positive,general
r-010,"Refund process was smooth, even if the product wasn't for me.",neutral,refund|support
```

Row `r-007` carries through with empty tags because it was filtered locally before upload, so it never reached the API.

Even a `completed` batch can have a populated `error_file_id` when some lines failed but the batch as a whole ran. A failed line carries a null `response` and a populated `error`. If you skip the local filter and let the API reject the empty row, the error file carries a line like this:

```json theme={null}
{
  "id": "batch_req_a7",
  "custom_id": "r-007",
  "response": null,
  "error": {
    "code": "invalid_request_error",
    "message": "messages[1].content must be a non-empty string",
    "param": "messages[1].content"
  }
}
```

Do not re-run the whole job. Build a follow-up JSONL from the original `input.jsonl`, keyed by the `custom_id`s that appear in the error file, then upload and create a new batch the same way as before:

```python recover.py theme={null}
import json
import os

import requests

BASE    = "https://api.zerogpu.ai"
HEADERS = {"x-api-key": os.environ["ZEROGPU_API_KEY"]}
# x-project-id is optional; send it only when scoping to a project
if os.environ.get("ZEROGPU_PROJECT_ID"):
    HEADERS["x-project-id"] = os.environ["ZEROGPU_PROJECT_ID"]


def errored_ids(error_file_id: str) -> set[str]:
    resp = requests.get(
        f"{BASE}/v1/files/{error_file_id}/content",
        headers=HEADERS
    )
    resp.raise_for_status()

    text = resp.text

    return {
        json.loads(line)["custom_id"]
        for line in text.splitlines() if line.strip()
    }


def retry(input_jsonl: str, error_file_id: str) -> str:
    ids = errored_ids(error_file_id)
    with open(input_jsonl, encoding="utf-8") as src, \
         open("retry.jsonl", "w", encoding="utf-8") as dst:
        for line in src:
            rec = json.loads(line)
            if rec["custom_id"] in ids:
                dst.write(line)
    # ... upload retry.jsonl and create a new batch the same way as before.
    return "retry.jsonl"
```

Common per-line error codes: `invalid_request_error` (repair the line locally and retry), `rate_limit_exceeded` (wait, raise quota, then retry the failed `custom_id`s), `model_error` (transient, safe to retry as-is), and `timeout` (shorten the input or lower `max_tokens`, then retry). For a malformed row like `r-007`, the right fix is upstream: drop or backfill it before building the JSONL, which is exactly what `build_jsonl` already does. See the [Errors reference](/docs/batch/errors) for the full table.

🎉 From one JSONL upload, the Batch API tagged every review row, returned results keyed by `custom_id`, and surfaced failures as a recoverable list, all in a single overnight job that costs a fraction of synchronous calls.

## 🌟 Highlights

This notebook has guided you through setting up and running a ZeroGPU Batch API workflow for tagging a CSV of customer reviews with sentiment and topics. You can adapt and expand this example for various other scenarios requiring high-volume, asynchronous classification over tabular data.

Key tools utilized in this notebook include:

* **ZeroGPU Batch API**: An asynchronous, OpenAI-compatible endpoint that takes thousands of `/v1/chat/completions` requests as a single JSONL file and returns the results within a completion window, at a lower per-request cost than synchronous calls. Here it tags every row of a customer-review CSV in one overnight job.
* **ZeroGPU**: An ultra-fast, compute-efficient inference provider for apps and agents. We run purpose-built small and nano language models across an edge-powered network for the high-volume, purpose-specific tasks your app or agent runs constantly. Plug in our OpenAI-compatible API and you're live - zero GPU infrastructure, serverless, auto-scaling by default.
* **LFM2.5-1.2B-Instruct**: A small, fast instruct model that handles short-form classification well, keeping per-row cost low while still producing open-vocabulary topics and a JSON tag. See the [model card](/api-reference/models/lfm2-5-1-2b-instruct) for context window and playground.

This comprehensive setup allows you to adapt and expand the example for various scenarios requiring high-volume, asynchronous classification over tabular data.


# 🔐 Sanitize a CSV of Customer Feedback with the ZeroGPU Router Plugin
Source: https://docs.zerogpu.ai/cookbook/claude-code-csv-sanitizer

Use the zerogpu-router plugin so Claude Code can redact names, emails, and phone numbers from a raw CSV export in one prompt, returning a clean copy plus a PII audit log.

This notebook demonstrates how to use the `zerogpu-router` plugin so that Claude Code can scrub personal data out of a raw CSV export, all from a single natural-language prompt. You hand Claude a `feedback_export.csv` whose free-text column is full of customer names, emails, and phone numbers, and you get back two files: a clean copy that is safe to share, and a PII audit log of exactly what was removed and where. By combining Claude Code's plugin system and ZeroGPU's PII-aware nano models, this notebook walks you through a practical pattern where Claude orchestrates the file work while ZeroGPU does the high-volume, well-defined redaction, so raw PII never has to live in your transcript.

For the full reference, see the [Claude Code plugin integration guide](/integrations/claude-code-plugin).

In this notebook, you'll explore:

* **Claude Code**: Anthropic's agentic coding tool that runs Claude directly in your terminal, with file editing, command execution, and a plugin system that extends sessions with custom slash commands and skills. Here it reads the CSV, loops over every row, and assembles the output files while routing the redaction work to ZeroGPU.
* **ZeroGPU**: An ultra-fast, compute-efficient inference provider for apps and agents. We run purpose-built small and nano language models across an edge-powered network for the high-volume, purpose-specific tasks your app or agent runs constantly. Plug in our OpenAI-compatible API and you're live - zero GPU infrastructure, serverless, auto-scaling by default.

This setup not only demonstrates a practical application of PII redaction at scale, but also provides a flexible framework that can be adapted to other real-world scenarios requiring consistent, auditable handling of sensitive free-text data.

## 🎥 Watch the Video Guide

<iframe title="YouTube video player" />

## 📦 Installation

First, install the ZeroGPU CLI, which is the binary every router skill wraps. You'll also need [Claude Code](https://docs.claude.com/en/docs/claude-code) itself (`npm install -g @anthropic-ai/claude-code`) and Node.js 20 or newer.

```bash theme={null}
npm install -g zerogpu-cli
zerogpu --version
```

Next, start a Claude Code session by running `claude` in your terminal, then add the marketplace and install the `zerogpu-router` plugin. This is what exposes every ZeroGPU command as a Claude Code skill:

```text theme={null}
/plugin marketplace add zerogpu/zerogpu-router
/plugin install zerogpu-router@zerogpu
/reload-plugins
```

Confirm it's loaded with `/plugin`. You should see `zerogpu-router - enabled`. For the full setup, including CI-friendly flags, see the [Claude Code plugin integration guide](/integrations/claude-code-plugin).

## 🔑 Setting Up API Keys

You'll need to set up your ZeroGPU credentials so that every skill call works without re-prompting. This ensures Claude Code can reach ZeroGPU's inference API securely.

You can go to [here](https://platform.zerogpu.ai/dashboard) to get an API key and Project ID from ZeroGPU. The key starts with `zgpu-api-` and the Project ID (UUID) is on the project settings page.

Sign in once from inside your Claude Code session. You'll be prompted for your API key and Project ID, and both are persisted to your config file:

```text theme={null}
/zerogpu-router:signin
```

Before you run anything, confirm the CLI is installed and you're signed in. `status` exits `0` and prints your masked API key when everything is wired up:

```bash theme={null}
zerogpu --version     # CLI is on your PATH
zerogpu status        # exits 0 and shows your masked API key when signed in
```

```
ZeroGPU CLI 1.x.x
Signed in as project 4ed3e5bb...fd1a
API key: zgpu-api-************XXXX
```

If `status` reports you're not signed in, run `/zerogpu-router:signin` again before continuing.

## 🔐 Redact PII with ZeroGPU

ZeroGPU is an ultra-fast, compute-efficient inference provider for apps and agents. We run purpose-built small and nano language models across an edge-powered network for the high-volume, purpose-specific tasks your app or agent runs constantly. Plug in our OpenAI-compatible API and you're live - zero GPU infrastructure, serverless, auto-scaling by default. In this section, we will redact PII from a single support comment as an example, so you can see exactly what the model gives back before pointing it at a whole file.

The `redact-pii` skill detects PII spans and replaces each one in-line with an uppercase `[LABEL]` placeholder. It routes to `gliner2.5-multi-v1` with `mask: "label"`.

```text theme={null}
/zerogpu-router:redact-pii "Spoke to Sarah Chen but my refund never came. Call me at +1 415-555-0182 or email dana.morris@gmail.com."
```

```
Spoke to [PERSON] but my refund never came. Call me at [PHONE_NUMBER] or email [EMAIL].
```

Note that only spans the model recognizes as PII are replaced. Names, phone numbers, and emails come back masked; an order number or internal ticket ID would pass through untouched.

🎉 **ZeroGPU effortlessly strips the personal data out of free text in one call, providing a cheap, consistent redaction layer for AI integration!**

## 🧾 Sanitize a CSV of Customer Feedback

*This section takes a raw CSV export whose free-text column is full of personal data and produces a clean copy plus a PII audit log, with Claude orchestrating the loop and ZeroGPU doing the redaction on every row.*

Your support tool exports `feedback_export.csv`. The `comment` column is open-ended text where customers typed whatever they wanted, including their names, emails, phone numbers, and sometimes billing addresses. Before this file can go to a dashboard, a Slack channel, or a Git fixture, the PII has to come out. Compliance also wants a record of what was scrubbed, not just a clean file.

Doing this by hand is error-prone, and one missed phone-number format leaks a customer. Regex is brittle. This recipe does it with a PII-aware model, consistently, across every row.

### Step 1: Prepare the input CSV

Place your export in the working directory. The recipe assumes a CSV with at least one free-text column to sanitize; all other columns pass through untouched.

```csv theme={null}
id,date,rating,comment
1001,2026-05-21,2,"Spoke to Sarah Chen but my refund never came. Call me at +1 415-555-0182 or email dana.morris@gmail.com."
1002,2026-05-22,5,"Marcus Rivera was super helpful, thanks!"
1003,2026-05-22,1,"Double charged again. Billing email is priya.patel@northwind-labs.com, acct under James Okafor."
```

Keep a stable, unique `id` column. It's what links a redacted row back to its audit entries. The `date` and `rating` columns are copied verbatim, and `comment` is the only column the models touch. If you don't have an `id` column, ask Claude to add a row index first.

### Step 2: Kick off the workflow with one prompt

In your Claude Code session, in the directory containing the CSV, paste this. That's the whole interaction; everything after it is what Claude does on your behalf.

```text theme={null}
Sanitize feedback_export.csv:
1. Redact PII in the `comment` column and write the result to feedback_clean.csv,
   keeping id, date, and rating unchanged.
2. Produce pii_audit.csv listing every PII entity found, one row per entity, with
   columns: id, category, label, value.
Leave all non-comment columns exactly as they are.
```

### Step 3: Claude reads and parses the CSV

First, Claude opens `feedback_export.csv`, identifies the header row, and isolates the `comment` column as the field to process. It holds the other columns aside to re-attach unchanged. No model calls happen yet; this is just file parsing.

### Step 4: Per row, redact the comment with `redact-pii`

For each row, Claude sends the `comment` value to `redact-pii`, which returns the masked text that goes into the clean sheet.

```text theme={null}
/zerogpu-router:redact-pii "Spoke to Sarah Chen but my refund never came. Call me at +1 415-555-0182 or email dana.morris@gmail.com."
```

```
Spoke to [PERSON] but my refund never came. Call me at [PHONE_NUMBER] or email [EMAIL].
```

### Step 5: Per row, inventory the PII with `extract-pii`

For the same comment, Claude also calls `extract-pii`, which returns the PII entities as structured JSON without modifying the text. This is what populates the audit log. Claude tags each returned entity with the row's `id` so it can be traced back.

```text theme={null}
/zerogpu-router:extract-pii "Spoke to Sarah Chen but my refund never came. Call me at +1 415-555-0182 or email dana.morris@gmail.com." -c identity,contact
```

```json theme={null}
[
  { "category": "identity", "label": "person", "text": "Sarah Chen", "score": 0.96 },
  { "category": "contact",  "label": "phone",  "text": "+1 415-555-0182", "score": 0.95 },
  { "category": "contact",  "label": "email",  "text": "dana.morris@gmail.com", "score": 0.99 }
]
```

Why two calls per row? `redact-pii` gives you the masked text; `extract-pii` gives you the itemized list of what was masked. They run on the same PII model but serve different outputs: the shareable file versus the compliance trail. `extract-pii` defaults to `-t 0.5` and `-c identity,contact`; add `financial`, `medical`, or `credentials` if your text contains them, and raise `-t` to reduce false positives.

### Step 6: Claude assembles the two output files

Claude loops Steps 4 and 5 across every row, then writes both files.

`feedback_clean.csv` keeps the same schema as the input, with `comment` now masked:

```csv theme={null}
id,date,rating,comment
1001,2026-05-21,2,"Spoke to [PERSON] but my refund never came. Call me at [PHONE_NUMBER] or email [EMAIL]."
1002,2026-05-22,5,"[PERSON] was super helpful, thanks!"
1003,2026-05-22,1,"Double charged again. Billing email is [EMAIL], acct under [PERSON]."
```

`pii_audit.csv` has one row per detected entity, joined to the source row by `id`:

```csv theme={null}
id,category,label,value
1001,identity,person,Sarah Chen
1001,contact,phone,+1 415-555-0182
1001,contact,email,dana.morris@gmail.com
1002,identity,person,Marcus Rivera
1003,contact,email,priya.patel@northwind-labs.com
1003,identity,person,James Okafor
```

### Step 7: Verify before you share

Run a few quick sanity checks before the clean file leaves your machine:

```bash theme={null}
# 1. Row counts match (header + same number of data rows)
wc -l feedback_export.csv feedback_clean.csv

# 2. No obvious leftovers; should print nothing
grep -E '@|\+?[0-9][0-9 ()-]{7,}' feedback_clean.csv

# 3. Eyeball the before/after diff
diff <(cut -d, -f4- feedback_export.csv) <(cut -d, -f4- feedback_clean.csv)
```

If check 2 surfaces anything, it's almost always a domain-specific identifier the standard PII model doesn't cover (internal hostnames, contract numbers, order IDs, card last-fours), not a missed name or email. For those, add an `extract-entities` pass with your own labels and mask those spans too:

```text theme={null}
/zerogpu-router:extract-entities "Order #88231 for acct A-4471 failed." --labels order_id,account_id -t 0.4
```

You end up with three files. `feedback_export.csv` is the original raw PII and is not safe to share. `feedback_clean.csv` has the same rows with `comment` masked and is safe to share. `pii_audit.csv` deliberately contains the original PII values, so treat it as a sensitive artifact: store it like any other secret, and never commit it to a public repo or drop it next to the clean file.

🎉 From a single prompt, Claude parsed the CSV, ran `redact-pii` and `extract-pii` on every row, and wrote both a shareable clean copy and an auditable PII log, all while the raw personal data stayed out of its reasoning context.

## 🌟 Highlights

This notebook has guided you through setting up and running a Claude Code workflow with ZeroGPU for sanitizing a CSV of customer feedback. You can adapt and expand this example for various other scenarios requiring consistent, auditable handling of sensitive free-text data.

Key tools utilized in this notebook include:

* **Claude Code**: Anthropic's agentic coding tool that runs Claude directly in your terminal, with file editing, command execution, and a plugin system that extends sessions with custom slash commands and skills. Here it reads the CSV, loops over every row, and assembles the output files while routing the redaction work to ZeroGPU.
* **ZeroGPU**: An ultra-fast, compute-efficient inference provider for apps and agents. We run purpose-built small and nano language models across an edge-powered network for the high-volume, purpose-specific tasks your app or agent runs constantly. Plug in our OpenAI-compatible API and you're live - zero GPU infrastructure, serverless, auto-scaling by default.

This comprehensive setup allows you to adapt and expand the example for various scenarios requiring consistent, auditable handling of sensitive free-text data.


# 🎯 Turn Raw Content into Targeted Copy with the IAB Classifier and Claude
Source: https://docs.zerogpu.ai/cookbook/claude-iab-targeted-copy

Connect the ZeroGPU MCP server in Claude Desktop so Claude can call the zlm-v1-iab-classify-edge model, then turn the returned IAB categories, audience segments, and confidence scores into three targeted outputs: an ad brief, a newsletter blurb, and a content pitch.

This guide shows how to combine ZeroGPU's `zlm-v1-iab-classify-edge` model with Claude entirely inside Claude Desktop, with no code to write or run. You connect the hosted ZeroGPU MCP server once, then Claude reads your article, calls the IAB classifier as a tool to get hard signals (IAB content categories, audience segments, and confidence scores), and uses those signals to write an Ad Brief, a Newsletter Blurb, and a Content Pitch, each in a distinct voice. By pairing ZeroGPU's edge classifier with Claude's writing, this guide walks you through a transparent, reproducible pipeline where the classification numbers visibly drive every line of copy, all from a chat window.

For the full reference, see the [zlm-v1-iab-classify-edge model card](/api-reference/models/zlm-v1-iab-classify-edge) and the [ZeroGPU MCP server guide](/integrations/mcp).

In this guide, you'll explore:

* **ZeroGPU IAB Classifier (`zlm-v1-iab-classify-edge`)**: An edge-served model that maps any text straight to the IAB Content Taxonomy, returning content categories, audience segments, and a per-label confidence score in a single call. No instructions or prompt engineering needed, you send text and get structured signals back. See the [model card](/api-reference/models/zlm-v1-iab-classify-edge).
* **ZeroGPU MCP Server**: The hosted Model Context Protocol server at `https://mcp.zerogpu.ai/mcp` that exposes ZeroGPU's edge models, including the IAB classifier as `zerogpu_classify_iab`, as MCP tools any client can call. Connect it once and Claude Desktop can reach the classifier directly. See the [MCP server guide](/integrations/mcp).
* **ZeroGPU**: An ultra-fast, compute-efficient inference provider for apps and agents. We run purpose-built small and nano language models across an edge-powered network for the high-volume, purpose-specific tasks your app or agent runs constantly. Plug in our OpenAI-compatible API and you're live - zero GPU infrastructure, serverless, auto-scaling by default.
* **Claude Desktop**: Anthropic's desktop app for Claude. Here it orchestrates the pipeline: it connects to the ZeroGPU MCP server, calls the classifier as a tool, reads the returned IAB categories, audience segments, and confidence scores, and writes three distinct outputs grounded in those numbers.

This setup not only demonstrates a practical application of signal-driven copywriting, but also provides a flexible framework that can be adapted to other real-world scenarios requiring classification signals to steer content generation.

## 🎥 Watch the Video Guide

Prefer a quick walkthrough? Watch the full demo here:

<iframe title="YouTube video player" />

## 🔑 Get Your ZeroGPU API Key

You'll need a ZeroGPU API key and Project ID so Claude Desktop can authenticate to the MCP server. This is the only credential setup required, there are no packages to install and no keys to paste into code.

You can go to [here](https://platform.zerogpu.ai/dashboard) to get an API key and Project ID from ZeroGPU. The key starts with `zgpu-api-` and the Project ID (UUID) is on the project settings page.

1. Sign in to the [ZeroGPU dashboard](https://platform.zerogpu.ai/dashboard).
2. Open **API Keys** and click **Create key**.
3. Copy the key (starts with `zgpu-api-`) and grab your Project ID (UUID) from the project settings page.

Keep both values handy for the next step. The classifier is reached through the hosted ZeroGPU MCP server, so no Anthropic API key or SDK is required, you just need the desktop app.

## 🔌 Connect the ZeroGPU MCP Server in Claude Desktop

The ZeroGPU MCP server is hosted at `https://mcp.zerogpu.ai/mcp` and authenticates with the `x-api-key` header (your API key) on every request; the `x-project-id` header (your Project ID) is optional. To make its tools available to Claude, add the server to Claude Desktop's MCP configuration.

In Claude Desktop, open **Settings → Developer → Edit Config**, then add the `zerogpu` entry to `mcpServers` (substitute your real key and Project ID):

```json theme={null}
{
  "mcpServers": {
    "zerogpu": {
      "type": "http",
      "url": "https://mcp.zerogpu.ai/mcp",
      "headers": {
        "x-api-key": "<YOUR_API_KEY>",
        "x-project-id": "<YOUR_PROJECT_ID>"
      }
    }
  }
}
```

Save the file and restart Claude Desktop. Once it reconnects, the ZeroGPU tools (including `zerogpu_classify_iab`) appear in the tools menu of any chat, and Claude can call them on its own. For the request and response shape behind the tool, see the [model card](/api-reference/models/zlm-v1-iab-classify-edge); for the full tool catalog, see the [MCP server guide](/integrations/mcp).

## 🏷️ Classify Content with the ZeroGPU MCP Server

ZeroGPU is an ultra-fast, compute-efficient inference provider for apps and agents. We run purpose-built small and nano language models across an edge-powered network for the high-volume, purpose-specific tasks your app or agent runs constantly. Plug in our OpenAI-compatible API and you're live - zero GPU infrastructure, serverless, auto-scaling by default. In this section, we will classify a short snippet of content against the IAB Content Taxonomy as a standalone example, straight from the Claude Desktop chat.

`zlm-v1-iab-classify-edge` maps your input directly to the taxonomy, so you only send text and get back `audience` segments and `content` categories (in both `iab_1_0` and `iab_2_2` versions), each with a confidence `score`. To try it, just ask Claude to classify a sentence:

```text theme={null}
Classify this against the IAB taxonomy with ZeroGPU:

"The Arc 2 smartwatch tracks heart-rate variability, blood oxygen, and
sleep stages on-device, with a nine-day battery and a $249 price."
```

Claude picks the `zerogpu_classify_iab` tool and calls it with your text:

```json theme={null}
{
  "name": "zerogpu_classify_iab",
  "arguments": {
    "text": "The Arc 2 smartwatch tracks heart-rate variability, blood oxygen, and sleep stages on-device, with a nine-day battery and a $249 price."
  }
}
```

The tool result is a content block whose body parses to the structured signals:

```json theme={null}
{
  "audience": [
    { "name": "Technology & Computing", "score": 0.7821 },
    { "name": "Consumer Electronics",   "score": 0.7402 },
    { "name": "Healthy Living",          "score": 0.6233 },
    { "name": "25-34",                   "score": 0.5990 }
  ],
  "content": {
    "iab_1_0": [
      { "name": "Wearable Technology",   "score": 0.7689 },
      { "name": "Technology & Computing", "score": 0.7421 }
    ],
    "iab_2_2": [
      { "name": "Wearable Technology",   "score": 0.7689 },
      { "name": "Smartphones",            "score": 0.7115 },
      { "name": "Technology & Computing", "score": 0.6902 }
    ]
  }
}
```

🎉 **ZeroGPU effortlessly turns a sentence into IAB categories, audience segments, and confidence scores in one call, the structured signal layer your copy pipeline runs on!**

## 🎯 Turn One Article into Three Targeted Outputs

This section hands Claude a full article and one plain prompt, lets it call the ZeroGPU classifier as a tool, and turns the returned signals into three clearly different pieces of copy.

For the classifier to be more than decoration, Claude has to actually call it and then let the numbers steer the writing. The cleanest way to lock that behavior in is a Claude Desktop **Project**: create one and paste the following into its custom instructions, so every chat in the project follows the same steps.

```text theme={null}
You are a content strategist.

For every article you are given, follow these steps in order:

1. Call the `zerogpu_classify_iab` tool with the article text. It returns IAB
   content categories, audience segments, and a confidence score (0-1) for each.

2. Use those signals explicitly to write THREE outputs. The highest-confidence
   content category and audience segments must visibly shape each one, and you
   should name the categories, segments, and scores you leaned on.

   - Ad Brief        -> persuasive, conversion-focused. Lead with the target
                        segments, a sharp hook, and a clear call to action.
   - Newsletter Blurb -> editorial, engaging. A short, readable paragraph with
                        a headline, written for a curious reader, not a buyer.
   - Content Pitch   -> strategic, high-level. An angle, a "why now", the
                        audience it serves, and where it should be distributed.

Format the response with these exact section markers and nothing before them:
=== AD BRIEF ===
=== NEWSLETTER BLURB ===
=== CONTENT PITCH ===
```

With the project set up, each run is a single, copy-pasteable message. There are no hidden steps: you paste your article in place of `[article text]` and Claude does the rest.

```text theme={null}
Here is my content: [article text]
Classify it and return:
1. Ad Brief
2. Newsletter Blurb
3. Content Pitch
```

If you'd rather not create a project, you can paste the strategist instructions at the top of the same message instead, just keep them above the article. Either way Claude reaches the classifier through the `zerogpu_classify_iab` tool, so the model never touches your ZeroGPU keys, it just asks for a classification and gets the structured result back.

For the example, here is one real article, a product announcement, pasted in place of `[article text]`:

```text theme={null}
Pulse Labs today unveiled the Arc 2, a smartwatch that pushes continuous
health monitoring further than anything in its price class. The Arc 2 tracks
heart-rate variability, blood-oxygen saturation, skin temperature, and sleep
stages around the clock, then surfaces the trends in a redesigned companion
app. A new on-device chip runs the analysis locally, so most insights appear
without ever leaving the wrist. Battery life climbs to nine days on a single
charge, and the titanium case shaves four grams off the previous model. At
$249, the Arc 2 undercuts every flagship wearable while matching their sensor
suite, a bet that the next wave of buyers cares more about daily health
signals than about another screen to check.
```

Claude calls `zerogpu_classify_iab` and the classifier returns the signals it builds on:

```json theme={null}
{
  "audience": [
    { "name": "Technology & Computing", "score": 0.7821 },
    { "name": "Consumer Electronics",   "score": 0.7402 },
    { "name": "Healthy Living",          "score": 0.6233 },
    { "name": "25-34",                   "score": 0.5990 }
  ],
  "content": {
    "iab_1_0": [
      { "name": "Wearable Technology",   "score": 0.7689 },
      { "name": "Technology & Computing", "score": 0.7421 }
    ],
    "iab_2_2": [
      { "name": "Wearable Technology",   "score": 0.7689 },
      { "name": "Smartphones",            "score": 0.7115 },
      { "name": "Technology & Computing", "score": 0.6902 }
    ]
  }
}
```

And here are the three outputs Claude writes from them. Notice how each one names the categories and segments it leaned on, and how the voice shifts between them:

```
=== AD BRIEF ===
Campaign: Pulse Labs Arc 2 launch
Lead segments: Technology & Computing (0.78), Consumer Electronics (0.74),
Healthy Living (0.62), ages 25-34 (0.60)
Buy against: Wearable Technology (0.77) and Consumer Electronics inventory,
programmatic display + paid social.

Hook: Flagship health sensors. Nine-day battery. $249.
Body: The Arc 2 tracks HRV, blood oxygen, skin temperature, and sleep on-device,
so the insight lands on your wrist, not in the cloud. The same sensor suite the
flagships charge double for. Built for the 25-34 buyer who tracks their health
daily and does not want to pay a flagship tax to do it.
CTA: Pre-order the Arc 2 - ships next week.

=== NEWSLETTER BLURB ===
The wearable that bets on your health, not your attention

Most smartwatches want one more glance from you. Pulse Labs went the other way.
The new Arc 2 reads heart-rate variability, blood oxygen, skin temperature, and
sleep through the night, runs the analysis on a new on-device chip, and only
surfaces what matters. Nine days of battery, a lighter titanium case, and a $249
price that quietly undercuts every flagship. One for the Healthy Living crowd
who want the sensor suite without the screen-time.

=== CONTENT PITCH ===
Working title: "The $249 line in the sand for health wearables"

Angle: Pulse Labs is reframing the category around daily health signal rather
than notifications - on-device analysis, longer battery, flagship sensors at
half the price.
Why now: The Arc 2 launch gives a concrete hook for a Wearable Technology piece
(top category, 0.77) that the Technology & Computing audience (0.78) already
follows closely.
Audience: 25-34, health-conscious tech buyers (Healthy Living, 0.62) who weigh
sensor quality against price.
Distribution: Lead on consumer-tech and wearables verticals; cross-post to
health-and-fitness newsletters where the Healthy Living segment over-indexes.
```

🎉 From one article and one plain prompt, the classifier produced hard signals and Claude turned them into a persuasive ad brief, an editorial blurb, and a strategic pitch, each visibly anchored to the same IAB categories and audience scores.

## 🧩 Plug In Your Content Source and Output Destinations

The example above pastes one article into the chat and reads the outputs back. In a real workflow you'll want to pull content from somewhere and push the results somewhere, and because everything runs through MCP, you can do both without writing code, just by adding more MCP servers to Claude Desktop alongside ZeroGPU.

* **Sending content in**: Connect an MCP server for your content source (a CMS, a Google Drive, an RSS or web-fetch server) and ask Claude to grab the latest article, then run it through the same strategist steps. The IAB classifier sees whatever text Claude hands it.
* **Sending outputs out**: Connect MCP servers for your destinations (an email tool, an ads platform, a docs or CMS server) and tell Claude where each section goes, for example "send the Ad Brief to the campaign draft, the Newsletter Blurb to the email block, and the Content Pitch to a new doc."

Because the classifier returns plain structured signals, you can also branch on them in plain language before any copy is written, for example only producing the newsletter blurb when a target segment clears a confidence threshold:

```text theme={null}
Classify this article, then only write the Newsletter Blurb if the top audience
segment scores at least 0.6. Otherwise just tell me the top segment and its score.
```

```text theme={null}
Top audience segment: Technology & Computing (0.7821) - clears 0.6, writing the
Newsletter Blurb now.
```

🎉 The same three-output pipeline now reads from your content source and writes to your channels, with the IAB scores available as routing logic at every step, all orchestrated by Claude over MCP.

## 🚀 Go Deeper

This pipeline is a foundation. A few directions to extend it:

* **Content monetisation**: Attach the returned IAB categories to each article as ad-targeting metadata, then use the confidence scores to pick the highest-value inventory automatically. Pages that classify cleanly into premium categories (for example Consumer Electronics at high confidence) can be routed to higher-CPM ad slots, while low-confidence pages fall back to broad inventory.
* **Personalization pipelines**: Run every incoming article through the classifier, then store the `audience` segments per piece. When a reader arrives, match their on-site behavior to those segments and let Claude rewrite the same blurb in the voice that segment responds to, turning one article into many audience-specific variants.
* **Audience segmentation workflows**: Batch-classify your whole content library and cluster pieces by their `iab_2_2` categories and audience segments. The clusters become ready-made segments for email lists, ad campaigns, and editorial calendars, and Claude can draft a tailored pitch for each cluster from its dominant signals.

For high-volume runs, pair this with the [ZeroGPU Batch API](/docs/batch/getting-started) to classify thousands of articles in one asynchronous job before the copy step.

## 🌟 Highlights

This guide has walked you through combining ZeroGPU's IAB classifier with Claude inside Claude Desktop to turn one article into three targeted outputs driven by real classification signals, with no code to write or run. You can adapt and expand this example for various other scenarios requiring classification signals to steer content generation.

Key tools utilized in this guide include:

* **ZeroGPU IAB Classifier (`zlm-v1-iab-classify-edge`)**: An edge-served model that maps any text straight to the IAB Content Taxonomy, returning content categories, audience segments, and a per-label confidence score in a single call. No instructions or prompt engineering needed, you send text and get structured signals back. See the [model card](/api-reference/models/zlm-v1-iab-classify-edge).
* **ZeroGPU MCP Server**: The hosted Model Context Protocol server at `https://mcp.zerogpu.ai/mcp` that exposes ZeroGPU's edge models, including the IAB classifier as `zerogpu_classify_iab`, as MCP tools any client can call. Connect it once and Claude Desktop can reach the classifier directly. See the [MCP server guide](/integrations/mcp).
* **ZeroGPU**: An ultra-fast, compute-efficient inference provider for apps and agents. We run purpose-built small and nano language models across an edge-powered network for the high-volume, purpose-specific tasks your app or agent runs constantly. Plug in our OpenAI-compatible API and you're live - zero GPU infrastructure, serverless, auto-scaling by default.
* **Claude Desktop**: Anthropic's desktop app for Claude. Here it orchestrates the pipeline: it connects to the ZeroGPU MCP server, calls the classifier as a tool, reads the returned IAB categories, audience segments, and confidence scores, and writes three distinct outputs grounded in those numbers.

This comprehensive setup allows you to adapt and expand the example for various scenarios requiring classification signals to steer content generation.


# 🎫 Route Support Tickets with Claude and ZeroGPU's Thinking Model
Source: https://docs.zerogpu.ai/cookbook/claude-support-ticket-routing

Use the ZeroGPU Skill in Claude to triage incoming support tickets. Claude builds an exact /v1/responses call to the LFM2.5-1.2B-Thinking model, which returns a team and priority for routing.

This guide shows how to use Claude as a support-ticket router with the ZeroGPU Skill. The Skill defines how Claude calls ZeroGPU: when a ticket comes in, Claude builds an exact `curl` request to the `/v1/responses` endpoint, sends it to the `LFM2.5-1.2B-Thinking` model, and reports the routing decision the model returns. Claude does not decide the team or priority itself; the label comes from the API. The result is a triage step that runs on each incoming ticket and produces a consistent team and priority.

For the full reference, see the [LFM2.5-1.2B-Thinking model card](/api-reference/models/lfm2-5-1-2b-thinking).

In this guide, you'll explore:

* **Claude with the ZeroGPU Skill**: Claude follows the `SKILL.md` contract for ZeroGPU. It reads the skill, requires an API key before doing anything, builds the exact `/v1/responses` request for the right model, and reports the response. It does not fabricate or simulate model outputs.
* **ZeroGPU**: An OpenAI-compatible inference provider that serves small, purpose-built language models over a serverless, auto-scaling API. It targets high-volume, narrow tasks and does not require you to provision or manage GPU infrastructure.
* **LFM2.5-1.2B-Thinking**: Liquid AI's 1.2B-parameter reasoning model. It reasons step by step internally before producing a final answer and is small enough to run on edge hardware.

This setup demonstrates automated ticket triage and can be adapted to other scenarios that require classifying short text into fixed categories.

## 🎥 Watch the Video Guide

<iframe title="YouTube video player" />

## 🎫 The Problem: Triaging Tickets at Volume

Support inboxes receive more tickets than agents can read in order. Each ticket needs two decisions before it can be handled: which team owns it (Billing, Technical, Account, General) and how urgent it is (Low, Medium, High). Done by hand, this first pass is slow and varies between agents, and a high-impact issue can sit in a general queue until someone picks it up.

The goal is a routing step that reads each ticket, returns a team and a priority, and runs on every ticket without provisioning GPU infrastructure.

## ⚡ Why ZeroGPU

Triage is a high-volume, narrow classification task. The `LFM2.5-1.2B-Thinking` model is a 1.2B-parameter reasoning model that runs on edge hardware, which keeps per-ticket cost and latency low and removes the need to manage your own servers.

It is a reasoning model, so it works through the ticket internally before committing to a decision, which suits cases where the team and priority depend on context (impact, urgency, number of accounts affected). ZeroGPU serves it over an OpenAI-compatible, serverless API, so the same `/v1/responses` call handles one ticket or many.

## 🔑 Setting Up Your API Key

The ZeroGPU Skill requires an API key before it builds any request. If the key is not available, Claude asks you for it and waits. Per the Skill contract, it does not proceed, does not build a `curl` request, and does not generate or describe any model output until you supply the key.

You can go to the [ZeroGPU dashboard](https://platform.zerogpu.ai/dashboard) to get an API key. The key starts with `zgpu-api-`.

Export the key so Claude can reference it in the request:

```bash theme={null}
export ZEROGPU_API_KEY="zgpu-api-..."
```

Requests authenticate with the `x-api-key` header, and all calls go to the `/v1/responses` endpoint at `https://api.zerogpu.ai/v1`.

## 🧠 The Workflow: How Claude Routes a Ticket

With the Skill loaded, routing a ticket follows a fixed sequence:

1. **Read the Skill.** Claude loads the ZeroGPU `SKILL.md` and selects `LFM2.5-1.2B-Thinking` for the triage task.
2. **Check authentication.** If `ZEROGPU_API_KEY` is not set, Claude prompts you for the key and waits. It does not proceed or produce any model output until the key is provided.
3. **Build the request.** Claude assembles a `curl` call to `/v1/responses`: the ticket text goes in `input`, and the routing rubric goes in `instructions`.
4. **Send it and report the result.** Claude runs the request and reports the routing decision returned by the API. It does not substitute its own label.

### Example prompt

You give Claude a ticket and ask it to route the ticket through ZeroGPU:

```text theme={null}
Route this support ticket with ZeroGPU's thinking model. Assign a team
and a priority.

"Customers on our Pro plan report that exported PDF invoices are missing
line items since this morning's release. Three accounts have emailed
already and one is threatening to churn."
```

### Exact curl request

Claude builds and runs this request. The `instructions` field encodes the routing rubric so the model returns a single, parseable decision line:

```bash theme={null}
curl https://api.zerogpu.ai/v1/responses \
  -H "content-type: application/json" \
  -H "x-api-key: $ZEROGPU_API_KEY" \
  -d '{
    "model": "LFM2.5-1.2B-Thinking",
    "input": "Customers on our Pro plan report that exported PDF invoices are missing line items since this morning'\''s release. Three accounts have emailed already and one is threatening to churn.",
    "instructions": "You are a support triage assistant. Briefly reason, then assign one team (Billing, Technical, Account, General) and one priority (Low, Medium, High). End with a single line: TEAM | PRIORITY."
  }'
```

### API response

The Thinking model reasons internally and ends its output with the routing line. The integration keeps only that final `TEAM | PRIORITY` line and discards the rest, so no intermediate reasoning is surfaced or stored:

```
Technical | High
```

## ✅ Final Outcome

Claude reads the final line, `Technical | High`, and routes the ticket to the engineering queue at high priority. A billing question would return `Billing | Low` or `Billing | Medium` and go to a different queue.

Wired into your inbox, the same `/v1/responses` call runs on every incoming ticket, so each one is assigned a team and a priority without a human first pass and without managing GPU infrastructure.

## 🌟 Highlights

This guide has walked you through using Claude and the ZeroGPU Skill to triage support tickets with the `LFM2.5-1.2B-Thinking` model, where Claude builds the exact `/v1/responses` call and the API returns the team-and-priority decision. You can adapt this example to other scenarios that require classifying short text into fixed categories.

Key tools utilized in this guide include:

* **Claude with the ZeroGPU Skill**: Claude follows the `SKILL.md` contract for ZeroGPU. It reads the skill, requires an API key before doing anything, builds the exact `/v1/responses` request for the right model, and reports the response. It does not fabricate or simulate model outputs.
* **ZeroGPU**: An OpenAI-compatible inference provider that serves small, purpose-built language models over a serverless, auto-scaling API. It targets high-volume, narrow tasks and does not require you to provision or manage GPU infrastructure.
* **LFM2.5-1.2B-Thinking**: Liquid AI's 1.2B-parameter reasoning model. It reasons step by step internally before producing a final answer and is small enough to run on edge hardware.

This setup can be adapted to other scenarios that require classifying short text into fixed categories.


# 🎓 Resume & profile extraction
Source: https://docs.zerogpu.ai/cookbook/data-extraction-tutorial

Step-by-step guide with example dataset. Text-only extraction, no OCR.

This tutorial walks through **three production-style use cases** for structured data extraction with the [gliner2.5-base-v1](/api-reference/models/gliner2-5-base-v1) model. You will use plain text only (resumes, LinkedIn-style exports, job posts). **Not** OCR or PDF parsing.

<Note>
  All sample people and companies in the [example
  dataset](https://github.com/zerogpu/cookbook/tree/main/demos/data-extraction/dataset)
  are **synthetic and fictional**.
</Note>

## 🎥 Watch the Video Guide

<iframe title="YouTube video player" />

## What you will build

| Step | Input | Output |
| - | - | - |
| 1 | Plain-text resume | `candidate` JSON (name, title, skills, …) |
| 2 | Scraped profile text | `profile` JSON (headline, role, location, …) |
| 3 | Job description | Labeled skill/tool spans (`ner`) |

**Model (steps 1-3):** `gliner2.5-base-v1`\
**Endpoint:** `POST https://api.zerogpu.ai/v1/responses`

Get your **API key** and **project ID** from the [dashboard](https://zerogpu.ai). See the [quickstart](/docs/quickstart).

## Before you start

Clone or browse the example data and scripts in the cookbook repo:

```
demos/data-extraction/
├── dataset/          # resumes.jsonl, profiles.jsonl, job_posts.jsonl
├── schemas/          # resume.json, linkedin-profile.json
├── scripts/          # run_batch_extraction.py
└── blog-post.md      # companion article (optional read)
```

GitHub: [zerogpu/cookbook/demos/data-extraction](https://github.com/zerogpu/cookbook/tree/main/demos/data-extraction) (see the [README](https://github.com/zerogpu/cookbook/blob/main/demos/data-extraction/README.md) for setup).

## Step 1: Extract a candidate from a resume

Open `dataset/resumes.jsonl` and pick `resume-001` (synthetic). Define a schema under `metadata.schema` with use case **`json`**.

<CodeGroup>
  ```bash cURL theme={null}
  curl --location 'https://api.zerogpu.ai/v1/responses' \
    --header 'content-type: application/json' \
    --header 'x-api-key: YOUR_API_KEY' \
    --header 'x-project-id: YOUR_PROJECT_ID' \
    --data '{
      "model": "gliner2.5-base-v1",
      "input": "ALEX RIVERA\nSan Francisco, CA | alex.rivera@example-mail.io | (415) 555-0198\n\nSUMMARY\nBackend engineer with 7 years building APIs and data pipelines...",
      "metadata": {
        "usecase": "json",
        "schema": {
          "candidate": [
            "full_name::str::Candidate full name",
            "email::str::Email address",
            "phone::str::Phone number",
            "location::str::City, state, or country",
            "current_title::str::Most recent job title",
            "current_company::str::Most recent employer",
            "skills::str::Comma-separated key skills mentioned"
          ]
        }
      }
    }'
  ```

  ```python Python theme={null}
  import json
  import requests

  url = "https://api.zerogpu.ai/v1/responses"
  headers = {
      "content-type": "application/json",
      "x-api-key": "YOUR_API_KEY",
      "x-project-id": "YOUR_PROJECT_ID",
  }

  # First line of demos/data-extraction/dataset/resumes.jsonl (clone cookbook repo)
  with open("resumes.jsonl", encoding="utf-8") as f:
      record = json.loads(f.readline())

  payload = {
      "model": "gliner2.5-base-v1",
      "input": record["text"],
      "metadata": {
          "usecase": "json",
          "schema": {
              "candidate": [
                  "full_name::str::Candidate full name",
                  "email::str::Email address",
                  "current_title::str::Most recent job title",
                  "current_company::str::Most recent employer",
                  "skills::str::Comma-separated key skills mentioned",
              ]
          },
      },
  }
  body = requests.post(url, headers=headers, json=payload).json()
  parsed = json.loads(body["output"][0]["content"][0]["text"])
  # GLiNER json responses are usually nested: data.candidate[0]
  candidate = parsed["data"]["candidate"][0]
  print(json.dumps(candidate, indent=2))
  ```
</CodeGroup>

**Check your result:** You should see a `candidate` object (often under `data.candidate[0]`) with populated strings. If a field is missing from the source text, it may be empty. Tighten field descriptions or add a second pass only where needed.

## Step 2: Structure LinkedIn-style profile text

Scrapers often return a single blob: headline, About, Experience. Use a **`profile`** schema (same `json` use case).

```json theme={null}
"schema": {
  "profile": [
    "name::str::Person name",
    "headline::str::Professional headline",
    "location::str::Profile location",
    "current_role::str::Current job title",
    "current_company::str::Current company"
  ]
}
```

Paste text from `profiles.jsonl` (`profile-001`) as `input`. Parse the response the same way as Step 1.

<Info>
  This tutorial assumes you already have **compliant text** from your pipeline.
  ZeroGPU does not scrape LinkedIn for you.
</Info>

## Step 3: Tag skills in a job post (NER)

When you need **categories** instead of fixed columns, set `metadata.usecase` to **`ner`** and pass `labels`:

<CodeGroup>
  ```bash cURL theme={null}
  curl --location 'https://api.zerogpu.ai/v1/responses' \
    --header 'content-type: application/json' \
    --header 'x-api-key: YOUR_API_KEY' \
    --header 'x-project-id: YOUR_PROJECT_ID' \
    --data '{
      "model": "gliner2.5-base-v1",
      "input": "We are hiring a Backend Engineer to design REST APIs in Python, deploy on AWS EKS, and maintain PostgreSQL databases.",
      "metadata": {
        "usecase": "ner",
        "labels": ["programming language", "database", "cloud platform", "framework"],
        "threshold": 0.3
      }
    }'
  ```

  ```python Python theme={null}
  # Use job_posts.jsonl record + same headers as Step 1
  payload = {
      "model": "gliner2.5-base-v1",
      "input": record["text"],
      "metadata": {
          "usecase": "ner",
          "labels": ["programming language", "database", "cloud platform", "certification"],
          "threshold": 0.3,
      },
  }
  ```
</CodeGroup>

Lower `threshold` to recall more spans; raise it when precision matters more.

## Step 4: Run the full example dataset

Batch all synthetic resumes or profiles locally:

```bash theme={null}
cd cookbook/demos/data-extraction/scripts
python3 -m venv .venv
.venv/bin/pip install -r requirements.txt
export ZEROGPU_API_KEY="zgpu-..."
export ZEROGPU_PROJECT_ID="your-project-uuid"
.venv/bin/python run_batch_extraction.py --dataset resumes --limit 3
.venv/bin/python run_batch_extraction.py --dataset profiles --limit 3
.venv/bin/python run_batch_extraction.py --dataset job_posts
```

Results are written to `demos/data-extraction/outputs/*.jsonl`. Inspect them before wiring into production ETL.

For many files at once, combine this with the [Batch API](/api-reference/create-batch) (parallelism, retries).

## Optional: PII on inbound text

Before storing user-submitted resumes, run PII extraction with the [gliner2.5-multi-v1](/api-reference/models/gliner2-5-multi-v1) model and `extract-pii` or `redact`.

## Production tips

* **Start small:** 5-8 schema fields beat 20 vague ones.
* **Golden set:** Keep 10-20 labeled examples from your real text shapes; re-run after schema changes.
* **Text only:** PDF and image pipelines are out of scope for this tutorial; convert to text upstream.
* **Monitor:** Use Logs and Usage in the [dashboard](https://platform.zerogpu.ai/dashboard).

## Go deeper

<CardGroup>
  <Card title="gliner2.5-base-v1 model" icon="book" href="/api-reference/models/gliner2-5-base-v1">
    API reference and schema examples for json and NER extraction.
  </Card>

  <Card title="Example dataset (GitHub)" icon="github" href="https://github.com/zerogpu/cookbook/tree/main/demos/data-extraction/dataset">
    Synthetic resumes, profiles, and job posts (JSONL).
  </Card>

  <Card title="Companion article" icon="newspaper" href="https://github.com/zerogpu/cookbook/blob/main/demos/data-extraction/blog-post.md">
    Long-form walkthrough in the cookbook repo.
  </Card>

  <Card title="gliner2.5-base-v1 playground" icon="play" href="/api-reference/models/gliner2-5-base-v1">
    Try schemas interactively in the model catalog.
  </Card>
</CardGroup>


# 🏛️ Raise Your Seed Round with a Hermes Agent
Source: https://docs.zerogpu.ai/cookbook/hermes-investor-outreach

Point Hermes Agent at ZeroGPU, then let it research investors, rank them against your pitch, write a personalized email for each, and save them all as Gmail drafts for a fraction of a frontier-model run.

This notebook demonstrates how to turn Hermes Agent into a fundraising research assistant, so that a single sentence produces five researched investors and five personalized email drafts sitting in your Gmail, ready for you to review and send. The agent reads your company profile, searches the live web for funds that match, pulls the messy prose into structured rows, ranks every candidate against your one-liner, writes an email that opens with something specific and true about each investor, and screens the copy before anything reaches your mailbox. By combining Hermes Agent with ZeroGPU's edge-optimized models, this notebook walks you through a workflow where the only step that needs a capable generative model is the writing, and everything around it - extraction, ranking, classification, summarization, moderation - runs on purpose-built small models at a fraction of the cost.

For the full reference, see the [Hermes Agent integration guide](/integrations/hermes).

In this notebook, you'll explore:

* **Hermes Agent**: An open-source agent from Nous Research that runs in your terminal, a TUI, or a self-hosted web UI. It brings file, terminal, and memory toolsets, a skills system for reusable procedures, and a built-in MCP client that picks up tools from any external server at startup. Here it orchestrates the whole outreach run and decides which tool to reach for at each step.
* **ZeroGPU**: An ultra-fast, compute-efficient inference provider for apps and agents. We run purpose-built small and nano language models across an edge-powered network for the high-volume, purpose-specific tasks your app or agent runs constantly. Plug in our OpenAI-compatible API and you're live - zero GPU infrastructure, serverless, auto-scaling by default.
* **Dappier**: A real-time web and licensed-data search layer exposed over MCP. It supplies the live investor news that the workflow reasons over, since ZeroGPU runs models rather than a search index.
* **Zapier MCP**: A hosted MCP endpoint that turns any of Zapier's app actions into a tool your agent can call. Here it exposes exactly one action, Gmail create draft, so the agent can write drafts and cannot send mail.

This setup not only demonstrates a practical application of agent-driven fundraising research, but also provides a flexible framework that can be adapted to other real-world scenarios requiring high-volume research, ranking, and personalized outbound writing.

## 🎥 Watch the Video Guide

Video walkthrough coming soon.

## 📦 Installation

This recipe assumes you already have Hermes Agent running, either locally or on a host like Hostinger. See the [Hermes Agent quickstart](https://hermes-agent.nousresearch.com/docs/getting-started/quickstart) if you do not.

Everything else is one script. Download it, fill in your keys, and run it:

```bash theme={null}
curl -O https://raw.githubusercontent.com/zerogpu/docs/main/skills/investor-outreach/zerogpu-setup.sh
```

Open the file and set the three values at the top. Only the first is required:

```bash theme={null}
ZEROGPU_API_KEY=""          # required - from platform.zerogpu.ai, starts with zgpu-api-
DAPPIER_API_KEY=""          # optional - real-time web search, from dappier.com
ZAPIER_MCP_URL=""           # optional - your personal Zapier MCP URL
```

Then run it:

```bash theme={null}
bash zerogpu-setup.sh
```

```
── Preflight ──
  ok  hermes Hermes Agent v0.20.4 (2026.8.18)
  ok  ZeroGPU key present (zgpu-api-...)
  ok  config: /data/config.yaml

── Model provider → ZeroGPU ──
  ok  provider configured - model deepseek-v4.1-flash
  ok  config check passed

── MCP servers ──
  ok  zerogpu connected (all tools enabled)
  ok  dappier connected (all tools enabled)
  ok  zapier connected (all tools enabled)

── Verify ──
    Provider:
      provider : custom:ZeroGPU
      model    : deepseek-v4.1-flash
      endpoint : https://api.zerogpu.ai/v1

    Testing the model provider...
  ok  model provider is live on ZeroGPU

── Done ──
    Model    : deepseek-v4.1-flash via custom:ZeroGPU
```

The agent's own reasoning now runs on ZeroGPU's `deepseek-v4.1-flash`, and fourteen ZeroGPU tools are available to it. For what the script does line by line, see the [Hermes Agent integration guide](/integrations/hermes).

## 🔑 Setting Up API Keys

You'll need keys for ZeroGPU, Dappier, and Zapier.

You can go to [here](https://platform.zerogpu.ai/dashboard) to get an API key and Project ID from ZeroGPU. The key starts with `zgpu-api-` and the Project ID (UUID) is on the project settings page.

The setup script writes your ZeroGPU key to Hermes' own `.env` and puts the other two into their MCP URLs. If you would rather set them by hand:

```bash theme={null}
hermes config set ZEROGPU_API_KEY 'zgpu-api-...'
hermes config get ZEROGPU_API_KEY
```

For Dappier, create a key at [dappier.com](https://dappier.com) and pass it as a query parameter:

```bash theme={null}
hermes mcp add dappier --url "https://mcp.dappier.com/mcp?apiKey=YOUR_DAPPIER_KEY" --connect-timeout 60
```

For Zapier, create a server at [mcp.zapier.com](https://mcp.zapier.com) and enable **only** the Gmail create-draft action. Your personal URL carries its own token:

```bash theme={null}
hermes mcp add zapier --url "YOUR_ZAPIER_MCP_URL" --connect-timeout 60
hermes mcp test zapier
```

<Note>
  Read that tool list before continuing. Scoping the Zapier server to a single
  draft action in the Zapier console is what makes this workflow unable to send
  mail, no matter what the agent decides to do. It is a stronger boundary than a
  prompt instruction, because no configuration change on the agent side can
  widen it.
</Note>

## 🧠 Install the Outreach Skill

A Hermes skill is a Markdown procedure the agent loads on demand. This one encodes the whole workflow, which ZeroGPU tool runs at each step, what the email may and may not say, and what the run has to report at the end.

```bash theme={null}
hermes skills install https://raw.githubusercontent.com/zerogpu/docs/main/skills/investor-outreach/SKILL.md
hermes skills list
```

```
  investor-outreach    Research, rank and draft investor outreach using ZeroGPU
```

If the skill does not appear, the YAML frontmatter failed to parse, usually a stray tab or an unquoted colon. There is no lint command, so the listing is the check.

## 📄 Add Your Company Profile

The agent needs to know what it is pitching. Download the template, fill it in, and save it where the skill expects:

```bash theme={null}
curl -O https://raw.githubusercontent.com/zerogpu/docs/main/skills/investor-outreach/company.md
```

```markdown theme={null}
# Dappier

One-liner: AI monetization and licensing infrastructure for publishers - lets content owners get paid when their data, articles, and media are used by AI agents, chatbots, and LLMs.

Stage: seed                              Raising: not publicly disclosed

Traction:
- Founded 2023 in Austin, TX; ~12-17 employees as of 2026
- $2M seed raised from Silverton Partners and Render Capital
- $440K revenue in 2025 with consistent quarter-over-quarter growth

Sector tags: ai-monetization, ad-tech, content-licensing, publisher-tools, agentic-commerce

Geo: Austin, TX (US); remote-friendly

Contact email: hello@dappier.com

Anti-targets: existing investors; funds with a competing portfolio company
```

The one-liner matters more than anything else in the file. It is the string the ranking step embeds, so a vague sentence produces a vague shortlist.

Save it to `/data/workspace/outreach/company.md` on your Hermes host. In the web UI, paste this into a new chat followed by the file contents:

```
Create the file /data/workspace/outreach/company.md, creating the directory if needed, with exactly this content:
```

## 🔎 Run the Workflow

Load the skill and give it one instruction:

```
/investor-outreach
```

```
Use ZeroGPU tools to find 5 investors for /data/workspace/outreach/company.md - funds backing ad-tech, publisher tools or agentic commerce - then draft an email to each in Gmail. Show each tool call.
```

The agent works through the skill's procedure. Watch the tool calls scroll:

```
mcp__dappier__ask_ai              searching: ad-tech seed investors, recent rounds
mcp__zerogpu__zerogpu_extract_json      → 11 investor rows from 6 articles
mcp__zerogpu__zerogpu_extract_entities  → organizations, people, funding rounds
mcp__zerogpu__zerogpu_redact_pii        → scrubbed 4 scraped blobs
mcp__zerogpu__zerogpu_embed             → ranked 11 candidates against the one-liner
mcp__zerogpu__zerogpu_classify_structured → stage / geo_fit / warmth
mcp__zerogpu__zerogpu_summarize         → 5 personalization hooks
mcp__zerogpu__zerogpu_chat              → 5 subject lines and bodies
mcp__zerogpu__zerogpu_moderate          → 5 clean, 0 flagged
mcp__zapier__gmail_create_draft         → 5 drafts created
```

Dappier finds the raw material. Every step that turns it into something useful is a ZeroGPU model, and each one is chosen for its job: a 205M-parameter extractor for pulling fields out of prose, a 384-dimension embedding model for ranking with no generative call at all, an 8B model for one-sentence hooks, and `deepseek-v4.1-flash` only for the writing.

🎉 **ZeroGPU handled nine of the ten steps on small models, leaving the frontier-priced work to the one step that actually needed it.**

<Note>
  If the agent finishes the research but stops short of creating drafts, tell it
  explicitly: `Now create the Gmail drafts using the Zapier MCP tool.` Agents
  sometimes treat an external write as needing confirmation, which is reasonable
  behavior when the action leaves the machine.
</Note>

## 💌 Check the Drafts

Open Gmail and go to **Drafts**. Five emails, each opening with something specific about that investor rather than a restatement of their thesis:

```
Subject: Publisher monetization, $440K ARR on a 4-person team

Hi Sarah,

Saw Foundry led Monda's round in March - the publisher-side
monetization thesis lines up almost exactly with what we're building.

Dappier is the payment and access-control layer between publisher
content and the AI agents that want to use it. Nearly 100 publisher
sites are testing our interactive ad unit, and we did $440K in 2025
on a team of four.

We're raising a seed round and would value 20 minutes.

- Dan
```

Now open **Sent**. It is empty, and it stays empty. The Zapier server exposes no send action, so the agent has no way to mail anyone. A human approves every one.

## 💰 What It Cost

Every ZeroGPU tool returns a `savings` block, and the skill rolls them into `savings.md` at the end of the run:

```
This run cost $0.0042 on ZeroGPU. The same work on a frontier model
would have cost $0.66345 - 158 times cheaper, 99.4% saved, across 24 calls.

Baseline: published frontier rate of $3.00/1M input, $15.00/1M output
Price table version: 2026-08-21

Per-model subtotal
  gliner2.5-base-v1              9 calls    $0.00019
  bge-small-en-v1.5            3 calls    $0.00047
  llama-3.1-8b-instruct-fast   5 calls    $0.00031
  deepseek-v4.1-flash          5 calls    $0.00318
  zlm-v1-moderation-edge       2 calls    $0.00005

Totals
  ZeroGPU        $0.00420
  Frontier       $0.66345
  Saved          $0.65925
  Calls          24

Cost per investor drafted
  ZeroGPU        $0.00084
  Frontier       $0.13269
```

That last pair is the number that scales. Five investors is a rounding error either way; the shape of the run is what matters:

| Investors | ZeroGPU | Frontier baseline |
| - | - | - |
| 5 | \$0.0042 | \$0.66 |
| 1,000 | \$0.84 | \$132.69 |
| 10,000 | \$8.40 | \$1,326.90 |

The comparison is not a benchmark. `baseline_cost_usd` prices the exact same token counts at a published frontier rate, so it is a like-for-like measurement of the same work done two ways.

🎉 **Twenty-four model calls, five researched and drafted investors, and less than half a cent of inference.**

## 🌟 Highlights

This notebook has guided you through setting up and running a Hermes Agent workflow with ZeroGPU for investor research and personalized outreach. You can adapt and expand this example for various other scenarios requiring high-volume research, ranking, and personalized outbound writing.

Key tools utilized in this notebook include:

* **Hermes Agent**: An open-source agent from Nous Research that runs in your terminal, a TUI, or a self-hosted web UI. It brings file, terminal, and memory toolsets, a skills system for reusable procedures, and a built-in MCP client that picks up tools from any external server at startup. Here it orchestrates the whole outreach run and decides which tool to reach for at each step.
* **ZeroGPU**: An ultra-fast, compute-efficient inference provider for apps and agents. We run purpose-built small and nano language models across an edge-powered network for the high-volume, purpose-specific tasks your app or agent runs constantly. Plug in our OpenAI-compatible API and you're live - zero GPU infrastructure, serverless, auto-scaling by default.
* **Dappier**: A real-time web and licensed-data search layer exposed over MCP. It supplies the live investor news that the workflow reasons over, since ZeroGPU runs models rather than a search index.
* **Zapier MCP**: A hosted MCP endpoint that turns any of Zapier's app actions into a tool your agent can call. Here it exposes exactly one action, Gmail create draft, so the agent can write drafts and cannot send mail.

This comprehensive setup allows you to adapt and expand the example for various scenarios requiring high-volume research, ranking, and personalized outbound writing.


# Overview
Source: https://docs.zerogpu.ai/cookbook/index

Ready-to-run recipes for the high-volume tasks ZeroGPU's nano models are built for.

ZeroGPU is an inference provider. We run specialized small and nano language models across an edge-powered network, purpose-built for the high-volume tasks specific verticals depend on, from ad-tech classification to content moderation to document extraction. The result: more efficient compute, faster latency, and roughly 50% lower cost than frontier-model workflows.

The cookbook turns those models into working code. Each recipe is copy, adapt, and run: a focused walkthrough that combines ZeroGPU with another tool, runtime, or SDK against the OpenAI-compatible endpoint at `https://api.zerogpu.ai/v1`. For single-endpoint request and response shapes, see the [API reference](/api-reference/chat-completions) and [model pages](/docs/model-catalog).

Before you begin, read the [quickstart](/docs/quickstart) to provision an [API key](https://platform.zerogpu.ai/dashboard) and pick a model from the [model catalog](/docs/model-catalog). Bring your own **API key**, **project ID**, and **model** from the [dashboard](https://platform.zerogpu.ai/dashboard).

## Tutorials

Longer, step-by-step walkthroughs with example data.

<CardGroup>
  <Card title="Resume & profile extraction" icon="graduation-cap" href="/cookbook/data-extraction-tutorial">
    Three production-style extraction use cases over plain text (resumes, profile exports, job posts). No OCR.
  </Card>

  <Card title="Tag customer reviews overnight" icon="clock" href="/cookbook/batch-review-tagging">
    Process a CSV of customer reviews with sentiment and topic tagging using the Batch API, including error recovery and result merging.
  </Card>
</CardGroup>

## Integrations & plugins

Recipes that combine ZeroGPU with another tool, runtime, or SDK.

<CardGroup>
  <Card title="Sanitize a CSV with the Claude Code plugin" icon="shield" href="/cookbook/claude-code-csv-sanitizer">
    Redact PII from a feedback export and produce an audit log, from a single natural-language prompt.
  </Card>

  <Card title="Screen resumes with LangChain" icon="crow" href="/cookbook/langchain-resume-screening">
    Extract entities, redact PII, and route a candidate from a PDF resume.
  </Card>

  <Card title="Turn content into targeted copy with Claude" icon="tags" href="/cookbook/claude-iab-targeted-copy">
    Classify an article with the IAB model, then have Claude write an ad brief, newsletter blurb, and content pitch from the signals.
  </Card>

  <Card title="Route support tickets with Claude" icon="layers" href="/cookbook/claude-support-ticket-routing">
    Have Claude build the exact /v1/responses call to the Thinking model, which reasons through each ticket and returns a team and priority.
  </Card>

  <Card title="Raise a seed round with Hermes Agent" icon="rocket" href="/cookbook/hermes-investor-outreach">
    Research investors, rank them against your pitch, and land five personalized Gmail drafts for under half a cent.
  </Card>

  <Card title="Summarize your inbox with OpenClaw" icon="file-text" href="/cookbook/openclaw-inbox-summarizer">
    Forward messy messages to your OpenClaw assistant and get back a PII-safe summary with topic and urgency labels.
  </Card>
</CardGroup>


# 🦜 Screen Resumes with LangChain and ZeroGPU
Source: https://docs.zerogpu.ai/cookbook/langchain-resume-screening



You can also run this cookbook in Colab [here](https://colab.research.google.com/drive/10OYr9s4kfp63twlN_FjXfuhtZpwjba6A?usp=sharing).

Recruiting teams drown in resumes. Every resume is a blob of unstructured text wrapped around a pile of personal data: names, emails, phone numbers, home addresses. You want the signal (skills, titles, companies) without holding the PII any longer than you must, and GDPR and CCPA make "any longer than you must" a legal line, not a preference. This notebook builds a small resume-screening pipeline that pulls the structured signal out of a PDF resume, strips the PII before anything is stored or shared, and routes the candidate to the right team, using three ZeroGPU tools through LangChain.

For the full reference, see the [`langchain-zerogpu` repository on GitHub](https://github.com/zerogpu/langchain-zerogpu) and the [package on PyPI](https://pypi.org/project/langchain-zerogpu/).

In this notebook, you'll explore:

* **LangChain**: The framework for building applications with LLMs and agents. Its `langchain-zerogpu` package exposes ZeroGPU's small and nano models as first-class tools, so you can drop entity extraction, PII redaction, and zero-shot classification straight into a pipeline without writing any HTTP code.
* **ZeroGPU**: An ultra-fast, compute-efficient inference provider for apps and agents. We run purpose-built small and nano language models across an edge-powered network for the high-volume, purpose-specific tasks your app or agent runs constantly. Plug in our OpenAI-compatible API and you're live - zero GPU infrastructure, serverless, auto-scaling by default.

This setup not only demonstrates a practical application of privacy-aware resume screening, but also provides a flexible framework that can be adapted to other real-world scenarios requiring structured extraction and compliant handling of sensitive documents.

## 🎥 Watch the Video Guide

<iframe title="YouTube video player" />

## 📦 Installation

First, install the `langchain-zerogpu` package, which ships the ZeroGPU tools. You'll also install `pypdf` to pull the text out of an uploaded PDF resume (this pipeline is text-only, no OCR):

```bash theme={null}
!pip install -U langchain-zerogpu pypdf
```

## 🔑 Setting Up API Keys

You'll need to set up your ZeroGPU credentials so the tools can reach the inference API securely. This ensures every tool call is authenticated without re-prompting.

You can go to [here](https://platform.zerogpu.ai/dashboard) to get an API key and Project ID from ZeroGPU. The key starts with `zgpu-api-` and the Project ID (UUID) is on the project settings page.

```python Python theme={null}
import os
from getpass import getpass

# Prompt for the ZeroGPU API key and Project ID securely
zerogpu_api_key = getpass('Enter your ZeroGPU API key: ')
os.environ["ZEROGPU_API_KEY"] = zerogpu_api_key

zerogpu_project_id = getpass('Enter your ZeroGPU Project ID: ')
os.environ["ZEROGPU_PROJECT_ID"] = zerogpu_project_id
```

Each tool reads `ZEROGPU_API_KEY` and `ZEROGPU_PROJECT_ID` from the environment when you construct it with no arguments. The key is held as a `SecretStr` and is never logged.

<Note>
  The example outputs throughout this notebook are illustrative placeholders that show the shape of each result. Rerun the cells with your own credentials and resume to capture live values.
</Note>

## 📄 Upload a Resume

Upload a single PDF resume straight from your machine, then read its text with `pypdf`. Everything downstream runs on `resume_text`.

```python Python theme={null}
from google.colab import files
from pypdf import PdfReader

# Pick one PDF resume from the file dialog
uploaded = files.upload()
pdf_path = next(iter(uploaded))

reader = PdfReader(pdf_path)
resume_text = "\n".join(page.extract_text() or "" for page in reader.pages)

print(f"Loaded {pdf_path} ({len(resume_text)} characters)")
print(resume_text[:500])
```

```
Loaded resume.pdf (1452 characters)
Daniel R. Okafor
Senior Software Engineer
Email: daniel.okafor@gmail.com  |  Phone: (415) 555-0192
Address: 2487 Folsom Street, Apt 5B, San Francisco, CA 94110
LinkedIn: linkedin.com/in/danielokafor
Professional Summary
Senior software engineer with 8+ years writing and shipping production software. Builds product features, web
services, and developer-facing APIs in Python, Go, and Java. Strong in software design, testing, and code
review.
Experience
Senior Software Engineer - Stripe
San Francis
```

## 🏷️ Extract Candidate Entities with ZeroGPU

ZeroGPU is an ultra-fast, compute-efficient inference provider for apps and agents. We run purpose-built small and nano language models across an edge-powered network for the high-volume, purpose-specific tasks your app or agent runs constantly. Plug in our OpenAI-compatible API and you're live - zero GPU infrastructure, serverless, auto-scaling by default. In this section, we will extract skills, job titles, companies, and locations from the resume as a structured candidate card.

`ZeroGPUExtractEntitiesTool` runs custom-label named-entity recognition on `gliner2.5-base-v1`. You pass the entity types you care about as `labels`, and it returns the matched spans grouped by label.

```python Python theme={null}
from langchain_zerogpu import ZeroGPUExtractEntitiesTool

extract_entities = ZeroGPUExtractEntitiesTool()
entities = extract_entities.invoke({
    "text": resume_text,
    "labels": ["skill", "job title", "company", "location"],
})
print(entities)
```

```
{'entities': {'skill': ['Python', 'Go', 'Java', 'Redis', 'SQL', 'TypeScript', 'PostgreSQL', 'React', 'Kubernetes', 'Docker', 'Git', 'AWS', 'REST', 'gRPC'], 'job title': ['Senior Software Engineer', 'Software Engineer'], 'company': ['Airbnb', 'Twilio', 'Stripe'], 'location': ['San Francisco', 'Illinois', 'California', 'Texas', 'Austin', 'Urbana']}}
```

The result nests every match under its label, so you can assemble a clean candidate card in a couple of lines:

```python Python theme={null}
card = entities["entities"]
print("Candidate card")
print("- Skills:    ", ", ".join(card.get("skill", [])))
print("- Titles:    ", ", ".join(card.get("job title", [])))
print("- Companies: ", ", ".join(card.get("company", [])))
print("- Locations: ", ", ".join(card.get("location", [])))
```

```
Candidate card
- Skills:     Python, Go, Java, Redis, SQL, TypeScript, PostgreSQL, React, Kubernetes, Docker, Git, AWS, REST, gRPC
- Titles:     Senior Software Engineer, Software Engineer
- Companies:  Airbnb, Twilio, Stripe
- Locations:  San Francisco, Illinois, California, Texas, Austin, Urbana
```

🎉 **ZeroGPU effortlessly turns a wall of resume text into a structured candidate card in one call, providing the cheap, repeatable extraction layer your screening pipeline runs on!**

## 🔐 Redact PII Before Sharing

Before the resume goes into a dashboard, an applicant-tracking system, or a shared review channel, the personal data has to come out. `ZeroGPURedactPIITool` masks PII inline on `gliner2.5-multi-v1`, replacing each match with an uppercase `[LABEL]` placeholder and returning the redacted text.

```python Python theme={null}
from langchain_zerogpu import ZeroGPURedactPIITool

redact_pii = ZeroGPURedactPIITool()
anonymized_resume = redact_pii.invoke({"text": resume_text})
print(anonymized_resume[:500])
```

```
{"redacted_text":"[PERSON]\nSenior Software Engineer\nEmail: [EMAIL]  |  Phone: [PHONE_NUMBER]\nAddress: [ADDRESS]\nLinkedIn: [SOCIAL_MEDIA_HANDLE]\nProfessional Summary\nSenior software engineer with 8+ years writing and shipping production software. Builds product features, web\nservices, and developer-facing APIs in Python, Go, and Java. Strong in software design, testing, and code\nreview.\nExperience\nSenior Software Engineer - [ORGANIZATION]\nSan Francisco, California  -  Mar 2021 to Prese
```

The tool returns a JSON envelope, not a bare string: `redacted_text` holds the masked copy, and an `entities` list records every span that was detected and replaced. Parse it with `json.loads(anonymized_resume)["redacted_text"]` when you just want the masked text.

Names, emails, phone numbers, addresses, and even company names come back masked, while skills and the rest of the prose pass through, so the redacted copy is still useful for review with the identifying details gone. This is the version that is safe to store or share.

🎉 **ZeroGPU strips the personal data out of the resume in a single call, giving you a compliant, shareable copy without touching the rest of the content!**

## 🧭 Route the Candidate with Zero-Shot Classification

With the signal extracted and the PII gone, the last step is routing: which team should review this candidate? `ZeroGPUClassifyZeroShotTool` scores the resume against a flat list of labels on `deberta-v3-small` and returns a score per label, so you can pick the best match.

```python Python theme={null}
from langchain_zerogpu import ZeroGPUClassifyZeroShotTool

classify = ZeroGPUClassifyZeroShotTool()
routing = classify.invoke({
    "text": resume_text,
    "labels": ["engineering", "design", "sales", "marketing"],
})
print(routing)

team = max(routing, key=routing.get)
print(f"Route to: {team}")
```

```
{'engineering': 0.483139, 'design': 0.27458, 'sales': 0.143601, 'marketing': 0.098679}
Route to: engineering
```

The tool returns every label with a score, so `max(routing, key=routing.get)` gives you the top team. Swap the `labels` list for your own org chart to route against any set of teams.

🎉 **ZeroGPU classifies the resume against your own team labels in one call, turning unstructured text into a routing decision!**

## 🤖 Run the Full Screening Pipeline

Now chain the three tools into one function that takes a resume and returns a structured candidate card, an anonymized copy, and a routing label. This is the end-to-end pipeline you'd wire into a real screening flow.

```python Python theme={null}
import json

from langchain_zerogpu import (
    ZeroGPUClassifyZeroShotTool,
    ZeroGPUExtractEntitiesTool,
    ZeroGPURedactPIITool,
)

extract_entities = ZeroGPUExtractEntitiesTool()
redact_pii = ZeroGPURedactPIITool()
classify = ZeroGPUClassifyZeroShotTool()


def screen_resume(text: str) -> dict:
    entities = extract_entities.invoke({
        "text": text,
        "labels": ["skill", "job title", "company", "location"],
    })
    anonymized = redact_pii.invoke({"text": text})
    routing = classify.invoke({
        "text": text,
        "labels": ["engineering", "design", "sales", "marketing"],
    })
    return {
        "candidate_card": entities["entities"],
        "anonymized_resume": anonymized,
        "routing_label": max(routing, key=routing.get),
        "routing_scores": routing,
    }


result = screen_resume(resume_text)
print(json.dumps(result, indent=2))
```

```
{
  "candidate_card": {
    "skill": [
      "Python",
      "Go",
      "Java",
      "Redis",
      "SQL",
      "TypeScript",
      "PostgreSQL",
      "React",
      "Kubernetes",
      "Docker",
      "Git",
      "AWS",
      "REST",
      "gRPC"
    ],
    "job title": [
      "Senior Software Engineer",
      "Software Engineer"
    ],
    "company": [
      "Airbnb",
      "Twilio",
      "Stripe"
    ],
    "location": [
      "San Francisco",
      "Illinois",
      "California",
      "Texas",
      "Austin",
      "Urbana"
    ]
  },
  "anonymized_resume": "{\"redacted_text\":\"[PERSON]\\nSenior Software Engineer\\nEmail: [EMAIL]  |  Phone: [PHONE_NUMBER]\\nAddress: [ADDRESS]\\nLinkedIn: [SOCIAL_MEDIA_HANDLE]\\nProfessional Summary\\nSenior software engineer with 8+ years writing and shipping production software. Builds product features, web\\nservices, and developer-facing APIs in Python, Go, and Java. Strong in software design, testing, and code\\nreview.\\nExperience\\nSenior Software Engineer - [ORGANIZATION]\\nSan Francisco, California  -  Mar 2021 to Present\\n- Developed a payments reconciliation service in Go that processes [TRANSACTION_NUMBER] per day.\\n- Designed and built new REST and gRPC APIs adopted by six product engineering teams.\\n- Wrote the automated test suite and led code reviews for the billing service.\\nSoftware Engineer - [ORGANIZATION]  -  Jun 2018 to Feb 2021\\n- Built the pricing API in Python serving over 2 billion requests per month.\\n- Implemented a data pipeline in Java and Spark that powered search ranking features.\\nSoftware Engineer - [ORGANIZATION]  -  Jul 2016 to May 2018\\n- Developed REST and gRPC services in Python and Java for the messaging product.\\n- Added integration tests and improved code quality across the team.\\nTechnical Skills\\nPython, Go, Java, TypeScript, SQL, React, PostgreSQL, Redis, Docker, Kubernetes, AWS, Git\\nEducation\\nB.S. Computer Science - University of Illinois\\nUrbana, Illinois  -  2012 to 2016\\n\",\"entities\":[{\"text\":\"Daniel R. Okafor\",\"label\":\"person\",\"start\":0,\"end\":16,\"score\":0.9927},{\"text\":\"daniel.okafor@gmail.com\",\"label\":\"email\",\"start\":49,\"end\":72,\"score\":0.9913},{\"text\":\"(415) 555-0192\",\"label\":\"phone number\",\"start\":84,\"end\":98,\"score\":0.9946},{\"text\":\"2487 Folsom Street, Apt 5B, San Francisco, CA 94110\",\"label\":\"address\",\"start\":108,\"end\":159,\"score\":0.9351},{\"text\":\"linkedin.com/in/danielokafor\",\"label\":\"social media handle\",\"start\":170,\"end\":198,\"score\":0.6243},{\"text\":\"Stripe\",\"label\":\"organization\",\"start\":482,\"end\":488,\"score\":0.9139},{\"text\":\"12M transactions\",\"label\":\"transaction number\",\"start\":606,\"end\":622,\"score\":0.6892},{\"text\":\"Airbnb\\nSan Francisco, California\",\"label\":\"organization\",\"start\":817,\"end\":849,\"score\":0.8436},{\"text\":\"Twilio\\nAustin, Texas\",\"label\":\"organization\",\"start\":1058,\"end\":1078,\"score\":0.7601}],\"entities_by_label\":{\"person\":[\"Daniel R. Okafor\"],\"email\":[\"daniel.okafor@gmail.com\"],\"phone number\":[\"(415) 555-0192\"],\"address\":[\"2487 Folsom Street, Apt 5B, San Francisco, CA 94110\"],\"social media handle\":[\"linkedin.com/in/danielokafor\"],\"organization\":[\"Stripe\",\"Airbnb\\nSan Francisco, California\",\"Twilio\\nAustin, Texas\"],\"transaction number\":[\"12M transactions\"]}}",
  "routing_label": "engineering",
  "routing_scores": {
    "engineering": 0.483139,
    "design": 0.27458,
    "sales": 0.143601,
    "marketing": 0.098679
  }
}
```

🎉 From a single PDF, the pipeline produced a structured candidate card, a PII-free copy safe to store or share, and a routing label, with each task running on a purpose-built ZeroGPU model instead of a frontier LLM.

## 🌟 Highlights

This notebook has guided you through setting up and running a LangChain workflow with ZeroGPU for privacy-aware resume screening. You can adapt and expand this example for various other scenarios requiring structured extraction and compliant handling of sensitive documents.

Key tools utilized in this notebook include:

* **LangChain**: The framework for building applications with LLMs and agents. Its `langchain-zerogpu` package exposes ZeroGPU's small and nano models as first-class tools, so you can drop entity extraction, PII redaction, and zero-shot classification straight into a pipeline without writing any HTTP code.
* **ZeroGPU**: An ultra-fast, compute-efficient inference provider for apps and agents. We run purpose-built small and nano language models across an edge-powered network for the high-volume, purpose-specific tasks your app or agent runs constantly. Plug in our OpenAI-compatible API and you're live - zero GPU infrastructure, serverless, auto-scaling by default.

This comprehensive setup allows you to adapt and expand the example for various scenarios requiring structured extraction and compliant handling of sensitive documents. For more, see the [`langchain-zerogpu` repository](https://github.com/zerogpu/langchain-zerogpu), the [PyPI page](https://pypi.org/project/langchain-zerogpu/), and the [ZeroGPU documentation](https://docs.zerogpu.ai).


# 🦞 Build a PII-Safe Inbox Summarizer with Your OpenClaw Assistant
Source: https://docs.zerogpu.ai/cookbook/openclaw-inbox-summarizer

Forward messy messages to your OpenClaw assistant and have it redact PII, summarize the thread, and classify topic and urgency, offloading every cheap step to ZeroGPU's nano models.

This notebook demonstrates how to turn your OpenClaw assistant into a PII-safe inbox summarizer, so that any long or messy message you forward it comes back as a short, shareable summary with a topic and urgency label, and never drags raw personal data into your primary model's context. You send a wall-of-text support email, a forwarded thread, or a noisy group-chat recap to the assistant in whatever chat app you already use, and it replies with the gist plus a couple of labels, having redacted the names, emails, and phone numbers first. By combining OpenClaw's chat-app gateway with ZeroGPU's edge-optimized nano models, this notebook walks you through a practical pattern where OpenClaw handles the conversation while ZeroGPU does the high-volume, well-defined summarize, classify, and redact work for a fraction of a frontier-model call.

For the full reference, see the [OpenClaw integration guide](/integrations/openclaw).

In this notebook, you'll explore:

* **OpenClaw**: A self-hosted, open-source personal AI assistant that connects the chat apps you already use - Telegram, Slack, Discord, WhatsApp, iMessage, and more - to AI agents and tools through a plugin system. Here it receives the messages you forward in your channel of choice and routes the cheap summarize, classify, and redact steps to ZeroGPU instead of your primary model.
* **ZeroGPU**: An ultra-fast, compute-efficient inference provider for apps and agents. We run purpose-built small and nano language models across an edge-powered network for the high-volume, purpose-specific tasks your app or agent runs constantly. Plug in our OpenAI-compatible API and you're live - zero GPU infrastructure, serverless, auto-scaling by default.

This setup not only demonstrates a practical application of privacy-safe message triage, but also provides a flexible framework that can be adapted to other real-world scenarios requiring fast, low-cost handling of high-volume, free-text messages.

## 🎥 Watch the Video Guide

Video walkthrough coming soon.

## 📦 Installation

This recipe assumes you already have an OpenClaw gateway installed (see the [OpenClaw docs](https://docs.openclaw.ai/)). You'll add the `zerogpu-router` plugin and the `zerogpu` CLI here, then connect a Telegram channel in a moment so you can actually talk to your assistant.

First, install the CLI and confirm it's on your PATH:

```bash theme={null}
npm install -g zerogpu-cli
zerogpu --version
```

Next, log in once, then add the plugin from ClawHub. This is what exposes every ZeroGPU command as an OpenClaw skill your assistant can auto-invoke:

```bash theme={null}
zerogpu login
openclaw plugins install clawhub:zerogpu-router
```

A freshly installed plugin loads its skills on the next gateway start, so if your gateway is already running, restart it (a managed gateway with config reload restarts itself). In a self-hosted or containerized OpenClaw gateway, also make sure the `zerogpu` binary is available inside the same environment the assistant runs in, not just on your laptop. For the full setup, including how skills auto-invoke, see the [OpenClaw integration guide](/integrations/openclaw).

## 🔑 Setting Up API Keys

You'll need to set up your ZeroGPU credentials so every skill call works without re-prompting. This ensures your OpenClaw assistant can reach ZeroGPU's inference API securely.

You can go to [here](https://platform.zerogpu.ai/dashboard) to get an API key from ZeroGPU. The key starts with `zgpu-api-`.

Sign in once. `zerogpu login` takes just your API key - no Project ID required - and persists it to your config file so every skill call is authenticated:

```bash theme={null}
zerogpu login
```

Before you forward anything, confirm the CLI is installed and you're signed in. `status` exits `0` and prints your masked API key when everything is wired up:

```bash theme={null}
zerogpu --version     # CLI is on your PATH
zerogpu status        # exits 0 and shows your masked API key when signed in
```

```
ZeroGPU CLI 1.x.x
Signed in ✓
API key: zgpu-api-************XXXX
```

If `status` reports you're not signed in, ask your assistant to sign you in, or run `zerogpu login` again before continuing.

## 💬 Connect a Telegram Channel

OpenClaw talks to you through the chat apps you already use, and Telegram is the quickest to wire up: you create a bot, drop its token into your OpenClaw config, start the gateway, and approve a pairing code. Every message you send the bot from then on reaches your assistant, which is where the triage workflow runs. Here's the whole thing, start to finish.

### Step 1: Create a bot with BotFather

In Telegram, message [`@BotFather`](https://t.me/BotFather) (confirm the handle is exactly `@BotFather`), send `/newbot`, and follow the prompts to pick a display name and a username ending in `bot`. BotFather replies with a token. Copy it and keep it secret, since anyone with the token controls your bot.

```text theme={null}
You:       /newbot
BotFather: Alright, a new bot. How are we going to call it?
You:       ZeroGPU Triage
BotFather: Good. Now let's choose a username for your bot. It must end in `bot`.
You:       zerogpu_triage_bot
BotFather: Done! Use this token to access the HTTP API:
           123456789:AAExampleTokenStringFromBotFather
```

### Step 2: Add the token to your OpenClaw config

OpenClaw reads its config from `~/.openclaw/openclaw.json` (it's JSON5, so comments, unquoted keys, and trailing commas are all fine). If that file isn't there yet, create it, or run `openclaw configure` for a guided wizard. To keep config elsewhere, point `OPENCLAW_CONFIG_PATH` at the file. Open it and add a `channels.telegram` block with your token. `dmPolicy: "pairing"` means new direct-message senders have to be approved with a pairing code (Step 4), and `requireMention` keeps the bot quiet in groups unless it's mentioned:

```json5 theme={null}
{
  channels: {
    telegram: {
      enabled: true,
      botToken: "123456789:AAExampleTokenStringFromBotFather",
      dmPolicy: "pairing",
      groups: { "*": { requireMention: true } },
    },
  },
}
```

Rather than hand-edit the file, you can set the same values from the CLI (or use the **Config** tab in the Control UI at `http://127.0.0.1:18789`):

```bash theme={null}
openclaw config set channels.telegram.enabled true
openclaw config set channels.telegram.botToken "123456789:AAExampleTokenStringFromBotFather"
openclaw config set channels.telegram.dmPolicy pairing
```

Prefer to keep the token out of the config file? Set it as an environment variable instead (this resolves for the default account only):

```bash theme={null}
export TELEGRAM_BOT_TOKEN="123456789:AAExampleTokenStringFromBotFather"
```

You can also point `tokenFile` at a file that holds the token if you'd rather not inline it.

### Step 3: Start the gateway

```bash theme={null}
openclaw gateway
```

This launches OpenClaw and connects your bot to Telegram. Leave it running, since it's the process that receives your messages and runs the skills.

### Step 4: Pair your Telegram account

Open Telegram, find your bot by its username, and send it any message, like `hi`. Because `dmPolicy` is `pairing`, OpenClaw holds that first request and prints a pairing code instead of replying. Approve it from another terminal:

```bash theme={null}
openclaw pairing list telegram
openclaw pairing approve telegram <CODE>
```

Pairing codes expire after 1 hour, so approve promptly. Once approved, your DMs reach the assistant and it starts replying.

### Step 5: Triage a shared group (optional)

To triage a shared support channel instead of your own DMs, add the bot to a Telegram group. With `requireMention` on, it only acts when someone mentions it, so it won't respond to every message in the group. To control who can invoke it, add your numeric Telegram user ID to `allowFrom`. The safest way to find that ID is to DM your bot and watch the logs:

```bash theme={null}
openclaw logs --follow
```

Read the `from.id` value from your message. Group chat IDs are the negative numbers that start with `-100`.

### Verify it works

Send your bot a quick message in Telegram:

```text theme={null}
hi
```

```
Hi! I'm connected and ready. Forward me a message and I'll help you triage it. 🦞
```

Everything in the sections below happens right here in this Telegram chat: you message the assistant, it runs the ZeroGPU skills, and it replies inline.

## 📨 Summarize a Message with ZeroGPU

ZeroGPU is an ultra-fast, compute-efficient inference provider for apps and agents. We run purpose-built small and nano language models across an edge-powered network for the high-volume, purpose-specific tasks your app or agent runs constantly. Plug in our OpenAI-compatible API and you're live - zero GPU infrastructure, serverless, auto-scaling by default. In this section, we will summarize a single forwarded message as an example, so you can see exactly what comes back before wiring it into your assistant's inbox workflow.

The `summarize` skill condenses a passage into a few sentences. It routes to `llama-3.1-8b-instruct-fast`. Send your assistant a message like this in your chat app:

```text theme={null}
summarize this: Hey team, quick recap of today's sync. We agreed to push the mobile
checkout redesign to the March release because the payments integration slipped a
sprint. QA flagged two blocker bugs on Android that engineering will prioritize this
week. Marketing still wants the new pricing page live before the end of the quarter,
so we will revisit scope on Thursday. Please update your tickets before standup.
```

The assistant recognizes the intent, routes to the `summarize` skill, and replies:

```
Mobile checkout redesign moved to the March release after the payments integration
slipped a sprint. QA found two Android blocker bugs for engineering to fix this week,
and marketing still wants the pricing page live by quarter-end, with scope revisited
Thursday. Everyone should update their tickets before standup.

model: llama-3.1-8b-instruct-fast · 84 tokens in / 58 out
```

Notice the trailing `model:` line: the work ran on a nano model, not your primary one. That single call is the building block for the whole inbox workflow below.

🎉 **ZeroGPU effortlessly turns a wall of text into a few clean lines in one call, providing a cheap, consistent summarization layer for AI integration!**

## 🦞 Set Up an Automatic Triage Workflow

*This is the recipe: you configure the triage workflow once by saving it to your assistant's memory, and from then on every message you forward is redacted, summarized, and labeled automatically on ZeroGPU's nano models. Set it up once, use it forever.*

The pattern is set-and-forget. Instead of describing the pipeline every time, you teach it to the assistant a single time as a standing rule. OpenClaw keeps that instruction in its memory for the channel, so future messages trigger the same redact-summarize-label workflow without you typing anything but the forward.

### Step 1: Save the workflow to memory (once)

Give your assistant the workflow as a standing instruction and ask it to remember it for this channel. You do this a single time:

```text theme={null}
Remember this as my triage workflow for this channel: whenever I forward a message,
first redact any PII, then summarize it in two or three lines, then label it by topic
(account, billing, bug, feature-request, other) and urgency (low, medium, high).
Reply with just the summary and the labels, nothing else.
```

The assistant confirms and stores the rule:

```
Saved. From now on, any message you forward to this channel will be redacted,
summarized, and labeled automatically. 🦞
```

That's the whole setup. You never send that instruction again. (Exactly how a rule is persisted depends on your OpenClaw configuration; see the [OpenClaw docs](https://docs.openclaw.ai/).)

### Step 2: Forward a message

Day to day, you do just one thing: forward the raw message, with no instruction attached.

```text theme={null}
Fwd: URGENT can't access my account -
From: Daniel Okafor <daniel.okafor@brightwave.io>

Hey, I've tried logging in like 10 times since this morning and it keeps saying
"session expired". I even reset my password twice. This is really frustrating because
I have a client demo at 2pm and all my files are locked in there. My number is
+1 206-555-0147 if it's faster to call. Account is under this email. Please help ASAP.
Also btw the mobile app just spins forever on the loading screen. Thanks, Daniel
```

### Step 3: The saved workflow runs automatically

The forward matches the rule you stored, so the assistant runs the three ZeroGPU skills in order on its own, each on a nano model. You don't send these; they are the tool calls it makes behind the scenes:

1. **`redact-pii`** (`gliner2.5-multi-v1`) masks the personal data before anything else touches the text:

```
Hey, I've tried logging in ... My number is [PHONE_NUMBER] if it's faster to call.
Account is under [EMAIL]. Also btw the mobile app just spins forever on the loading
screen. Thanks, [PERSON]
```

2. **`summarize`** (`llama-3.1-8b-instruct-fast`) condenses the masked text:

```
Customer cannot log in since this morning ("session expired") despite two password
resets, and the mobile app hangs on load. Time-sensitive: client demo at 2pm.
```

3. **`classify-structured`** (`gliner2.5-base-v1`) tags it against your label schema:

```json theme={null}
{ "topic": "account", "urgency": "high" }
```

Because redaction runs first, the customer's name, email, and phone never enter the summary, the labels, or your primary model's context.

### Step 4: The reply you get

The assistant returns the finished result in a single message:

```
🧾 Support triage

Summary: Customer cannot log in since this morning ("session expired") despite two
password resets, and the mobile app hangs on load. Time-sensitive: client demo at 2pm.

Topic: account   ·   Urgency: high
PII redacted ✓ (name, email, phone)

💰 ZeroGPU savings so far: ≈ $3 (24,100 frontier-model tokens offloaded)
```

One forward in, one clean result out - no instruction, because the workflow already lives in the assistant's memory. Your primary model only gets pulled in for the genuinely ambiguous cases.

### Step 5: Go fully hands-off (optional)

To remove even the forward, connect a shared support channel as an OpenClaw channel. The saved workflow then fires on every message that arrives, so the channel stays triaged with no action from you at all. Check the running savings any time:

```text theme={null}
how much have I saved with ZeroGPU?
```

```
ZeroGPU cost savings
--------------------
Calls offloaded:   312
Tokens offloaded:  24,100
Estimated savings: ≈ $3.00  (baseline: claude-opus-4-8)
```

🎉 You set the workflow up once, saved it to your assistant's memory, and now every forwarded message is redacted, summarized, and labeled automatically on ZeroGPU's nano models, with the customer's personal data never reaching your primary model.

## 🌟 Highlights

This notebook has guided you through setting up and running an OpenClaw workflow with ZeroGPU for triaging forwarded messages into PII-safe, labeled summaries. You can adapt and expand this example for various other scenarios requiring fast, low-cost handling of high-volume, free-text messages.

Key tools utilized in this notebook include:

* **OpenClaw**: A self-hosted, open-source personal AI assistant that connects the chat apps you already use - Telegram, Slack, Discord, WhatsApp, iMessage, and more - to AI agents and tools through a plugin system. Here it receives the messages you forward in your channel of choice and routes the cheap summarize, classify, and redact steps to ZeroGPU instead of your primary model.
* **ZeroGPU**: An ultra-fast, compute-efficient inference provider for apps and agents. We run purpose-built small and nano language models across an edge-powered network for the high-volume, purpose-specific tasks your app or agent runs constantly. Plug in our OpenAI-compatible API and you're live - zero GPU infrastructure, serverless, auto-scaling by default.

This comprehensive setup allows you to adapt and expand the example for various scenarios requiring fast, low-cost handling of high-volume, free-text messages.


# Ad Tech
Source: https://docs.zerogpu.ai/docs/ad-tech

Edge IAB classifiers that map content to taxonomy, audience, and intent signals for programmatic advertising.

Ad tech models map content to the IAB Content Taxonomy and the audience signals that programmatic advertising runs on. ZeroGPU offers a base IAB classifier tuned for real-time categorization at the edge, an enriched variant that adds topics, keywords, and inferred intent, and a signal extractor that turns free text into structured targeting signals. Each request is shown for both the [Responses API](/api-reference/responses) and the OpenAI-compatible [Chat Completions API](/api-reference/chat-completions).

| Model | Input /1M | Output /1M | Max tokens |
| - | -: | -: | -: |
| <a href="/api-reference/models/zlm-v1-iab-classify-edge"><img alt="zlm-v1-iab-classify-edge" /> <code>zlm-v1-iab-classify-edge</code></a> | \$0.02 | \$0.05 | 400 |
| <a href="/api-reference/models/zlm-v2-iab-classify-edge-enriched"><img alt="zlm-v2-iab-classify-edge-enriched" /> <code>zlm-v2-iab-classify-edge-enriched</code></a> | \$0.05 | \$0.15 | 800 |
| <a href="/api-reference/models/zlm-v1-iab-domain-classifier"><img alt="zlm-v1-iab-domain-classifier" /> <code>zlm-v1-iab-domain-classifier</code></a> | \$0.02 | \$0.05 | 100 |
| <a href="/api-reference/models/zlm-v1-signal-extract"><img alt="zlm-v1-signal-extract" /> <code>zlm-v1-signal-extract</code></a> | \$0.04 | \$0.10 | 400 |

## zlm-v1-iab-classify-edge

> ZeroGPU's IAB classifier maps any text to the industry-standard IAB Content Taxonomy in a single, fast inference call. Each call returns categories across both the 1.0 and 2.2 taxonomies plus matched audience segments, every result scored for confidence. With support for 50+ languages, it classifies multilingual content without a translation step. At 90M parameters on ONNX, it's built for the high-volume, sub-millisecond classification that ad tech and content platforms demand, right at the edge. When you need clean category and audience signals on every request, this is the model — and when you need the full profile, reach for the enriched variant.

**References:** [Model docs](https://docs.zerogpu.ai/) • [Terms](https://zerogpu.ai/terms) • [Privacy](https://zerogpu.ai/privacy-policy)

<CodeGroup>
  ```bash Responses API theme={null}
  curl https://api.zerogpu.ai/v1/responses \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "input": "Technology has quietly reshaped the rhythm of everyday life, weaving itself into routines so seamlessly that it often goes unnoticed. From the moment a smartphone alarm wakes someone in the morning to the final glance at a glowing screen before sleep, digital systems guide communication, navigation, work, and entertainment. This transformation did not happen overnight. It emerged through decades of incremental innovation, each new tool building upon the last, until the modern world became deeply interconnected.One of the most significant changes has been the speed at which information travels.",
    "model": "zlm-v1-iab-classify-edge"
  }'
  ```

  ```bash Chat Completions theme={null}
  curl https://api.zerogpu.ai/v1/chat/completions \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "model": "zlm-v1-iab-classify-edge",
    "messages": [
      {
        "role": "user",
        "content": "What are the basics of exercise and physical fitness?Exercise is anything that gets your body moving. Regular exercise is one of the best things you can do for your health. It has many benefits, including improving your overall health and fitness, and reducing your risk for many chronic (long-term) diseases.Every physical fitness routine is built on a few simple ideas. These include:Make exercise a habit, as your body adapts to the type of activity you do most often. Regular practice will help you improve.Build up your activity level slowly to help you continue to get stronger, faster, or more flexible without pushing too hard all at once.Challenge yourself by lifting slightly heavier weights, adding a few more minutes to your walk, or increasing your pace."
      }
    ]
  }'
  ```
</CodeGroup>

Returns IAB `audience` categories plus `content` matches across taxonomy versions, each with a confidence score.

```json Response theme={null}
{
  "audience": [
    { "name": "Technology & Computing", "score": 0.7686 },
    { "name": "Consumer Electronics", "score": 0.7206 },
    { "name": "65-69", "score": 0.5950 }
  ],
  "content": {
    "iab_1_0": [
      { "name": "Cell Phones", "score": 0.7597 },
      { "name": "Technology & Computing", "score": 0.7387 }
    ],
    "iab_2_2": [
      { "name": "Smartphones", "score": 0.7597 },
      { "name": "Wearable Technology", "score": 0.7387 },
      { "name": "Technology & Computing", "score": 0.7109 }
    ]
  }
}
```

## zlm-v2-iab-classify-edge-enriched

> The enriched variant of ZeroGPU's IAB classifier turns a single inference call into a full content-intelligence profile not just a label, but everything a contextual pipeline needs to act on. Each call returns IAB categories across both the 1.0 and 2.2 taxonomies (down to tier-3), audience and interest segments, topics, keywords, and user-intent classification — all with confidence scores with edge-native, sub-millisecond speeds. With support for 50+ languages, it delivers the same full profile on multilingual content without a translation step. Built for contextual ad targeting, brand-safety scoring, publisher content categorization, and signal pipelines that need full metadata on every request.

**References:** [Model docs](https://docs.zerogpu.ai/) • [Terms](https://zerogpu.ai/terms) • [Privacy](https://zerogpu.ai/privacy-policy)

<CodeGroup>
  ```bash Responses API theme={null}
  curl https://api.zerogpu.ai/v1/responses \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "input": "Technology has quietly reshaped the rhythm of everyday life, weaving itself into routines so seamlessly that it often goes unnoticed. From the moment a smartphone alarm wakes someone in the morning to the final glance at a glowing screen before sleep, digital systems guide communication, navigation, work, and entertainment. This transformation did not happen overnight. It emerged through decades of incremental innovation, each new tool building upon the last, until the modern world became deeply interconnected.One of the most significant changes has been the speed at which information travels.",
    "model": "zlm-v2-iab-classify-edge-enriched"
  }'
  ```

  ```bash Chat Completions theme={null}
  curl https://api.zerogpu.ai/v1/chat/completions \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "model": "zlm-v2-iab-classify-edge-enriched",
    "messages": [
      {
        "role": "user",
        "content": "What are the basics of exercise and physical fitness?Exercise is anything that gets your body moving. Regular exercise is one of the best things you can do for your health. It has many benefits, including improving your overall health and fitness, and reducing your risk for many chronic (long-term) diseases.Every physical fitness routine is built on a few simple ideas. These include:Make exercise a habit, as your body adapts to the type of activity you do most often. Regular practice will help you improve.Build up your activity level slowly to help you continue to get stronger, faster, or more flexible without pushing too hard all at once.Challenge yourself by lifting slightly heavier weights, adding a few more minutes to your walk, or increasing your pace."
      }
    ]
  }'
  ```
</CodeGroup>

In addition to scored `audience` and `content` categories, the enriched model returns taxonomy codes and parent IDs, plus `topics`, `keywords`, and an inferred `user_intent`.

```json Response theme={null}
{
  "audience": [
    {
      "id": 687,
      "parent_id": 206,
      "name": "Technology & Computing",
      "tier1_name": "Interest",
      "tier2_name": "Technology & Computing",
      "score": 0.7686
    },
    {
      "id": 703,
      "parent_id": 687,
      "name": "Consumer Electronics",
      "tier1_name": "Interest",
      "tier2_name": "Technology & Computing",
      "tier3_name": "Consumer Electronics",
      "score": 0.7206
    }
  ],
  "content": {
    "iab_1_0": [
      { "code": "IAB19-6", "name": "Cell Phones", "tier": 2, "parent_code": "IAB19", "score": 0.7597 },
      { "code": "IAB19", "name": "Technology & Computing", "tier": 1, "parent_code": null, "score": 0.7387 }
    ]
  },
  "topics": [
    { "name": "smartphones", "score": 0.7597 },
    { "name": "wearable technology", "score": 0.7387 },
    { "name": "technology & computing", "score": 0.7109 }
  ],
  "keywords": ["technology", "reshaped", "everyday", "routines", "quietly", "weaving", "rhythm", "itself", "life", "into"],
  "user_intent": {
    "name": "mastering technology quietly reshaped rhythm everyday life",
    "category": "informational",
    "score": 0.5
  }
}
```

## zlm-v1-iab-domain-classifier

> ZeroGPU's Domain IAB Classifier maps a raw domain name straight to the IAB Content Taxonomy in a single, fast inference call, returning content categories, topics, keywords, and user-intent signals. Because it needs only the domain as input, it reduces payload size by up to 10x compared to page-level classification workflows while still surfacing the contextual signals that bidstream enrichment, contextual targeting, and domain-level intelligence pipelines run on. At 149M parameters on ONNX, it's built for the high-volume, sub-millisecond classification that ad tech and content platforms demand, right at the edge.

**References:** [Model docs](https://docs.zerogpu.ai/) • [Terms](https://zerogpu.ai/terms) • [Privacy](https://zerogpu.ai/privacy-policy)

### Domain-level classification

Page-level classifiers like `zlm-v1-iab-classify-edge` and `zlm-v2-iab-classify-edge-enriched` take the full body of an article or page as input and map that content to the taxonomy. `zlm-v1-iab-domain-classifier` works one level up: it takes only the raw domain name (for example `nytimes.com`) and infers the categories, topics, keywords, and intent that characterize the site as a whole.

That shift has two practical effects:

* **Up to 10x smaller payloads.** You send a single hostname instead of crawling and shipping page text, which makes it well suited to bidstream enrichment, where the domain is often the only contextual signal available and per-request size matters.
* **No page fetch required.** Use it when you have the domain but not the rendered content — domain-level intelligence pipelines, allow/deny list scoring, and contextual targeting at the inventory level. When you do have the page text and need per-URL precision, reach for the page-level classifiers instead.

<CodeGroup>
  ```bash Responses API theme={null}
  curl https://api.zerogpu.ai/v1/responses \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "input": "nytimes.com",
    "model": "zlm-v1-iab-domain-classifier"
  }'
  ```

  ```bash Chat Completions theme={null}
  curl https://api.zerogpu.ai/v1/chat/completions \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "model": "zlm-v1-iab-domain-classifier",
    "messages": [
      {
        "role": "user",
        "content": "nytimes.com"
      }
    ]
  }'
  ```
</CodeGroup>

Returns scored `content` categories across taxonomy versions plus `topics`, `keywords`, and an inferred `user_intent` for the domain.

```json Response theme={null}
{
  "content": {
    "iab_1_0": [
      { "name": "News", "score": 0.8124 },
      { "name": "Politics", "score": 0.6533 }
    ],
    "iab_2_2": [
      { "name": "News and Politics", "score": 0.8124 },
      { "name": "Business and Finance", "score": 0.5980 }
    ]
  },
  "topics": [
    { "name": "news", "score": 0.8124 },
    { "name": "journalism", "score": 0.6740 },
    { "name": "current events", "score": 0.6533 }
  ],
  "keywords": ["news", "politics", "world", "business", "opinion", "media", "journalism"],
  "user_intent": {
    "name": "reading news and current events",
    "category": "informational",
    "score": 0.5
  }
}
```

## zlm-v1-signal-extract

> ZeroGPU's signal extractor turns unstructured text into structured signals that downstream systems can act on. One inference call returns topics, keywords, intent, and other contextual attributes as structured output, so content enrichment, contextual intelligence, ad targeting, agent routing, recommendation systems, and analytics pipelines all read from the same pass. At 80M parameters it is built for high-volume workloads where a general-purpose LLM is more machinery than the job needs.

**References:** [Terms](https://zerogpu.ai/terms) • [Privacy](https://zerogpu.ai/privacy-policy)

Pass the signal categories you want in `categories`; `max_keywords` caps how many keywords come back.

<CodeGroup>
  ```bash Responses API theme={null}
  curl https://api.zerogpu.ai/v1/responses \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "model": "zlm-v1-signal-extract",
    "input": "How can I pay using my credit card?",
    "categories": [
      "signals"
    ]
  }'
  ```

  ```bash Chat Completions theme={null}
  curl https://api.zerogpu.ai/v1/chat/completions \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "model": "zlm-v1-signal-extract",
    "messages": [
      {
        "role": "user",
        "content": "How can I pay using my credit card?"
      }
    ],
    "categories": [
      "signals"
    ],
    "max_keywords": 6
  }'
  ```
</CodeGroup>

The signals arrive as a JSON string in the output text. Parsed:

```json Response theme={null}
{
  "keywords": [
    "credit card",
    "login",
    "official website",
    "account access"
  ],
  "user_intent": {
    "name": "pay using credit card",
    "category": "navigational",
    "score": 0.95
  }
}
```


# Audio Models
Source: https://docs.zerogpu.ai/docs/audio

Open-weight speech-to-text and text-to-speech models on ZeroGPU, billed per minute of audio.

ZeroGPU serves two open-weight audio models on OpenAI-compatible endpoints: `whisper-tiny` for speech-to-text on [`POST /v1/audio/transcriptions`](/api-reference/audio-transcriptions), and `chatterbox-nano` for text-to-speech, with voice cloning, on [`POST /v1/audio/speech`](/api-reference/audio-speech). Authenticate with `x-api-key` or `Authorization: Bearer`, so the OpenAI SDKs work with only the base URL and API key changed.

## At a glance

| Model | Price /min | Task |
| - | -: | - |
| <a href="/api-reference/models/whisper-tiny"><img alt="whisper-tiny" /> <code>whisper-tiny</code></a> | \$0.02 | Speech-to-Text |
| <a href="/api-reference/models/chatterbox-nano"><img alt="chatterbox-nano" /> <code>chatterbox-nano</code></a> | \$0.01 | Text-to-Speech |

Audio is billed per minute, rounded up to the next whole minute with a one-minute minimum: uploaded audio for `whisper-tiny`, generated audio for `chatterbox-nano`. The billed length is returned in the `x-audio-duration-seconds` header.

## Model cards

<CardGroup>
  <Card href="/api-reference/models/whisper-tiny">
    <div>
      <img alt="whisper-tiny" />

      <div>whisper-tiny</div>
    </div>

    <div><span>Speech-to-Text</span><span>39M params</span><span>\$0.02 / minute</span></div>

    OpenAI's Whisper Tiny is the smallest model in the Whisper family of speech recognition models, at 39M parameters. It is the multilingual checkpoint: it transcribes speech in the language it was spoken…
  </Card>

  <Card href="/api-reference/models/chatterbox-nano">
    <div>
      <img alt="chatterbox-nano" />

      <div>chatterbox-nano</div>
    </div>

    <div><span>Text-to-Speech</span><span>Voice cloning</span><span>110M params</span><span>\$0.01 / minute</span></div>

    Resemble AI's Chatterbox Nano is a 110M-parameter English text-to-speech model built for fast, low-cost speech generation. It speaks your text in a built-in voice, or clones a speaker…
  </Card>
</CardGroup>

## whisper-tiny

> OpenAI's Whisper Tiny is the smallest model in the Whisper family of speech recognition models, at 39M parameters. It is the multilingual checkpoint: it transcribes speech in the language it was spoken, detecting that language on its own when you don't name it, and it can translate speech into English. Its size makes it fast and cheap enough for high-volume transcription where throughput and cost matter more than the accuracy of the larger Whisper models: voice agents, meeting notes, subtitles, podcasts, and audio indexing.

**References:** [Model card](https://huggingface.co/openai/whisper-tiny) • [License](https://github.com/openai/whisper/blob/main/LICENSE) • [Terms](https://zerogpu.ai/terms) • [Privacy](https://zerogpu.ai/privacy-policy)

<CodeGroup>
  ```bash cURL theme={null}
  curl --location 'https://api.zerogpu.ai/v1/audio/transcriptions' \
    --header 'x-api-key: YOUR_API_KEY' \
    --form 'model="whisper-tiny"' \
    --form 'file=@"/path/to/audio.mp3"'
  ```

  ```python OpenAI SDK theme={null}
  from openai import OpenAI

  client = OpenAI(
      base_url="https://api.zerogpu.ai/v1",
      api_key="YOUR_API_KEY",  # sent as Authorization: Bearer
  )

  with open("/path/to/audio.mp3", "rb") as audio:
      transcript = client.audio.transcriptions.create(
          model="whisper-tiny",
          file=audio,
      )

  print(transcript.text)
  ```
</CodeGroup>

```json Response theme={null}
{
  "text": "the quick brown fox jumps over the lazy dog."
}
```

See [Transcriptions](/api-reference/audio-transcriptions) for all fields, response formats, and limits.

## chatterbox-nano

> Resemble AI's Chatterbox Nano is a 110M-parameter English text-to-speech model built for fast, low-cost speech generation. It speaks your text in a built-in voice, or clones a speaker from a short reference clip you upload with the request, and inline tags such as `[laugh]`, `[chuckle]`, and `[cough]` add paralinguistic cues. Its compact size keeps generation cheap enough for high-volume work: voice agents, conversational AI, gaming, accessibility, audiobooks, and content creation.

**References:** [Model card](https://huggingface.co/ResembleAI/chatterbox-nano) • [License](https://github.com/resemble-ai/chatterbox/blob/master/LICENSE) • [Terms](https://zerogpu.ai/terms) • [Privacy](https://zerogpu.ai/privacy-policy)

<CodeGroup>
  ```bash cURL (voice cloning) theme={null}
  curl --location 'https://api.zerogpu.ai/v1/audio/speech' \
    --header 'x-api-key: YOUR_API_KEY' \
    --form 'model="chatterbox-nano"' \
    --form 'input="Testing model to generate voice. Hey how are you today?"' \
    --form 'response_format="mp3"' \
    --form 'seed="1234"' \
    --form 'voice_sample=@"/path/to/file"' \
    --output speech.mp3
  ```

  ```python OpenAI SDK theme={null}
  from openai import OpenAI

  client = OpenAI(
      base_url="https://api.zerogpu.ai/v1",
      api_key="YOUR_API_KEY",  # sent as Authorization: Bearer
  )

  with client.audio.speech.with_streaming_response.create(
      model="chatterbox-nano",
      voice="alloy",  # OpenAI voice names map to the built-in voice
      input="Hello from ZeroGPU.",
      response_format="mp3",
  ) as response:
      response.stream_to_file("speech.mp3")
  ```
</CodeGroup>

Response from a test run of the SDK request above:

```text Response theme={null}
HTTP/2 200
content-type: audio/mpeg
x-audio-duration-seconds: 1.76

<binary MP3 audio>
```

To clone a voice, attach a 6 to 30 second reference clip as `voice_sample` in a `multipart/form-data` request, as in the cURL example. The OpenAI SDKs send JSON and can't attach a clip. See [Speech](/api-reference/audio-speech) for all fields and limits.


# Errors
Source: https://docs.zerogpu.ai/docs/batch/errors

Every HTTP status, every validation message, every JSONL error code, and how to recover.

This page is a catalog of every error code you can encounter when using the
Batch and Files APIs, what each one means, and what to do about it.

## Error response shape

All errors (other than CORS preflight) return JSON in the OpenAI-compatible
envelope:

```json theme={null}
{
  "error": {
    "message": "Human-readable description.",
    "type":    "invalid_request_error",
    "code":    "invalid_purpose",
    "param":   "purpose",
    "line":    5
  }
}
```

* `message` is always present.
* `type` is always present. Values mirror OpenAI's vocabulary:
  `invalid_request_error`, `authentication_error`, `rate_limit_exceeded`,
  `internal_error`, plus `insufficient_quota` on `429` responses.
* `code` is a machine-readable string (e.g. `invalid_purpose`,
  `file_not_found`, `insufficient_quota`). `null` when not meaningful.
* `param` names the offending parameter (`"model"`, `"messages"`,
  `"input_file_id"`, …) or is `null`.
* `line` is included for JSONL validation failures during
  `POST /v1/batches` to point at the offending line (1-based).

Every response, success or error, also carries an `x-request-id`
response header (format `req_<token>`). Pass it along when reporting
issues to support so the call can be located in logs.

## HTTP status codes

| Status | Meaning |
| - | - |
| `200` | Success. |
| `204` | Success with no body (CORS preflight). |
| `400` | Your request is malformed or fails validation. Fix and retry. |
| `401` | Missing `x-api-key` header. |
| `403` | Headers were supplied but credentials are invalid, inactive, or don't belong to the supplied project. |
| `404` | The resource doesn't exist, or doesn't belong to your project. Both cases return `404` to avoid leaking cross-project information. |
| `413` | Uploaded file is larger than 100 MB. |
| `429` | Rate-limit / insufficient quota. The `error.type` distinguishes: `insufficient_quota` (no balance and no free credits) vs `rate_limit_exceeded` (transient throttling). |
| `500` | Internal server error. Retry with exponential backoff; if persistent, contact support. |

<Note>
  **Quota errors return <code>429</code>**

  Catch <code>429</code> for both quota exhaustion (<code>error.type:
  "insufficient\_quota"</code>) and rate-limiting (<code>error.type:
  "rate\_limit\_exceeded"</code>). This matches OpenAI's behaviour and works
  with the official SDK's built-in retry logic.
</Note>

## Common synchronous errors

### `401 Unauthorized`, missing headers

```json theme={null}
{ "error": { "message": "Missing authentication headers" } }
```

**Cause:** `x-api-key` was not sent.
**Fix:** Send the required `x-api-key` header on every request. `x-project-id`
is optional, and only needed to scope the request to a specific project.

### `403 Forbidden`, invalid credentials

```json theme={null}
{ "error": { "message": "Invalid credentials" } }
```

**Cause:** The API key doesn't exist, is inactive, or belongs to a different
project than the one in `x-project-id`.
**Fix:** Verify the key, and verify the project UUID matches.

### `429 Insufficient Quota`

```json theme={null}
{
  "error": {
    "message": "You exceeded your current quota, please check your plan and billing details.",
    "type":    "insufficient_quota",
    "code":    "insufficient_quota",
    "param":   null
  }
}
```

**Cause:** Your organization has zero balance and zero free credits.
**Fix:** Top up your balance.

> **Important:** quota is checked at line-execution time. If your balance
> drops to zero **during** a batch, the remaining lines fail with
> `error.code: "insufficient_quota"` in the error file (the JSONL line's
> `error.message` includes the legacy `[legacy:http_420] ` prefix for
> forensics). The batch as a whole still transitions to `completed` once
> all lines have been accounted for.

### `404 Not Found`, file or batch

```json theme={null}
{ "error": { "message": "File file-xxx not found" } }
```

```json theme={null}
{ "error": { "message": "Batch not found: batch_xxx" } }
```

**Cause:** The ID does not exist, or it belongs to another project, or it
has been soft-deleted. The same `404` is returned in all three cases.
**Fix:** Verify the ID and the `x-project-id` you're sending.

***

## Files API errors

| Status | Code | When |
| - | - | - |
| `400` | `invalid_content_type` | Upload request was not `multipart/form-data`. |
| `400` | `invalid_multipart` | The multipart body could not be parsed. |
| `400` | `invalid_purpose` | `purpose` was anything other than `"batch"`. |
| `400` | `missing_file` | No `file` field in the multipart body. |
| `400` | `empty_file` | The uploaded file had zero bytes. |
| `400` | `invalid_limit` | List `limit` query parameter was not a positive integer. |
| `404` | `file_not_found` | File does not exist or is not accessible. |
| `413` | `file_too_large` | File exceeded 100 MB. |
| `500` | `storage_error` | Backing storage error; retry. |

***

## Batches API. `POST /v1/batches` validation errors

When you create a batch, the service streams the entire input JSONL and
validates every line **synchronously** before returning a response. Any
failure returns `400` with the offending `line` (1-based, mirroring
OpenAI's `BatchError.line`). The batch is also persisted with
`status: "failed"` and a populated `errors` list, so subsequent
`GET /v1/batches/{id}` calls can introspect what went wrong.

| `message` | Cause |
| - | - |
| `input_file_id is required` | Missing or non-string `input_file_id` field. |
| `endpoint is required` | Missing or non-string `endpoint` field. |
| `completion_window must be "24h"` | Anything other than `"24h"` was supplied. |
| `Input file not found: <id>` | `input_file_id` doesn't exist or isn't in this project. (404) |
| `Input file object missing: <id>` | DB row exists but the stored object is gone. Report to support. (404) |
| `Input file contains no JSONL lines` | Uploaded file is empty or only whitespace. |
| `Input file exceeds maximum size of 209715200 bytes` | File total > 200 MB. |
| `Input file exceeds maximum of 50000 lines` | More than 50,000 lines. |
| `Line N is not valid JSON` | Line N could not be parsed as JSON. |
| `Line N must be a JSON object` | Line N parsed as an array or primitive. |
| `Line N exceeds maximum size of 1048576 bytes` | Line N > 1 MB. |
| `Line N is missing custom_id` | Missing or non-string `custom_id`. |
| `Line N duplicates custom_id "X"` | A previous line already used custom\_id `X`. |
| `Line N method must be "POST"` | `method` (case-insensitive) was not `"POST"`. |
| `Line N is missing url` | Missing or non-string `url`. |
| `Line N url "X" is not an allowed batch endpoint` | `url` is not `/v1/chat/completions`, the only [supported batch endpoint](/docs/batch/supported-endpoints). |
| `Line N url "X" does not match the batch endpoint "Y"` | Lines in the file disagree on `url`. |
| `Line N is missing body` | `body` is not a non-empty object. |
| `Line N has stream=true; streaming is not supported in batch mode` | `body.stream === true`. |
| `endpoint "X" does not match the url "Y" used by the input file` | The `endpoint` field on the create request disagrees with what's in the file. |

Validation errors carry `line` in the JSON body so you can locate and
fix the offending line:

```json theme={null}
{
  "error": {
    "message": "Line 5 duplicates custom_id \"req-1\"",
    "type":    "invalid_request_error",
    "code":    "invalid_request_error",
    "param":   null,
    "line":    5
  }
}
```

***

## Errors that appear in the `error_file_id` JSONL

A line can fail **after** the batch was successfully created. Each failed
line is recorded in the error file with the line's original `custom_id` and
an `error` object. These are the codes you'll see:

| `error.code` | Triggered when | Recovery |
| - | - | - |
| `invalid_request_error` | The underlying endpoint rejected the line's body (bad shape, missing field, schema violation). Legacy prefix on `error.message`: `[legacy:http_400]` or `[legacy:http_422]`. | Fix the line's `body` and re-submit just that line. |
| `authentication_error` | The API key used to submit the batch lost permission mid-flight, was deactivated, or otherwise became unusable. Legacy: `[legacy:http_401]` / `[legacy:http_403]` / `[legacy:api_key_unavailable]`. | Verify key status; issue a new key if needed. |
| `not_found_error` | A resource referenced in the body (e.g. model ID) doesn't exist. Legacy: `[legacy:http_404]`. | Verify model IDs. |
| `request_too_large` | The line body exceeded the per-request size or token budget. Legacy: `[legacy:http_413]`. | Reduce input length. |
| `insufficient_quota` | Organization quota ran out partway through the batch. Legacy: `[legacy:http_420]` / `[legacy:http_429]`. | Top up balance and re-submit the failed lines. |
| `rate_limit_exceeded` | The endpoint throttled the line (non-quota rate limit). | Re-submit the line; transient. |
| `internal_error` | The upstream service returned 5xx on every retry, or the queue exhausted delivery attempts. Legacy: `[legacy:http_5xx]` / `[legacy:retries_exhausted]` / `[legacy:dlq_exhausted]`. | Re-submit the line; transient. |
| `batch_cancelled` | The parent batch was cancelled via `POST /v1/batches/{id}/cancel` before this line was dispatched. | The line did not run; submit a fresh batch if you still need the result. |
| `batch_expired` | The 24-hour completion window elapsed before this line was dispatched. | Submit a fresh batch with the remaining lines. |

The original ZeroGPU code is preserved as a `[legacy:<old_code>] ` prefix
on `error.message` for forensics. No `error.details` field is emitted -
the relevant upstream detail (if any) is folded into the message.

***

## Recovering from failures

A batch that ends with some failures is **still** `status: "completed"`.
The pattern for handling this is:

1. Read `output_file_id` and write successful results to your data store.
2. Read `error_file_id` and partition by `error.code`:
   * `invalid_request_error`, `not_found_error`, `request_too_large`:
     client-side issues, log, fix the input lines, and resubmit.
   * `internal_error`, `rate_limit_exceeded`: transient, resubmit as-is.
   * `insufficient_quota`, `authentication_error`: operational, fix the
     underlying issue first, then resubmit.
   * `batch_cancelled`, `batch_expired`: the lines never ran; resubmit
     them in a fresh batch.
3. Build a smaller JSONL containing only the lines you want to retry, upload,
   and create a new batch.

Because `custom_id` is preserved end-to-end, retries are easy to deduplicate
into your existing data store.

***

## When to contact support

Reach out to ZeroGPU support if you see:

* Persistent `500` responses from any endpoint.
* `Input file object missing` on a file you just uploaded.
* A batch stuck in `finalizing` for more than 10 minutes.
* A batch that completes but is missing both `output_file_id` and
  `error_file_id` while `request_counts.total` is non-zero.

## Next steps

<CardGroup>
  <Card title="Objects & lifecycle →" href="/docs/batch/objects">
    The Batch object, status lifecycle, and every endpoint.
  </Card>

  <Card title="JSONL format →" href="/docs/batch/jsonl-format">
    Line schemas for input, output, and error files.
  </Card>

  <Card title="Quickstart →" href="/docs/batch/getting-started">
    First batch in under 10 minutes, auth, upload, create, download.
  </Card>
</CardGroup>


# Quickstart
Source: https://docs.zerogpu.ai/docs/batch/getting-started

First batch in under 10 minutes, auth, upload, create, poll, download.

This page walks through a complete batch from start to finish in both `curl`
and the Python `openai` SDK. By the end you'll have submitted 3 chat
completions through the Batch API and read the results.

## Prerequisites

| Credential | Where to find it |
| - | - |
| **API key** (`x-api-key`) | ZeroGPU dashboard → API Keys |
| **Project ID** (`x-project-id`) | ZeroGPU dashboard → Projects |

Both headers are required on every request. Missing either returns `401`.

<Warning>
  **Keep your API key out of source control**

  Store it in environment variables, secret managers, or your CI's secret
  store, never commit it.
</Warning>

```bash theme={null}
export ZGPU_API_KEY="your-api-key"
export ZGPU_PROJECT_ID="your-project-uuid"
```

## Base URLs

| Environment | URL |
| - | - |
| Production | `https://api.zerogpu.ai` |
| Staging | `https://staging.api.zerogpu.ai` |
| Development | `https://dev.api.zerogpu.ai` |

The Batch and Files endpoints live under these hostnames:

* `POST   /v1/files`, `GET /v1/files`, `GET /v1/files/{id}`, `GET /v1/files/{id}/content`, `DELETE /v1/files/{id}`
* `POST   /v1/batches`, `GET /v1/batches`, `GET /v1/batches/{batch_id}`

***

## 1. Build the input JSONL

Every batch is driven by a JSONL file where each line is one inference
request:

```json theme={null}
{"custom_id": "req-1", "method": "POST", "url": "/v1/chat/completions", "body": { ... }}
```

| Field | Required | Description |
| - | - | - |
| `custom_id` | Yes | Your identifier for the request. Must be **unique within the batch**. Echoed back in the output so you can match results to inputs. |
| `method` | Yes | Must be `"POST"`. |
| `url` | Yes | Must be `/v1/chat/completions`, the only [supported batch endpoint](/docs/batch/supported-endpoints). All lines in a batch must share the same `url`. |
| `body` | Yes | The JSON body you would send to that endpoint synchronously. `"stream": true` is rejected, batches are non-streaming. |

Full schema and validation rules: [JSONL format](/docs/batch/jsonl-format).

<CodeGroup>
  ```bash cURL theme={null}
  cat > input.jsonl <<'EOF'
  {"custom_id":"q-1","method":"POST","url":"/v1/chat/completions","body":{"model":"<model-id>","messages":[{"role":"user","content":"What is the capital of France?"}]}}
  {"custom_id":"q-2","method":"POST","url":"/v1/chat/completions","body":{"model":"<model-id>","messages":[{"role":"user","content":"What is the capital of Germany?"}]}}
  {"custom_id":"q-3","method":"POST","url":"/v1/chat/completions","body":{"model":"<model-id>","messages":[{"role":"user","content":"What is the capital of Italy?"}]}}
  EOF
  ```

  ```python Python theme={null}
  import json

  questions = [
      ("q-1", "What is the capital of France?"),
      ("q-2", "What is the capital of Germany?"),
      ("q-3", "What is the capital of Italy?"),
  ]

  with open("input.jsonl", "w") as f:
      for custom_id, question in questions:
          f.write(json.dumps({
              "custom_id": custom_id,
              "method":    "POST",
              "url":       "/v1/chat/completions",
              "body": {
                  "model":    "<model-id>",
                  "messages": [{"role": "user", "content": question}],
              },
          }) + "\n")
  ```
</CodeGroup>

## 2. Upload the file

Send the JSONL to `POST /v1/files` with `purpose=batch`. The response
contains the `file_id` you'll reference when creating the batch.

<CodeGroup>
  ```bash cURL theme={null}
  curl -X POST https://api.zerogpu.ai/v1/files \
    -H "x-api-key: $ZGPU_API_KEY" \
    -H "x-project-id: $ZGPU_PROJECT_ID" \
    -F purpose=batch \
    -F file=@input.jsonl
  ```

  ```python Python theme={null}
  from openai import OpenAI

  client = OpenAI(
      api_key="ignored-by-zerogpu",
      base_url="https://api.zerogpu.ai/v1",
      default_headers={
          "x-api-key":    os.environ["ZGPU_API_KEY"],
          "x-project-id": os.environ["ZGPU_PROJECT_ID"],
      },
  )

  uploaded = client.files.create(
      file=open("input.jsonl", "rb"),
      purpose="batch",
  )
  print(uploaded.id)  # file-abc...
  ```
</CodeGroup>

Response:

```json theme={null}
{
  "id":         "file-abc123...",
  "object":     "file",
  "bytes":      612,
  "created_at": 1736290000,
  "filename":   "input.jsonl",
  "purpose":    "batch",
  "status":     "processed",
  "expires_at": 1738882000
}
```

## 3. Create the batch

Submit the batch with the file ID, the target endpoint, and a 24-hour
completion window. The response returns immediately with
`status: "in_progress"`, actual processing is asynchronous.

<CodeGroup>
  ```bash cURL theme={null}
  curl -X POST https://api.zerogpu.ai/v1/batches \
    -H "x-api-key: $ZGPU_API_KEY" \
    -H "x-project-id: $ZGPU_PROJECT_ID" \
    -H "content-type: application/json" \
    -d '{
      "input_file_id":     "file-abc123...",
      "endpoint":          "/v1/chat/completions",
      "completion_window": "24h"
    }'
  ```

  ```python Python theme={null}
  batch = client.batches.create(
      input_file_id=uploaded.id,
      endpoint="/v1/chat/completions",
      completion_window="24h",
      metadata={"job": "capitals-demo"},
  )
  print(batch.id, batch.status)  # batch_01HZX... in_progress
  ```
</CodeGroup>

Response:

```json theme={null}
{
  "id":             "batch_01HZX...",
  "object":         "batch",
  "endpoint":       "/v1/chat/completions",
  "status":         "in_progress",
  "input_file_id":  "file-abc123...",
  "output_file_id": null,
  "error_file_id":  null,
  "created_at":     1736290000,
  "expires_at":     1736376400,
  "request_counts": { "total": 3, "completed": 0, "failed": 0 }
}
```

<Note>
  **Validation runs at create time**

  The server streams the entire input JSONL and validates every line before
  responding. If anything is wrong, duplicate <code>custom\_id</code>, line
  over 1 MB, <code>stream: true</code>, mismatched <code>url</code>, you'll
  get a 400 with the offending <code>line</code>. Once the response
  returns, the batch is durably committed.
</Note>

## 4. Poll until complete

Poll `GET /v1/batches/{batch_id}` until `status` is `completed`, `expired`,
or `failed`. A 30-second interval is a reasonable default.

<CodeGroup>
  ```bash cURL theme={null}
  while true; do
    RESP=$(curl -s "https://api.zerogpu.ai/v1/batches/$BATCH_ID" \
      -H "x-api-key: $ZGPU_API_KEY" \
      -H "x-project-id: $ZGPU_PROJECT_ID")
    STATUS=$(echo "$RESP" | jq -r '.status')
    echo "$RESP" | jq -c '{status, request_counts}'
    case "$STATUS" in completed|failed|expired) break;; esac
    sleep 30
  done
  ```

  ```python Python theme={null}
  import time

  while batch.status not in ("completed", "failed", "expired"):
      time.sleep(30)
      batch = client.batches.retrieve(batch.id)
      print(batch.status, batch.request_counts)
  ```
</CodeGroup>

When `status` is `completed`, the response contains `output_file_id` (and,
if any line failed, `error_file_id`).

## 5. Download the results

Stream the output and error files via `GET /v1/files/{file_id}/content`.

<CodeGroup>
  ```bash cURL theme={null}
  curl -s "https://api.zerogpu.ai/v1/files/$OUTPUT_FILE_ID/content" \
    -H "x-api-key: $ZGPU_API_KEY" \
    -H "x-project-id: $ZGPU_PROJECT_ID" \
    -o output.jsonl

  jq -r '"\(.custom_id): \(.response.body.choices[0].message.content)"' output.jsonl
  ```

  ```python Python theme={null}
  if batch.output_file_id:
      data = client.files.content(batch.output_file_id).read()
      with open("output.jsonl", "wb") as f:
          f.write(data)

      for line in data.decode().splitlines():
          rec = json.loads(line)
          print(rec["custom_id"], rec["response"]["body"]["choices"][0]["message"]["content"])
  ```
</CodeGroup>

Output line shape (one per successful line, order **not preserved**, match
by `custom_id`):

```json theme={null}
{"id": "batch_req_a", "custom_id": "q-1", "response": {"status_code": 200, "request_id": "req_xyz", "body": { ... }}}
```

Error line shape (one per failed line):

```json theme={null}
{"id": "batch_req_b", "custom_id": "q-2", "response": null, "error": {"code": "invalid_request_error", "message": "[legacy:http_400] ...", "param": null}}
```

Full schemas: [JSONL format](/docs/batch/jsonl-format).

## Scale it up

The same five steps handle a real workload. This script sends 1,000 prompts
and collects the answers into a CSV, generating the JSONL from your own data,
polling once a minute, and matching results back by `custom_id`:

```python theme={null}
import csv, json, os, time
from openai import OpenAI

client = OpenAI(
    api_key="ignored-by-zerogpu",
    base_url="https://api.zerogpu.ai/v1",
    default_headers={
        "x-api-key":    os.environ["ZGPU_API_KEY"],
        "x-project-id": os.environ["ZGPU_PROJECT_ID"],
    },
)

# 1. Write JSONL from your data (anything yielding (doc_id, prompt) tuples)
prompts = load_my_dataset()  # 1000 records
with open("chat.jsonl", "w") as f:
    for doc_id, prompt in prompts:
        f.write(json.dumps({
            "custom_id": doc_id,
            "method":    "POST",
            "url":       "/v1/chat/completions",
            "body":      {"model": "<model-id>", "messages": [{"role": "user", "content": prompt}]},
        }) + "\n")

# 2-3. Upload + create
uploaded = client.files.create(file=open("chat.jsonl", "rb"), purpose="batch")
batch    = client.batches.create(
    input_file_id=uploaded.id,
    endpoint="/v1/chat/completions",
    completion_window="24h",
)

# 4. Poll
while batch.status not in ("completed", "failed", "expired", "cancelled"):
    time.sleep(60)
    batch = client.batches.retrieve(batch.id)
    print(f"{batch.status}: {batch.request_counts.completed}/{batch.request_counts.total}")

if batch.status != "completed":
    raise SystemExit(f"Batch ended with status {batch.status}")

# 5. Download, parse, write CSV
output = client.files.content(batch.output_file_id).read().decode()
results = {
    rec["custom_id"]: rec["response"]["body"]["choices"][0]["message"]["content"]
    for rec in (json.loads(line) for line in output.splitlines() if line.strip())
}
with open("results.csv", "w", newline="") as f:
    writer = csv.writer(f)
    writer.writerow(["doc_id", "answer"])
    writer.writerows(results.items())
```

For larger jobs, also read `error_file_id` and resubmit only the failed
lines, see [Errors reference](/docs/batch/errors) for the recovery pattern.

## Next steps

<CardGroup>
  <Card title="JSONL format →" href="/docs/batch/jsonl-format">
    Exact line schema for input, output, and error files.
  </Card>

  <Card title="Supported endpoints →" href="/docs/batch/supported-endpoints">
    Body and response shape for `/v1/chat/completions`.
  </Card>

  <Card title="Objects & lifecycle →" href="/docs/batch/objects">
    Status lifecycle, the full Batch object schema, and every endpoint.
  </Card>

  <Card title="Errors reference →" href="/docs/batch/errors">
    Recover from failed lines without re-running the whole batch.
  </Card>
</CardGroup>


# Overview
Source: https://docs.zerogpu.ai/docs/batch/index

Run large workloads of AI inference asynchronously, wire-compatible with `/v1/batches` and `/v1/files`.

The Batch API processes hundreds, thousands, or up to 50,000 inference
requests at a discounted rate within a 24-hour window. It's the right
choice when you don't need a real-time response and want to avoid per-request
rate limits.

<CardGroup>
  <Card title="Quickstart" href="/docs/batch/getting-started">
    Run your first batch end to end in curl and Python: upload, create, poll, download.
  </Card>

  <Card title="Objects & lifecycle" href="/docs/batch/objects">
    The File and Batch objects, the status lifecycle, and every endpoint.
  </Card>

  <Card title="JSONL format" href="/docs/batch/jsonl-format">
    Line schemas for input, output, and error files, plus validation rules.
  </Card>

  <Card title="Supported endpoints" href="/docs/batch/supported-endpoints">
    The one endpoint a batch line can target, with its request and response body.
  </Card>

  <Card title="Errors" href="/docs/batch/errors">
    Every status and error-file code, with recovery guidance.
  </Card>
</CardGroup>

## Quick facts

| | |
| - | - |
| **Base URL (production)** | `https://api.zerogpu.ai` |
| **Auth headers** | `x-api-key` (required), `x-project-id` (optional) |
| **Completion window** | 24 hours (fixed) |
| **Supported batch endpoint** | `/v1/chat/completions` (only) |
| **Max requests per batch** | 50,000 |
| **Max input file size** | 200 MB total, 1 MB per line |
| **Max upload size** | 100 MB |
| **File retention** | 30 days |

## When to use the Batch API

| You need… | Use |
| - | - |
| A single immediate response | The synchronous endpoint directly (e.g. `POST /v1/chat/completions`) |
| Hundreds-to-thousands of completions, can wait minutes-to-hours | The Batch API |
| To avoid per-second rate limits during a backfill | The Batch API |
| Streaming responses | The synchronous endpoint, streaming is **not** supported in batch mode |

Every endpoint also has an interactive playground under
[API Reference → Batch API](/api-reference/upload-file).


# JSONL Format
Source: https://docs.zerogpu.ai/docs/batch/jsonl-format

Mandatory line schemas for input, output, and error JSONL files.

Every batch is driven by three JSONL files:

| File | Direction | Purpose |
| - | - | - |
| **Input** | You upload it | One request per line; describes what the batch should do |
| **Output** | Service writes it | One result per successful line |
| **Error** | Service writes it | One record per failed line |

This page is the canonical specification for all three line schemas.

## Input JSONL

The input is a JSONL file (one JSON object per line, separated by `\n`). You
upload it via [`POST /v1/files`](/api-reference/upload-file)
with `purpose=batch`, then reference its ID when you call
[`POST /v1/batches`](/api-reference/create-batch).

### Line schema

```json theme={null}
{
  "custom_id": "request-1",
  "method":    "POST",
  "url":       "/v1/chat/completions",
  "body":      { /* endpoint-specific payload */ }
}
```

| Field | Type | Required | Description |
| - | - | - | - |
| `custom_id` | string | **Yes** | Your identifier for this request. Must be non-empty and **unique within the batch**. Returned verbatim in the corresponding output or error line so you can match results to inputs. |
| `method` | string | **Yes** | HTTP method, case-insensitive. Only `POST` (any case) is accepted today. |
| `url` | string | **Yes** | Must be `/v1/chat/completions`, the [only batchable endpoint](/docs/batch/supported-endpoints). **All lines in a single batch must share the same `url`**, and it must match the `endpoint` field on the create call. |
| `body` | object | **Yes** | The request body that would be sent to the synchronous endpoint. See [Supported endpoints](/docs/batch/supported-endpoints) for the shape. The field `stream: true` is rejected. |

### Validation rules

The service performs all of the following checks at `POST /v1/batches` time
(not at upload time). Failure returns `400` with the offending `line`:

* Each line must be valid JSON (not arrays, not primitives, must be an object).
* `custom_id` non-empty string, unique across the whole batch.
* `method` is case-insensitive; the upper-cased value must equal `"POST"`.
* `url` must be exactly `/v1/chat/completions` on every line in the file.
  Other ZeroGPU sync routes are rejected here.
* `body` must be a non-empty object.
* `body.stream === true` is rejected (streaming is not supported in batch
  mode).
* Per-line size ≤ **1 MB**.
* Total file size ≤ **200 MB**.
* Total line count ≤ **50,000**.
* The file must contain at least one line.

### Format details

* Lines are separated by `\n` (LF). A trailing newline is allowed.
* Blank lines are skipped.
* UTF-8 encoding is required.
* The `url` value matches the `endpoint` field on the create request
  **exactly**, no trailing slash, no query string, no protocol/host prefix.

### Example, chat completions

```jsonl theme={null}
{"custom_id":"req-1","method":"POST","url":"/v1/chat/completions","body":{"model":"<model-id>","messages":[{"role":"user","content":"What is the capital of France?"}]}}
{"custom_id":"req-2","method":"POST","url":"/v1/chat/completions","body":{"model":"<model-id>","messages":[{"role":"user","content":"What is the capital of Germany?"}]}}
{"custom_id":"req-3","method":"POST","url":"/v1/chat/completions","body":{"model":"<model-id>","messages":[{"role":"user","content":"What is the capital of Italy?"}]}}
```

See [Supported endpoints](/docs/batch/supported-endpoints) for the
`/v1/chat/completions` body schema.

***

## Output JSONL

When a batch reaches `status: "completed"`, the service writes the successful
results to a file referenced by `output_file_id` on the Batch object. You
download it via
[`GET /v1/files/{file_id}/content`](/api-reference/download-file).

Only lines that received a **2xx** response from the underlying endpoint
appear in the output file. Failed lines go to the error file (see below).

The order of lines in the output file is **not guaranteed** to match the
order of lines in the input file, match results to inputs by `custom_id`.

### Line schema

```json theme={null}
{
  "id":        "batch_req_abc123...",
  "custom_id": "request-1",
  "response": {
    "status_code": 200,
    "request_id":  "req_xyz789...",
    "body":        { /* the endpoint's normal response body */ }
  }
}
```

| Field | Type | Description |
| - | - | - |
| `id` | string | Line-level identifier (prefix `batch_req_`). Stable across retries. |
| `custom_id` | string | Echo of the `custom_id` you supplied on the input line. |
| `response.status_code` | integer | HTTP status from the underlying endpoint (will always be 2xx in this file). |
| `response.request_id` | string \| null | The synchronous request id (`req_…`) of the underlying call to the orchestration API. Useful for cross-referencing logs. `null` if the upstream service didn't surface a request id. |
| `response.body` | object | The same JSON body the synchronous endpoint would have returned for this request. |

See [Supported endpoints](/docs/batch/supported-endpoints) for the exact
`response.body` shape (OpenAI `chat.completion`).

### Example, chat completions output

```jsonl theme={null}
{"id":"batch_req_a","custom_id":"req-1","response":{"status_code":200,"request_id":"req_xyz","body":{"id":"chatcmpl-abc","object":"chat.completion","created":1736295000,"model":"<model-id>","choices":[{"index":0,"message":{"role":"assistant","content":"Paris."},"finish_reason":"stop"}],"usage":{"prompt_tokens":12,"completion_tokens":2,"total_tokens":14},"system_fingerprint":null}}}
{"id":"batch_req_b","custom_id":"req-2","response":{"status_code":200,"request_id":"req_uvw","body":{"id":"chatcmpl-def","object":"chat.completion","created":1736295001,"model":"<model-id>","choices":[{"index":0,"message":{"role":"assistant","content":"Berlin."},"finish_reason":"stop"}],"usage":{"prompt_tokens":12,"completion_tokens":2,"total_tokens":14},"system_fingerprint":null}}}
```

### Reading the output

```python theme={null}
import json

with open("output.jsonl") as f:
    for line in f:
        record = json.loads(line)
        custom_id = record["custom_id"]
        body      = record["response"]["body"]
        answer    = body["choices"][0]["message"]["content"]
        print(f"{custom_id}: {answer}")
```

***

## Error JSONL

When a batch reaches `status: "completed"` (or `expired`) and at least one
line failed, the service writes the failures to a file referenced by
`error_file_id`. Successful lines go to the output file; the two are
disjoint.

### Line schema

```json theme={null}
{
  "id":        "batch_req_def456...",
  "custom_id": "request-5",
  "response":  null,
  "error": {
    "code":    "invalid_request_error",
    "message": "[legacy:http_400] request body is invalid",
    "param":   "model"
  }
}
```

| Field | Type | Description |
| - | - | - |
| `id` | string | Line-level identifier (prefix `batch_req_`). |
| `custom_id` | string | Echo of the `custom_id` you supplied on the input line. |
| `response` | null | Always `null` on error lines. Mirrors OpenAI's `{id, custom_id, response, error}` discriminator so SDK type-guards work unchanged. |
| `error.code` | string | OpenAI semantic error code. See the table below. |
| `error.message` | string | Human-readable description. The original ZeroGPU internal code is preserved as a `[legacy:<old>] ` prefix on this string for forensics. |
| `error.param` | string \| null | Name of the offending parameter when known. |

Each error line carries an `error.code` (`invalid_request_error`,
`internal_error`, `batch_cancelled`, and so on). The full list, what triggers
each, and how to recover is in the [Errors reference](/docs/batch/errors).

### Example, error file

```jsonl theme={null}
{"id":"batch_req_e1","custom_id":"req-5","response":null,"error":{"code":"invalid_request_error","message":"[legacy:http_400] {\"message\":\"messages: missing required field\",\"type\":\"invalid_request_error\"}","param":null}}
{"id":"batch_req_e2","custom_id":"req-9","response":null,"error":{"code":"internal_error","message":"[legacy:retries_exhausted] Line failed after 4 attempts: orchestration-api returned 503: upstream timeout","param":null}}
{"id":"batch_req_e3","custom_id":"req-12","response":null,"error":{"code":"internal_error","message":"[legacy:dlq_exhausted] Message exhausted main-queue retries and was routed to the DLQ","param":null}}
```

### Reading the error file

```python theme={null}
import json

with open("errors.jsonl") as f:
    for line in f:
        record = json.loads(line)
        line_id   = record["id"]            # batch_req_…
        custom_id = record["custom_id"]
        code      = record["error"]["code"]  # OpenAI semantic code
        msg       = record["error"]["message"]
        print(f"FAIL {custom_id} ({line_id}) [{code}]: {msg}")
```

***

## Matching outputs and errors back to inputs

The output and error files together account for **all** lines in your input
file, but their order is not preserved and they may be split across the two
files. To reconstruct results for your dataset:

```python theme={null}
import json

results = {}                                # custom_id -> result-or-error
with open("output.jsonl") as f:
    for line in f:
        rec = json.loads(line)
        results[rec["custom_id"]] = ("ok", rec["response"]["body"])
with open("errors.jsonl") as f:
    for line in f:
        rec = json.loads(line)
        results[rec["custom_id"]] = ("err", rec["error"])

# now iterate your original inputs and look up by custom_id
```

If a `custom_id` you submitted is missing from both files, the batch is not
complete yet, re-check `status` and download again.

<Tip>
  **Always match by <code>custom\_id</code>**

  Output and error lines are <em>not</em> guaranteed to appear in input
  order. Index your inputs by <code>custom\_id</code> in your code and look
  results up by that key.
</Tip>

## Next steps

<CardGroup>
  <Card title="Supported endpoints →" href="/docs/batch/supported-endpoints">
    Body and response shape for `/v1/chat/completions`.
  </Card>

  <Card title="Quickstart →" href="/docs/batch/getting-started">
    Full end-to-end walkthrough in curl and Python.
  </Card>

  <Card title="Errors reference →" href="/docs/batch/errors">
    Every JSONL validation message and error code, with recovery guidance.
  </Card>
</CardGroup>


# Objects & lifecycle
Source: https://docs.zerogpu.ai/docs/batch/objects

The File and Batch objects, the batch status lifecycle, and how the two APIs fit together.

The Batch and Files APIs share one data model. You upload an input **file**,
create a **batch** that points at it, and when the batch finishes the service
writes result and error **files** back. This page describes those two objects,
the lifecycle a batch moves through, and where each endpoint lives.

For request and response details, every endpoint has an interactive page under
**API Reference**, linked from the tables below. To run a batch end to end,
start with the [Quickstart](/docs/batch/getting-started).

## Authentication

Both APIs authenticate with headers, not a bearer token:

```
x-api-key:    <your-api-key>
x-project-id: <your-project-uuid>
```

Missing either header returns `401`; a key that is invalid, inactive, or does
not belong to the supplied project returns `403`; an organization with no
balance and no free credits returns `429` with `insufficient_quota`. See the
[Errors reference](/docs/batch/errors) for the complete list.

## Endpoints

The Files API stores the JSONL that drives a batch:

| Method | Path | Reference |
| - | - | - |
| `POST` | `/v1/files` | [Upload file](/api-reference/upload-file) |
| `GET` | `/v1/files` | [List files](/api-reference/list-files) |
| `GET` | `/v1/files/{file_id}` | [Retrieve file](/api-reference/retrieve-file) |
| `GET` | `/v1/files/{file_id}/content` | [Download file](/api-reference/download-file) |
| `DELETE` | `/v1/files/{file_id}` | [Delete file](/api-reference/delete-file) |

The Batches API submits and monitors the jobs:

| Method | Path | Reference |
| - | - | - |
| `POST` | `/v1/batches` | [Create batch](/api-reference/create-batch) |
| `GET` | `/v1/batches` | [List batches](/api-reference/list-batches) |
| `GET` | `/v1/batches/{batch_id}` | [Retrieve batch](/api-reference/retrieve-batch) |
| `POST` | `/v1/batches/{batch_id}/cancel` | [Cancel batch](/api-reference/cancel-batch) |

Both APIs are wire-compatible with OpenAI's `/v1/files` and `/v1/batches`, so
the official SDKs work once pointed at the ZeroGPU base URL. The ZeroGPU
specifics are: `completion_window` must be `"24h"`, `endpoint` must be
`/v1/chat/completions` (the only [batchable endpoint](/docs/batch/supported-endpoints)),
files are soft-deleted, and error files carry an `is_error` flag (below).

## The File object

Returned by upload and retrieve, and inside `data[]` on list.

```json theme={null}
{
  "id":         "file-abc123...",
  "object":     "file",
  "bytes":      12345,
  "created_at": 1736290000,
  "filename":   "input.jsonl",
  "purpose":    "batch",
  "status":     "processed",
  "expires_at": 1738882000
}
```

| Field | Type | Description |
| - | - | - |
| `id` | string | File identifier, always prefixed `file-`. Legacy `file_` ids written before the prefix was normalised remain resolvable. |
| `object` | string | Always `"file"`. |
| `bytes` | integer | Size in bytes measured by the server after upload. |
| `created_at` | integer | Unix timestamp (seconds). |
| `filename` | string | The filename you uploaded, or `{file_id}.jsonl` if none was supplied. |
| `purpose` | string | `"batch"` for uploads; `"batch_output"` for both result and error files the service writes. |
| `status` | string | `"processed"` today. The wider `uploaded` / `processed` / `error` union matches OpenAI's published type for forward compatibility. |
| `expires_at` | integer \| omitted | When the file is removed. Uploads expire 30 days after upload; may be absent on server-written files. |
| `is_error` | boolean \| omitted | ZeroGPU extension. Present and `true` only on the error file referenced by `Batch.error_file_id`. OpenAI SDKs ignore it unless explicitly read. |

Output and error files are both written with `purpose: "batch_output"`; the
`is_error: true` flag is what distinguishes the error file when you
`GET /v1/files?purpose=batch_output`.

## The Batch object

Returned by create, retrieve, and cancel, and inside `data[]` on list.

```json theme={null}
{
  "id":                "batch_01HZX...",
  "object":            "batch",
  "endpoint":          "/v1/chat/completions",
  "errors":            null,
  "input_file_id":     "file-abc123...",
  "completion_window": "24h",
  "status":            "in_progress",
  "output_file_id":    null,
  "error_file_id":     null,
  "created_at":        1736290000,
  "in_progress_at":    1736290001,
  "expires_at":        1736376400,
  "finalizing_at":     null,
  "completed_at":      null,
  "failed_at":         null,
  "expired_at":        null,
  "cancelling_at":     null,
  "cancelled_at":      null,
  "request_counts":    { "total": 1500, "completed": 0, "failed": 0 },
  "metadata":          { "job": "nightly-classify" }
}
```

| Field | Type | Description |
| - | - | - |
| `id` | string | Batch identifier, prefix `batch_`. |
| `object` | string | Always `"batch"`. |
| `endpoint` | string | The endpoint every line in this batch targets. |
| `errors` | object \| null | When `status` is `failed`, holds `{ object: "list", data: [BatchError] }` describing the validation failure. Each `BatchError` is `{ code, message, line, param }`. `null` otherwise. |
| `input_file_id` | string | The `file-` id you supplied at creation. |
| `completion_window` | string | Always `"24h"`. |
| `status` | string | Lifecycle state, see below. |
| `output_file_id` | string \| null | File of successfully completed lines, populated when `status` is `completed`. Download via `GET /v1/files/{id}/content`. |
| `error_file_id` | string \| null | File of failed lines, populated when at least one line failed. Listed with `purpose: "batch_output"`, `is_error: true`. |
| `created_at` | integer | When the batch was created. |
| `in_progress_at` | integer \| null | When validation finished and processing started. |
| `expires_at` | integer | 24 hours after `created_at`; anything unfinished by then becomes `expired`. |
| `finalizing_at` | integer \| null | When all lines finished and the output files began writing. |
| `completed_at` / `failed_at` / `expired_at` / `cancelling_at` / `cancelled_at` | integer \| null | Timestamps for each terminal or transitional state; `null` until reached. |
| `request_counts` | object | `{ total, completed, failed }`. `completed + failed` equals `total` only once the batch is terminal. |
| `metadata` | object | Arbitrary JSON you supplied at creation, echoed back unchanged. Empty `{}` when omitted, never `null`. |

## Status lifecycle

```mermaid theme={null}
stateDiagram-v2
    direction LR
    [*] --> validating: POST /v1/batches
    validating --> in_progress: JSONL OK
    validating --> failed: JSONL rejected
    in_progress --> finalizing: all lines processed
    in_progress --> cancelling: POST /cancel
    cancelling --> cancelled: drained
    finalizing --> completed: outputs written
    in_progress --> expired: 24 h elapsed
    completed --> [*]
    expired --> [*]
    failed --> [*]
    cancelled --> [*]
```

| Status | Meaning |
| - | - |
| `validating` | Row persisted; the JSONL is being streamed and validated synchronously. Transient. |
| `in_progress` | Validation succeeded; lines are being processed. |
| `finalizing` | All lines processed; output and error files are being written. Transient. |
| `completed` | Terminal. `output_file_id` (and `error_file_id` if any line failed) are populated. |
| `failed` | Terminal. JSONL validation rejected the input; `errors` holds the offending line(s). Still retrievable so SDKs can introspect. |
| `expired` | Terminal. The 24-hour window elapsed first. Whatever finished is in `output_file_id` / `error_file_id`; the rest is in the error file with `code: "batch_expired"`. |
| `cancelling` | Cancellation requested. Dispatched lines finish; pending lines are short-circuited to the error file with `code: "batch_cancelled"`. |
| `cancelled` | Terminal. Cancellation drained; files reflect whatever processed before the request. |

Validation is **synchronous**, processing is **asynchronous**: the create call
returns only after the JSONL passes (or fails) validation, and lines are then
worked through an internal queue. Once a batch is terminal, no further state
changes occur, so stop polling. Retrieve returns live `request_counts`; the
list endpoint may lag by a few seconds, so poll a specific batch for accurate
progress.

## Reading results

When `status` is `completed`:

* `output_file_id` is set if any line returned 2xx. Download it and match
  results to inputs by `custom_id`, order is not preserved.
* `error_file_id` is set if any line failed. Partition by `error.code` to
  decide what to retry.

Both files follow the same 30-day retention as your uploads, and the line
schemas live in [JSONL format](/docs/batch/jsonl-format). Deleting an input
file does not stop a running batch, the input is read at creation time. For
size and rate limits, see the [Overview](/docs/batch) quick facts.

## Next steps

<CardGroup>
  <Card title="JSONL format →" href="/docs/batch/jsonl-format">
    Exact line schema for input, output, and error files.
  </Card>

  <Card title="Errors reference →" href="/docs/batch/errors">
    Every status, validation message, and error-file code, with recovery steps.
  </Card>
</CardGroup>


# Supported Endpoints
Source: https://docs.zerogpu.ai/docs/batch/supported-endpoints

The only endpoint a batch line can target. Same request body, same response body, only the routing is asynchronous.

Every line in a batch input file must target `/v1/chat/completions` in the
`url` field, and the `endpoint` field on the create call must match. This
matches OpenAI's published batchable surface for this API version.

| `url` | Use case | Sync equivalent |
| - | - | - |
| `/v1/chat/completions` | OpenAI-style chat completions | `POST /v1/chat/completions` |

The request body inside each JSONL line's `body` field is the **same** body
you would send synchronously. The response in the output JSONL's
`response.body` is the **same** response the synchronous endpoint would
return.

<Note>
  **Other ZeroGPU endpoints are sync-only**

  Internal routes like <code>/summary</code>, <code>/v1/responses</code>,
  <code>/usecase/classify/iab</code>, and <code>/gliner</code> remain
  reachable as synchronous calls but are <strong>not</strong> batchable.
  Submitting a JSONL line with one of these <code>url</code> values is
  rejected at create time with
  <code>"is not an allowed batch endpoint"</code>.
</Note>

***

## `/v1/chat/completions`

Standard OpenAI-style chat completions. Suitable for any model that supports
chat-completion semantics.

### Request body (inside JSONL `body`)

```json theme={null}
{
  "model": "<model-id>",
  "messages": [
    { "role": "system", "content": "You are a helpful assistant." },
    { "role": "user",   "content": "What is the capital of France?" }
  ]
}
```

| Field | Required | Description |
| - | - | - |
| `model` | Yes | A model ID enabled for your project. |
| `messages` | Yes | Array of `{role, content}`. `role` is one of `user`, `assistant`, `system`, `tool`, `developer`. |
| `stream` | - | **Must not be `true`.** Rejected at batch creation. |

Per-model token limits apply (configured in model metadata). If input tokens
exceed the model's `max_tokens` setting, behavior depends on the model's
`error_on_max_token` flag: either the line fails with
`code: "context_length_exceeded"` (HTTP `400`), or the input is silently
truncated. Verify with your model's configuration.

### Response body (inside output JSONL `response.body`)

```json theme={null}
{
  "id":      "chatcmpl-...",
  "object":  "chat.completion",
  "created": 1736295000,
  "model":   "<model-id>",
  "choices": [
    {
      "index": 0,
      "message": {
        "role":        "assistant",
        "content":     "Paris.",
        "annotations": []
      },
      "finish_reason": "stop",
      "logprobs":      null
    }
  ],
  "usage": {
    "prompt_tokens":     12,
    "completion_tokens": 2,
    "total_tokens":      14,
    "prompt_tokens_details":     { "cached_tokens": 0, "audio_tokens": 0 },
    "completion_tokens_details": {
      "reasoning_tokens": 0,
      "audio_tokens": 0,
      "accepted_prediction_tokens": 0,
      "rejected_prediction_tokens": 0
    }
  },
  "service_tier":       "default",
  "system_fingerprint": null
}
```

`system_fingerprint` is always `null` today. ZeroGPU does not yet expose
backend-determinism fingerprints. The field is present for SDK
compatibility (OpenAI's `ChatCompletion` type declares it as optional but
real OpenAI populates it on every response).

For **classification models** invoked via `/v1/chat/completions`,
`choices[0].message.content` is a JSON-stringified classification response
rather than free-form text. Parse it as JSON to extract the structured
result.

## Next steps

<CardGroup>
  <Card title="JSONL format →" href="/docs/batch/jsonl-format">
    How to wrap each request body in a valid input JSONL line.
  </Card>

  <Card title="Objects & lifecycle →" href="/docs/batch/objects">
    The Batch object, status lifecycle, and every endpoint.
  </Card>

  <Card title="Quickstart →" href="/docs/batch/getting-started">
    End-to-end walkthrough in curl and Python.
  </Card>
</CardGroup>


# Get Started
Source: https://docs.zerogpu.ai/docs/edge-providers/get-started

Get your Edge Provider credentials, add the ZeroGPU Edge SDK to your app, and bring your first device online.

This guide takes you from a new account to a device serving inference on the ZeroGPU edge network.

<Steps>
  <Step title="Get your ZeroGPU Edge Provider credentials">
    Sign up at the [Edge Operator Portal](https://edge.zerogpu.ai/auth/sign-up), then choose **Add New App** and fill in:

    | Field | What to enter |
    | - | - |
    | **App name** | The name of your app |
    | **Website / store URL** | Your website, store listing, or Telegram `t.me` link |
    | **Category** | Productivity, Utilities, Creative, Finance, News, Lifestyle, or Gaming |
    | **Estimated monthly users** | Your approximate monthly active users |
    | **Primary region** | Where most of your users are |
    | **App types** | The platforms you'll integrate: Android, Browser, Chrome Extension, Telegram Mini App, macOS |

    Accept the terms and select **Submit for approval**. Your app shows as **Pending Review**; review usually takes 24–48 hours. Once it's **Approved**, open the app in the portal to copy its **SDK key**.

    <Note>
      The SDK key (`zgpu-sdk-…`) is for Edge Providers: it lets your app's devices join the network. It is different from the API key (`zgpu-api-…`) used to *call* the ZeroGPU API. Don't mix them up.
    </Note>
  </Step>

  <Step title="Select your platform">
    Pick the SDK that matches your app. Each SDK page covers requirements, setup, lifecycle, and troubleshooting.

    <Columns>
      <Card title="Android" icon="android" href="/docs/edge-providers/sdk/android" />

      <Card title="Chrome / Web" icon="chrome" href="/docs/edge-providers/sdk/web" />

      <Card title="Telegram Mini Apps" icon="telegram" href="/docs/edge-providers/sdk/telegram" />

      <Card title="macOS" icon="apple" href="/docs/edge-providers/sdk/macos" />
    </Columns>
  </Step>

  <Step title="Install the SDK">
    <Tabs>
      <Tab title="Android">
        ```kotlin build.gradle.kts theme={null}
        dependencies {
            implementation("ai.zerogpu:android-sdk:0.3.0")
        }
        ```
      </Tab>

      <Tab title="Web">
        ```html theme={null}
        <script async src="https://js-sdk.zerogpu.ai/dist/loader.js"></script>
        ```
      </Tab>

      <Tab title="Chrome extension">
        ```bash theme={null}
        npm install @zerogpu/browser-extension-sdk
        npx zerogpu init
        ```
      </Tab>

      <Tab title="Telegram Mini Apps">
        ```html theme={null}
        <script async src="https://js-sdk.zerogpu.ai/dist/loader.js"></script>
        ```
      </Tab>

      <Tab title="macOS">
        ```bash theme={null}
        npm install @zerogpu/worker
        ```
      </Tab>
    </Tabs>
  </Step>

  <Step title="Initialize ZeroGPU">
    Pass your SDK key when you create the SDK. Keep it out of source control and inject it at build time.

    <Tabs>
      <Tab title="Android">
        ```kotlin theme={null}
        val sdk = ZeroGpuSdk(
            context = applicationContext,
            operatorKey = BuildConfig.ZEROGPU_OPERATOR_KEY, // "zgpu-sdk-…"
        )
        ```
      </Tab>

      <Tab title="Web / Telegram">
        ```html theme={null}
        <script
          async
          src="https://js-sdk.zerogpu.ai/dist/loader.js"
          edgeOperatorKey="zgpu-sdk-your-key"
        ></script>
        ```
      </Tab>

      <Tab title="Chrome extension">
        ```js offscreen.js theme={null}
        import { initZeroGpuOffscreen } from "./vendor/zerogpu/zerogpu-offscreen.js";

        initZeroGpuOffscreen({ edgeOperatorKey: "zgpu-sdk-your-key" });
        ```
      </Tab>

      <Tab title="macOS">
        ```js theme={null}
        const worker = new ZeroGpuWorker({ operatorKey: ZEROGPU_OPERATOR_KEY });
        ```
      </Tab>
    </Tabs>
  </Step>

  <Step title="Register the device">
    Nothing to call: registration is automatic. When the SDK starts, it registers the device with the ZeroGPU device registry using your SDK key, downloads and verifies its assigned model, runs a warm-up inference, and opens a secure connection for tasks.
  </Step>

  <Step title="Start contributing compute">
    On Android and macOS, call `start()`. In browsers, Telegram Mini Apps, and Chrome extensions, the SDK starts as soon as it's initialized with your SDK key. From then on the device receives inference tasks from the network and runs them locally; you never feed it tasks or poll it.
  </Step>

  <Step title="Confirm the device is online">
    * **SDK logs:** turn on debug logging (each SDK page shows how) and look for the device registering, loading its model, and becoming ready.
    * **Portal:** check your app's status and the **Analytics** page in the [Edge Operator Portal](https://edge.zerogpu.ai). Apps serving traffic show as **Active/Earning**.

    A connected device only receives work when the network needs it, so it may sit ready for a while before its first task.
  </Step>
</Steps>

<Warning>
  **Tell your users.** Before the SDK starts, show your users the End User Disclosure from the Edge Operator Portal, word for word, with your app's name, and get their agreement. Give them a way to turn resource sharing off.
</Warning>

## Choose your SDK

<Columns>
  <Card title="Android" icon="android" href="/docs/edge-providers/sdk/android">
    `ai.zerogpu:android-sdk` for Android apps.
  </Card>

  <Card title="Chrome / Web" icon="chrome" href="/docs/edge-providers/sdk/web">
    Script tag for websites; npm package for Chrome extensions.
  </Card>

  <Card title="Telegram Mini Apps" icon="telegram" href="/docs/edge-providers/sdk/telegram">
    One script tag in your Mini App.
  </Card>

  <Card title="macOS" icon="apple" href="/docs/edge-providers/sdk/macos">
    `@zerogpu/worker` for Electron desktop apps.
  </Card>
</Columns>


# Edge Providers
Source: https://docs.zerogpu.ai/docs/edge-providers/index

Connect compatible devices and applications to ZeroGPU and contribute idle compute to distributed AI inference across the ZeroGPU network.

Become an Edge Provider by connecting compatible devices and applications to ZeroGPU. Contribute compute from idle devices and make that capacity available for distributed AI inference across the ZeroGPU network.

You add a ZeroGPU Edge SDK to an app you already ship. While your users run the app, their device becomes an inference node: it downloads a small model, registers with the network, and serves inference tasks locally. ZeroGPU handles routing, model assignment, and verification; you don't run any servers.

## Edge SDKs

<Columns>
  <Card title="Android SDK" icon="android" href="/docs/edge-providers/sdk/android">
    Connect supported Android devices to the ZeroGPU edge network.
  </Card>

  <Card title="Chrome / Web SDK" icon="chrome" href="/docs/edge-providers/sdk/web">
    Contribute compute from websites and Chrome extensions running in supported browsers.
  </Card>

  <Card title="Telegram Mini Apps SDK" icon="telegram" href="/docs/edge-providers/sdk/telegram">
    Connect Telegram Mini Apps to ZeroGPU and contribute available device compute to the edge network.
  </Card>

  <Card title="macOS SDK" icon="apple" href="/docs/edge-providers/sdk/macos">
    Connect supported Apple Silicon devices to the ZeroGPU edge network.
  </Card>
</Columns>

## How it works

<Steps>
  <Step title="Add your app">
    Register your app in the [Edge Operator Portal](https://edge.zerogpu.ai) and submit it for approval.
  </Step>

  <Step title="Get your SDK key">
    Once your app is approved, copy its SDK key (`zgpu-sdk-…`) from the portal.
  </Step>

  <Step title="Integrate the SDK">
    Add the SDK for your platform and start it with your SDK key. Devices register themselves automatically.
  </Step>

  <Step title="Contribute compute">
    Devices running your app serve inference tasks. Track activity and earnings in the portal.
  </Step>
</Steps>

<Card title="Get started" icon="arrow-right" href="/docs/edge-providers/get-started">
  Set up your first Edge Provider integration.
</Card>


# Android
Source: https://docs.zerogpu.ai/docs/edge-providers/sdk/android

Turn devices running your Android app into ZeroGPU edge inference nodes with the ZeroGPU Android SDK.

## Overview

The ZeroGPU Android SDK (`ai.zerogpu:android-sdk`) turns the devices running your app into on-device inference nodes on the ZeroGPU network. Once started, it registers the device, downloads the model the network assigns, and serves inference tasks locally with ONNX Runtime (and a bundled llama.cpp for GGUF models).

The SDK follows your app's lifecycle: it contributes compute while the app is in the foreground and disconnects when the app goes to the background.

## Requirements

| Requirement | Value |
| - | - |
| **Android version** | Android 8.0+ (`minSdk 26`); built against `compileSdk 36` |
| **Language** | Kotlin with coroutines |
| **ABIs** | `arm64-v8a` (physical devices) and `x86_64` (emulators) |
| **Credentials** | An SDK key (`zgpu-sdk-…`) from the [Edge Operator Portal](https://edge.zerogpu.ai) |

The SDK's manifest declares `INTERNET` and `ACCESS_NETWORK_STATE` and sets `android:largeHeap="true"`. These merge into your app automatically.

## Installation

Make sure Maven Central is a repository:

```kotlin settings.gradle.kts theme={null}
dependencyResolutionManagement {
    repositories {
        google()
        mavenCentral()
    }
}
```

Add the dependency to your app module:

```kotlin build.gradle.kts theme={null}
dependencies {
    implementation("ai.zerogpu:android-sdk:0.3.0")
}
```

The SDK bundles native libraries (ONNX Runtime and llama.cpp). Add this to the `android { }` block so duplicate copies of `libc++_shared.so` don't fail the build:

```kotlin build.gradle.kts theme={null}
android {
    packaging {
        jniLibs {
            useLegacyPackaging = true
            pickFirsts += setOf("**/libc++_shared.so")
        }
    }
}
```

<Tip>
  Using Expo / React Native? Install the wrapper instead: `npx expo install @zerogpu/expo-android-sdk`, add `"@zerogpu/expo-android-sdk"` to your app config plugins, and run `npx expo prebuild`.
</Tip>

## Initialization

Create **one** `ZeroGpuSdk` for your whole app, typically in `Application.onCreate()`. The base URL and environment default to production, so you only pass your SDK key:

```kotlin theme={null}
import ai.zerogpu.android.sdk.ZeroGpuSdk

class MyApp : Application() {
    lateinit var zeroGpu: ZeroGpuSdk

    override fun onCreate() {
        super.onCreate()
        zeroGpu = ZeroGpuSdk(
            context = this,
            operatorKey = BuildConfig.ZEROGPU_OPERATOR_KEY, // "zgpu-sdk-…"
        )
        zeroGpu.start()
    }
}
```

Optional constructor parameters:

| Parameter | Default | Description |
| - | - | - |
| `enableLogs` | `false` | Write the SDK's logcat output. Turn on while integrating. |
| `enableTelemetry` | `true` | Send lifecycle and error events to ZeroGPU. |
| `resourcePolicy` | `ResourcePolicy()` | Local limits: `maxModelBytes`, `memoryBudgetFraction`, `disableGuardrails`. |
| `listener` | `null` | A `ZeroGpuListener` for lifecycle callbacks (see [Device lifecycle](#device-lifecycle)). |
| `customParams` | `emptyMap()` | Up to three reporting slots, `cust_param1`–`cust_param3`. Values only; no personal data. |

## Authentication

The SDK authenticates with your **SDK key** (`zgpu-sdk-…`), passed as `operatorKey`. Don't hard-code it. Put it in `local.properties` (git-ignored) and expose it through `BuildConfig`:

```kotlin build.gradle.kts theme={null}
import java.util.Properties

val localProps = Properties().apply {
    rootProject.file("local.properties").takeIf { it.exists() }?.inputStream()?.use { load(it) }
}

android {
    defaultConfig {
        buildConfigField(
            "String",
            "ZEROGPU_OPERATOR_KEY",
            "\"${localProps.getProperty("zerogpu.operatorKey", "")}\"",
        )
    }
    buildFeatures { buildConfig = true }
}
```

```properties local.properties theme={null}
# never commit
zerogpu.operatorKey=zgpu-sdk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
```

## Register device

Registration is automatic. After `start()`, while the app is in the foreground, the SDK:

1. Registers the device with the ZeroGPU device registry using your SDK key.
2. Downloads its assigned model(s), verifies them, and caches them on device. Models are only re-downloaded when they change.
3. Runs a warm-up inference.
4. Opens a WebSocket and advertises the device as ready.

## Start contributing compute

Call `start()` once. From then on the device manages itself: it follows the app in and out of the foreground, reconnects after network drops, and serves whatever tasks the network sends. You don't poll it or feed it tasks.

## Stop / pause

```kotlin theme={null}
zeroGpu.stop()
```

`stop()` tears everything down: it closes the connection, unloads models, and cancels in-flight work. There is no separate pause call; to pause, call `stop()`, and call `start()` again to resume.

Going to the background already pauses contribution: the SDK closes its connection and reconnects when the app returns to the foreground.

## Device lifecycle

| State | What's happening |
| - | - |
| **Stopped** | Created but not started, or torn down by `stop()`. |
| **Registering** | Registering the device with the network. |
| **Loading models** | Downloading, verifying, and warming up models (first registration only). |
| **Ready** | Connected and waiting for tasks. |
| **Processing** | Running inference tasks. |
| **Reconnecting** | Recovering from a network drop, with backoff. |
| **Backgrounded** | App is in the background; the connection is deliberately closed. |

Before downloading models, the SDK checks free memory and storage, and blocks downloads when the battery is below 20% and the device isn't charging.

Subscribe to lifecycle changes with a `ZeroGpuListener`. Callbacks arrive on a background thread:

```kotlin theme={null}
val listener = object : ZeroGpuListener {
    override fun onDegraded(reason: String) { /* resource pressure; contribution limited */ }
    override fun onRecovered() { /* back to normal */ }
    override fun onShutdown(reason: String) { /* the SDK stopped itself */ }
}
```

## Example integration

```kotlin MainActivity.kt theme={null}
import ai.zerogpu.android.sdk.ZeroGpuSdk

class MainActivity : ComponentActivity() {

    private val sdk = ZeroGpuSdk(
        context = this,
        operatorKey = BuildConfig.ZEROGPU_OPERATOR_KEY, // "zgpu-sdk-…"
        // baseUrl defaults to https://devices.zerogpu.ai
        // env     defaults to "production"
    )

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        // ...your UI...
        sdk.start()
    }

    override fun onDestroy() {
        sdk.stop()
        super.onDestroy()
    }
}
```

To confirm the device is online, construct the SDK with `enableLogs = true`, run the app in the foreground, and watch logcat:

```bash theme={null}
adb logcat -s ZeroGpuSdk:* ZeroGpuWebSocket:* ModelDownloader:*
```

Once you see `status":"idle"` and `WebSocket ready for tasks`, the device is serving the network.

## Troubleshooting

| Symptom | Likely cause / fix |
| - | - |
| `Register failed: HTTP 401/403` | Wrong or inactive SDK key. Check the key in the Edge Operator Portal. |
| Never reaches `ready for tasks` | The app isn't in the foreground, or there's no network. The connection closes in the background by design. |
| Device is ready but serves no tasks | The network hasn't needed it yet; work depends on demand. Keep the app running. |
| No SDK lines in logcat | Logging is off by default. Pass `enableLogs = true`. |
| Build error: duplicate `libc++_shared.so` | Add the `packaging { jniLibs { … } }` block from [Installation](#installation). |
| Native model won't load on some devices | Only `arm64-v8a` and `x86_64` are shipped. Use an `arm64` device or an `x86_64` emulator. |


# macOS
Source: https://docs.zerogpu.ai/docs/edge-providers/sdk/macos

Turn Macs running your Electron desktop app into ZeroGPU edge inference nodes with @zerogpu/worker.

## Overview

`@zerogpu/worker` turns the Macs running your desktop app into compute nodes on the ZeroGPU network. It runs in its own background process (an Electron [utility process](https://www.electronjs.org/docs/latest/api/utility-process)), registers the Mac, downloads its assigned model, and serves inference tasks locally. There are no servers for you to run.

Unlike the browser and Android SDKs, the worker doesn't watch battery, sleep, or window focus. It keeps serving until you stop it, so **your app decides when it runs**. The recommended policy below runs it while your app is open and the Mac is on power.

## Requirements

| Requirement | Value |
| - | - |
| **App shell** | Electron 29 or newer (Node.js 20+). Other shells that can ship Node.js 20+ also work; see [Other app shells](#other-app-shells). |
| **Hardware** | Apple Silicon (`arm64`) or Intel (`x64`) Mac |
| **Disk** | About 100 MB per assigned ONNX model (GGUF models: 2–5 GB), plus about 140 MB of native runtimes in your app bundle |
| **Credentials** | An SDK key (`zgpu-sdk-…`) from the [Edge Operator Portal](https://edge.zerogpu.ai) |

## Installation

```bash theme={null}
npm install @zerogpu/worker
```

This adds the worker and its native inference runtimes (ONNX Runtime, and llama.cpp with Metal on Apple Silicon for GGUF models). The worker must ship as a real dependency in your packaged app's `node_modules`; don't bundle it into your main-process code.

## Initialization

Create `zerogpu-worker.mjs` next to your main-process code. It starts the worker in a utility process with a free loopback port and a per-launch token, keeps its state in your app's data directory, and stops it cleanly:

```js zerogpu-worker.mjs theme={null}
import { app, utilityProcess } from "electron";
import { randomBytes } from "node:crypto";
import { createWriteStream, mkdirSync } from "node:fs";
import { createRequire } from "node:module";
import { createServer } from "node:net";
import { join } from "node:path";

const require = createRequire(import.meta.url);

/** Ask the OS for a free loopback port. */
function findFreePort() {
  return new Promise((resolve, reject) => {
    const probe = createServer();
    probe.once("error", reject);
    probe.listen(0, "127.0.0.1", () => {
      const { port } = probe.address();
      probe.close(() => resolve(port));
    });
  });
}

export class ZeroGpuWorker {
  #child = null;
  #log = null;
  #exited = null;
  #starting = null;
  #port = 0;
  #token = "";

  constructor({ operatorKey, environment = "production", logLevel = "warn", onExit }) {
    this.operatorKey = operatorKey;
    this.environment = environment;
    this.logLevel = logLevel;
    this.onExit = onExit;
  }

  get running() {
    return this.#child !== null;
  }

  /** Start the worker. Idempotent; safe to call repeatedly. */
  start() {
    this.#starting ??= this.#spawn().finally(() => {
      this.#starting = null;
    });
    return this.#starting;
  }

  async #spawn() {
    if (this.#child) return;
    const stateDir = join(app.getPath("userData"), "zerogpu");
    mkdirSync(stateDir, { recursive: true });
    this.#port = await findFreePort();
    this.#token = randomBytes(24).toString("hex");

    const child = utilityProcess.fork(require.resolve("@zerogpu/worker/cli"), [], {
      serviceName: "ZeroGPU Worker",
      stdio: "pipe",
      env: {
        ...process.env,
        ZG_OPERATOR_KEY: this.operatorKey,
        ZG_ENV: this.environment,
        ZG_LOCAL_API_PORT: String(this.#port),
        ZG_LOCAL_API_TOKEN: this.#token,
        ZG_MODELS_DIR: join(stateDir, "models"),
        ZG_DATA_DIR: join(stateDir, "data"),
        ZG_MAX_CONCURRENCY: "1",
        ZG_LOG_LEVEL: this.logLevel,
      },
    });

    this.#log ??= createWriteStream(join(app.getPath("logs"), "zerogpu-worker.log"), { flags: "a" });
    child.stdout?.pipe(this.#log, { end: false });
    child.stderr?.pipe(this.#log, { end: false });

    this.#exited = new Promise((resolve) => {
      child.once("exit", (code) => {
        if (this.#child === child) this.#child = null;
        this.onExit?.(code);
        resolve(code);
      });
    });
    this.#child = child;
  }

  /** Worker health: lifecycle status, model states, served/network counters. */
  async status() {
    if (!this.#child) return { running: false, ready: false };
    try {
      const res = await fetch(`http://127.0.0.1:${this.#port}/healthz`, {
        signal: AbortSignal.timeout(2000),
      });
      return { running: true, ...(await res.json()) };
    } catch {
      return { running: true, ready: false, status: "starting" };
    }
  }

  /** Stop the worker: it leaves the network cleanly, then exits (≤ 5 s). */
  async stop(timeoutMs = 7000) {
    await this.#starting;
    const child = this.#child;
    if (!child) return;
    this.#child = null;
    child.kill(); // SIGTERM → graceful shutdown
    await Promise.race([this.#exited, new Promise((resolve) => setTimeout(resolve, timeoutMs))]);
  }
}
```

The worker is configured entirely through environment variables:

| Variable | Set to | Why |
| - | - | - |
| `ZG_OPERATOR_KEY` | your SDK key | Joins your fleet. Required. |
| `ZG_ENV` | `production` | Use `staging` or `develop` only if ZeroGPU gave you access. |
| `ZG_LOCAL_API_PORT` | a free port | The local API, bound to `127.0.0.1`. |
| `ZG_LOCAL_API_TOKEN` | random, per launch | Only your app can call the local inference routes. |
| `ZG_MODELS_DIR`, `ZG_DATA_DIR` | under `userData` | Survive app updates. Don't wipe them: a new identity makes the Mac a newcomer that gets less work at first. |
| `ZG_MAX_CONCURRENCY` | `1` | One inference at a time, gentle on a laptop. |
| `ZG_LOG_LEVEL` | `warn` | Use `info` while integrating. |

## Authentication

The worker authenticates with your **SDK key** (`zgpu-sdk-…`) through `ZG_OPERATOR_KEY`. The key only joins devices to your fleet, but it ships inside every copy of your app, so keep it out of source control and inject it at build time.

<Warning>
  Never ship a ZeroGPU **API key** (`zgpu-api-…`) inside a desktop app. It authorizes paid inference for your whole organization, and anything in an app bundle can be extracted.
</Warning>

## Register device

Registration is automatic. On start, the worker runs a short CPU benchmark (about 5 seconds), creates a random device ID stored in `ZG_DATA_DIR`, registers the Mac with ZeroGPU, downloads and verifies its assigned model, loads it, and opens an outbound connection for tasks. Expect under 30 seconds on first launch and a few seconds afterwards.

All connections to ZeroGPU are outbound (HTTPS and WSS). The only listener is the local API on `127.0.0.1`.

## Start contributing compute

Create **one** `ZeroGpuWorker` in your main process and start it when the Mac is on power:

```js main.mjs theme={null}
import { app, powerMonitor } from "electron";

import { ZEROGPU_OPERATOR_KEY } from "./build-config.mjs"; // generated at build time, not committed
import { ZeroGpuWorker } from "./zerogpu-worker.mjs";

const worker = new ZeroGpuWorker({
  operatorKey: ZEROGPU_OPERATOR_KEY,
  logLevel: app.isPackaged ? "warn" : "info",
});

const startIfOnPower = () => {
  if (!powerMonitor.isOnBatteryPower()) void worker.start();
};

app.whenReady().then(() => {
  startIfOnPower();
  powerMonitor.on("on-ac", startIfOnPower);
  powerMonitor.on("resume", startIfOnPower);
  powerMonitor.on("on-battery", () => void worker.stop());
  powerMonitor.on("suspend", () => void worker.stop());
});
```

## Stop / pause

`worker.stop()` sends `SIGTERM`: the worker leaves the network, flushes telemetry, unloads its model, and exits within 5 seconds. To pause, call `stop()`; to resume, call `start()`. The power policy above does exactly this on battery and sleep.

Let the worker leave the network before your app quits:

```js main.mjs theme={null}
let workerStopped = false;
app.on("before-quit", (event) => {
  if (workerStopped) return;
  event.preventDefault();
  worker.stop().finally(() => {
    workerStopped = true;
    app.quit();
  });
});
```

## Device lifecycle

Check status from your main process with `await worker.status()`:

```json theme={null}
{
  "running": true,
  "status": "ready",
  "ready": true,
  "capacity": { "inFlight": 0, "queued": 0, "maxConcurrency": 1 },
  "network": { "completedTasks": 3, "failedTasks": 0 }
}
```

`ready` turns `true` once a model is loaded and has passed a warm-up inference. `network.completedTasks` counts tasks ZeroGPU sent to this Mac. Use the `onExit` option to show status or restart after a delay; the worker protects itself against crash loops with a cooldown.

## Example integration

1. Start your app with `logLevel: "info"`.

2. Open the worker log at `~/Library/Logs/<Your App>/zerogpu-worker.log`. A healthy first start looks like this:

   ```text theme={null}
   INFO  [main] ZeroGPU Worker starting (env=production, fallback=reject)
   INFO  [register] Registering device 57d6219e-… with https://devices.zerogpu.ai/register (env=production)
   INFO  [register] Registered. 1 model recipe(s) assigned
   INFO  [models] Model zlm-v1-iab-classify-onnx@1 verified (4 files)
   INFO  [lifecycle] Device ready: iab_classify
   ```

3. Quit your app. The log ends with `[lifecycle] Shutdown (host_stop)`.

Switch `logLevel` back to `warn` for release builds.

### Packaging and signing

Native libraries must live outside the ASAR archive and be signed with your Developer ID. With electron-builder:

```json package.json theme={null}
{
  "build": {
    "asar": true,
    "asarUnpack": [
      "**/node_modules/onnxruntime-node/**/*",
      "**/node_modules/sharp/**/*",
      "**/node_modules/@img/**/*",
      "**/node_modules/node-llama-cpp/**/*",
      "**/node_modules/@node-llama-cpp/**/*"
    ],
    "mac": {
      "target": [{ "target": "dmg", "arch": ["arm64", "x64"] }],
      "notarize": true
    }
  }
}
```

Build `arm64` and `x64` separately, each after `npm ci --os=darwin --cpu=<arch>`, so every architecture gets its matching native binaries.

### Other app shells

Not using Electron? Any app that can ship a Node.js 20+ runtime can run the same entry point as a child process with the same variables:

```bash theme={null}
ZG_OPERATOR_KEY="zgpu-sdk-…" \
ZG_LOCAL_API_PORT=52817 \
ZG_LOCAL_API_TOKEN="$(openssl rand -hex 24)" \
ZG_MODELS_DIR="$HOME/Library/Application Support/YourApp/zerogpu/models" \
ZG_DATA_DIR="$HOME/Library/Application Support/YourApp/zerogpu/data" \
node node_modules/@zerogpu/worker/dist/cli.js
```

Stop it with `SIGTERM`, allow 5 seconds for it to exit, and make sure it never outlives your app.

## Troubleshooting

| Symptom | Fix |
| - | - |
| Worker exits immediately with code `1`: `ZG_OPERATOR_KEY … is required` | The key didn't reach the worker's environment. Check the value you pass as `operatorKey`. |
| `Registration auth failure (401)`, then `policy says stop` | The key isn't valid for the target environment. Fix the key or `environment` and restart. |
| `Registration politely refused: <reason>` | Normal: the network is declining this Mac for now. The worker retries by itself. |
| `status()` shows `ready: false` at first | Still booting. Allow up to \~30 seconds on first launch. |
| Packaged app: `dlopen` errors or `Could not load the "sharp" module` | Native files are still inside the ASAR archive, or the build is missing binaries for that architecture. See [Packaging and signing](#packaging-and-signing). |
| Packaged app: `Cannot find module '@zerogpu/worker/cli'` | Your bundler dropped the dependency. Keep `@zerogpu/worker` external and shipped in `node_modules`. |
| `network.completedTasks` stays at `0` | The Mac is connected, but the network hasn't needed it yet. Keep `ZG_DATA_DIR` persistent. |
| `npm install` fails with a `404` from another registry | Your `.npmrc` maps `@zerogpu` elsewhere. Use `npm install @zerogpu/worker --@zerogpu:registry=https://registry.npmjs.org`. |


# Telegram Mini Apps
Source: https://docs.zerogpu.ai/docs/edge-providers/sdk/telegram

Connect your Telegram Mini App to ZeroGPU and contribute device compute to the edge network with one script tag.

## Overview

Telegram Mini Apps run on the ZeroGPU browser SDK: one script tag in your Mini App's HTML turns the devices opening it into inference nodes on the ZeroGPU network. The SDK detects the Telegram WebApp environment automatically and adapts to it.

## Requirements

* A Telegram Mini App (a web app opened inside Telegram).
* An SDK key (`zgpu-sdk-…`) from the [Edge Operator Portal](https://edge.zerogpu.ai). When you add your app, select **Telegram Mini App** as an app type and use your `t.me` link as the website.
* The same browser requirements as the [Chrome / Web SDK](/docs/edge-providers/sdk/web#requirements): WebAssembly, IndexedDB, and Web Workers, all available in Telegram's in-app browser.

## Installation

Add the loader to your Mini App's HTML. It always resolves the latest released SDK:

```html theme={null}
<script async src="https://js-sdk.zerogpu.ai/dist/loader.js"></script>
```

## Initialization

The simplest setup is auto-init: put your SDK key on the script tag and the SDK starts when the page loads.

```html theme={null}
<script
  async
  src="https://js-sdk.zerogpu.ai/dist/loader.js"
  edgeOperatorKey="zgpu-sdk-your-key"
></script>
```

For manual init, use `ZeroGpuSdk.initZeroGpuSdk({ edgeOperatorKey })` exactly as in the [Chrome / Web SDK](/docs/edge-providers/sdk/web#initialization). All the same options apply.

## Authentication

The SDK authenticates with your **SDK key** (`zgpu-sdk-…`), passed as `edgeOperatorKey`. The key only joins devices to your fleet; it can't call the inference API or spend credits.

## Register device

Registration is automatic. Inside Telegram, the SDK:

* Detects `window.Telegram.WebApp` and registers the device with the `telegram-webapp` platform.
* Uses the Telegram user ID as `appUserId`, so you don't need to pass it.
* Downloads and caches its assigned model, then opens a connection for tasks.

## Start contributing compute

The device contributes compute while your Mini App is open. There's nothing else to call.

## Stop / pause

Keep the handle returned by `initZeroGpuSdk` and call `shutdown()` to stop:

```js theme={null}
const zerogpu = await ZeroGpuSdk.initZeroGpuSdk({ edgeOperatorKey: 'zgpu-sdk-your-key' });

// later, e.g. when the user turns resource sharing off
await zerogpu.shutdown();
```

There is no separate pause call. Closing the Mini App also stops contribution.

## Device lifecycle

The lifecycle matches the [Chrome / Web SDK](/docs/edge-providers/sdk/web#device-lifecycle): `starting`, `running`, `degraded`, `shutdown`, or `error`, with `degraded`, `recovered`, and `shutdown` events.

Telegram's in-app WebViews behave differently from regular browsers, so the SDK adapts its memory and storage checks there, and includes a guard that stops it from restarting repeatedly if the WebView keeps crashing and reloading.

## Example integration

```html index.html theme={null}
<!doctype html>
<html>
  <head>
    <script src="https://telegram.org/js/telegram-web-app.js"></script>
    <script
      async
      src="https://js-sdk.zerogpu.ai/dist/loader.js"
      edgeOperatorKey="zgpu-sdk-your-key"
      enableConsoleLogs="true"
    ></script>
  </head>
  <body>
    <!-- your Mini App -->
  </body>
</html>
```

To verify, open the Mini App in Telegram Desktop or Telegram Web with developer tools, and watch the console as the SDK registers the device, loads a model, and becomes ready. Turn `enableConsoleLogs` off for release.

## Troubleshooting

| Symptom | Fix |
| - | - |
| Rejected at registration / `401` | The SDK key is wrong or inactive. Check it in the Edge Operator Portal. |
| SDK pinned to an old version | Load `dist/loader.js`, not `zerogpu-browser-sdk.umd.js` directly. |
| No SDK activity in the console | Add `enableConsoleLogs="true"` to the script tag. |
| `Init skipped: crash-loop guard active` | The WebView crashed repeatedly while loading the model, so the SDK is in a cooldown. It starts again once the cooldown ends. |

See [Chrome / Web troubleshooting](/docs/edge-providers/sdk/web#troubleshooting) for worker and CSP issues.


# Chrome / Web
Source: https://docs.zerogpu.ai/docs/edge-providers/sdk/web

Contribute compute from websites and Chrome extensions with the ZeroGPU browser SDK.

## Overview

The ZeroGPU browser SDK turns the browsers running your website or Chrome extension into on-device inference nodes on the ZeroGPU network. It downloads a small model, registers the device, and serves inference tasks locally, in a Web Worker so your page's main thread stays responsive.

There are two ways to integrate:

| You're building | Use |
| - | - |
| **A website or web app** | The always-latest loader script from `js-sdk.zerogpu.ai`, or the npm package |
| **A Chrome extension (Manifest V3)** | The npm package, vendored into your extension with the `zerogpu` CLI. MV3 doesn't allow remotely hosted code. |

Building a Telegram Mini App? See [Telegram Mini Apps](/docs/edge-providers/sdk/telegram).

## Requirements

* A modern desktop or mobile browser with WebAssembly, IndexedDB (for the model cache), and Web Workers.
* An SDK key (`zgpu-sdk-…`) from the [Edge Operator Portal](https://edge.zerogpu.ai).
* **Websites with a strict Content Security Policy:** allow `blob:` workers, or host the worker yourself and set `overrides.inference.workerUrl`.
* **Chrome extensions:** Manifest V3, plus Node.js and npm at build time only. Nothing from `node_modules` ships in your extension.

## Installation

<Tabs>
  <Tab title="Website">
    Add the loader to your page. It always resolves the latest released SDK, so you never pin or bump versions:

    ```html theme={null}
    <script async src="https://js-sdk.zerogpu.ai/dist/loader.js"></script>
    ```

    Or install from npm if you bundle your own JavaScript:

    ```bash theme={null}
    npm install @zerogpu/browser-extension-sdk
    ```
  </Tab>

  <Tab title="Chrome extension">
    ```bash theme={null}
    npm install @zerogpu/browser-extension-sdk
    npx zerogpu init
    ```

    `npx zerogpu init` copies the SDK runtime into `vendor/zerogpu/`, creates `offscreen.html`, `offscreen.js`, and `background.js` (it never overwrites your files), and adds what the SDK needs to `manifest.json`: the `offscreen` and `storage` permissions, a CSP entry allowing `'wasm-unsafe-eval'`, host permissions for model downloads, and an `externally_connectable` entry for the ZeroGPU settings page.

    Using your own service worker? Merge the generated `background.js` snippet into it, and add `"prebuild": "zerogpu vendor"` to your build scripts.
  </Tab>
</Tabs>

## Initialization

<Tabs>
  <Tab title="Website">
    **Script tag (auto-init).** The SDK starts as soon as `edgeOperatorKey` is present:

    ```html theme={null}
    <script
      async
      src="https://js-sdk.zerogpu.ai/dist/loader.js"
      edgeOperatorKey="zgpu-sdk-your-key"
    ></script>
    ```

    **Manual init.** Load the loader without a key and call `initZeroGpuSdk`. Calls made before the SDK finishes loading are queued and replayed:

    ```html theme={null}
    <script async src="https://js-sdk.zerogpu.ai/dist/loader.js"></script>
    <script>
      ZeroGpuSdk.initZeroGpuSdk({
        edgeOperatorKey: 'zgpu-sdk-your-key'
      });
    </script>
    ```

    **ESM:**

    ```ts theme={null}
    import { initZeroGpuSdk } from '@zerogpu/browser-extension-sdk';

    const zerogpu = await initZeroGpuSdk({
      edgeOperatorKey: 'zgpu-sdk-your-key',
    });
    ```

    | Option | Description |
    | - | - |
    | `edgeOperatorKey` | Your SDK key. Required. |
    | `appUserId` | Optional stable user identifier for your own reporting. |
    | `customParams` | Up to three reporting slots, `cust_param1`–`cust_param3`. Values only; no personal data. Script tag: `data-cust-param1`…`3`. |
    | `overrides.telemetry.enableConsoleLogs` | Log SDK activity to the console (redacted). Script tag: `enableConsoleLogs="true"`. |
    | `overrides.inference.workerUrl` | Same-origin worker URL for pages whose CSP forbids `blob:` workers. |
  </Tab>

  <Tab title="Chrome extension">
    Open the generated `offscreen.js` and add your SDK key:

    ```js offscreen.js theme={null}
    import { initZeroGpuOffscreen } from "./vendor/zerogpu/zerogpu-offscreen.js";

    initZeroGpuOffscreen({
      edgeOperatorKey: "zgpu-sdk-your-key",
      enableConsoleLogs: true
    });
    ```

    | Option | Required | Description |
    | - | - | - |
    | `edgeOperatorKey` | yes | Your SDK key. |
    | `enableConsoleLogs` | no | Log SDK activity to the offscreen console. |
    | `inferenceWorker` | no | Run inference in the SDK's worker. Default `true`. |
    | `appUserId` | no | A stable identifier for the current user. |
    | `customParams` | no | `{ cust_param1, cust_param2, cust_param3 }` reporting slots. |
  </Tab>
</Tabs>

## Authentication

The SDK authenticates with your **SDK key** (`zgpu-sdk-…`), passed as `edgeOperatorKey`. The key can only join devices to your fleet; it can't call the inference API or spend credits. Still, inject it at build time rather than committing it.

## Register device

Registration is automatic. On start, the SDK collects coarse device capabilities (platform, CPU cores, approximate memory, battery level, connection type), downloads and caches its assigned model in IndexedDB, registers with the ZeroGPU device registry, and opens a WebSocket for tasks.

## Start contributing compute

* **Websites:** the SDK starts when it's initialized with your SDK key and keeps contributing while the page is open.
* **Chrome extensions:** the SDK runs in a background offscreen document and starts automatically whenever your extension loads.

## Stop / pause

<Tabs>
  <Tab title="Website">
    `initZeroGpuSdk` returns a handle. Call `shutdown()` to stop gracefully:

    ```ts theme={null}
    await zerogpu.shutdown();
    ```

    Shutdown lasts until the page initializes the SDK again; there is no separate pause call.
  </Tab>

  <Tab title="Chrome extension">
    Users must be able to turn resource sharing off. New installs are opted in; while a user is opted out, the offscreen document is never created, so nothing starts or connects. The choice persists across restarts.

    Add the opt-out link to an extension page you already show, such as your popup:

    ```html theme={null}
    <a data-zerogpu-settings>Resource sharing</a>
    <script src="vendor/zerogpu/zerogpu-settings-link.js"></script>
    ```

    Or control it from your service worker:

    ```js theme={null}
    await ZeroGpuBackground.getZeroGpuConsent();
    await ZeroGpuBackground.setZeroGpuConsent(false); // stops the SDK, stays off
    ```

    If your consent flow requires an explicit yes first, write `false` on install and flip it when the user agrees.
  </Tab>
</Tabs>

## Device lifecycle

On websites, the handle's `status` is one of `starting`, `running`, `degraded`, `shutdown`, or `error`. Subscribe to changes:

```ts theme={null}
zerogpu.on('degraded', ({ code, message }) => { /* resource pressure */ });
zerogpu.on('recovered', () => { /* back to normal */ });
zerogpu.on('shutdown', ({ code }) => { /* the SDK stopped */ });
```

`initZeroGpuSdk` never throws: if the SDK can't run on a device, `status` reports it instead of breaking your page. Before downloading models, the SDK checks memory and storage, and blocks downloads when the battery is below 20% and the device isn't charging.

## Example integration

<Tabs>
  <Tab title="Website">
    ```html index.html theme={null}
    <!doctype html>
    <html>
      <head>
        <script
          async
          src="https://js-sdk.zerogpu.ai/dist/loader.js"
          edgeOperatorKey="zgpu-sdk-your-key"
          enableConsoleLogs="true"
        ></script>
      </head>
      <body>
        <!-- your app -->
      </body>
    </html>
    ```

    Open the page and check the console: with `enableConsoleLogs` on, you'll see the SDK register the device, load a model, and become ready.
  </Tab>

  <Tab title="Chrome extension">
    1. Go to `chrome://extensions`, turn on **Developer mode**, and click **Load unpacked**.
    2. On your extension's card, open **Inspect views: offscreen.html**.
    3. With `enableConsoleLogs: true`, watch the SDK start, register the device, load a model, and become ready.

    Check your setup at any time:

    ```bash theme={null}
    npx zerogpu doctor
    ```

    After each SDK update, run `npx zerogpu vendor` (or wire it into `prebuild`) and republish your extension.
  </Tab>
</Tabs>

## Troubleshooting

| Symptom | Fix |
| - | - |
| Rejected at registration / `401` | The SDK key is wrong or inactive. Check it in the Edge Operator Portal. |
| `Inference worker unavailable; models run on the main thread` | Websites: your CSP blocks `blob:` workers; set `overrides.inference.workerUrl`. Extensions: the vendored worker is missing or stale; run `npx zerogpu vendor`. |
| Extension: CSP error mentioning `'wasm-unsafe-eval'` | Your manifest's CSP must allow `'wasm-unsafe-eval'`. Re-run `npx zerogpu init` or `npx zerogpu doctor`. |
| Extension: import fails / runtime files missing | Run `npx zerogpu vendor` and confirm `vendor/zerogpu/` exists. |
| Extension: model download blocked | Allow the model host in your manifest's host permissions; `npx zerogpu doctor` flags this. |
| Extension: out of date after an update | `npx zerogpu doctor` reports drift; run `npx zerogpu vendor`. |
| Website pinned to an old SDK | You're loading `zerogpu-browser-sdk.umd.js` directly. Use `dist/loader.js` instead. |


# Embeddings
Source: https://docs.zerogpu.ai/docs/embeddings

384-dimensional text embeddings for semantic search, RAG, clustering, and deduplication.

Embedding models turn text into a vector so that distance between vectors tracks closeness in meaning. That is what powers semantic search, retrieval-augmented generation, deduplication, clustering, and recommendation. Call the dedicated [Embeddings API](/api-reference/embeddings) (`POST /v1/embeddings`). An embedding model is routable only on this endpoint and returns OpenAI's native embeddings envelope, so an existing client only needs its base URL and model id changed.

| Model | Input /1M | Output /1M | Max tokens |
| - | -: | -: | -: |
| <a href="/api-reference/models/all-minilm-l6-v2"><img alt="all-minilm-l6-v2" /> <code>all-minilm-l6-v2</code></a> | \$0.004 | Not billed | 512 |
| <a href="/api-reference/models/bge-small-en-v1-5"><img alt="bge-small-en-v1.5" /> <code>bge-small-en-v1.5</code></a> | \$0.004 | Not billed | 512 |

Both models return **384-dimensional** vectors, so they are interchangeable in an existing index. Embeddings are billed on input tokens only.

## all-minilm-l6-v2

> Sentence-Transformers' all-MiniLM-L6-v2 is the default workhorse of semantic search. It maps a sentence or short paragraph to a 384-dimensional vector, trained with contrastive learning on more than a billion sentence pairs, so cosine distance between two vectors tracks how close the two texts are in meaning. At 22.7M parameters it embeds fast and cheap enough to index a whole corpus and re-embed it whenever your content changes. Inputs are truncated at 256 tokens, so embed chunks rather than whole documents.

**References:** [Model card](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) • [Terms](https://zerogpu.ai/terms) • [Privacy](https://zerogpu.ai/privacy-policy)

```bash Embeddings API theme={null}
curl https://api.zerogpu.ai/v1/embeddings \
  -H "content-type: application/json" \
  -H "x-api-key: YOUR_API_KEY" \
  -H "x-project-id: YOUR_PROJECT_ID" \
  -d '{
  "model": "all-minilm-l6-v2",
  "input": "ZeroGPU runs high-volume inference tasks on small models at the edge."
}'
```

```json Response theme={null}
{
  "object": "list",
  "data": [
    {
      "object": "embedding",
      "index": 0,
      "embedding": [0.012601, -0.072932, 0.043332, 0.028129, 0.033064, -0.078302, -0.153619, -0.011621]
    }
  ],
  "model": "all-MiniLM-L6-v2",
  "usage": { "prompt_tokens": 18, "total_tokens": 18 }
}
```

The vector above is truncated for readability; a real response carries all 384 values. The `model` field reads `all-MiniLM-L6-v2`, the upstream capitalization of the id you send.

## bge-small-en-v1.5

> BAAI's BGE-small-en-v1.5 is a retrieval-first English embedding model and one of the strongest performers on the MTEB benchmark for its size. It produces the same 384-dimensional vectors as all-minilm-l6-v2, so it is a drop-in swap in an existing index, but it takes a 512-token window instead of 256 and is tuned specifically for dense retrieval rather than general sentence similarity. At 33M parameters it is the one to reach for when the job is ranking passages against a query.

**References:** [Model card](https://huggingface.co/BAAI/bge-small-en-v1.5) • [Terms](https://zerogpu.ai/terms) • [Privacy](https://zerogpu.ai/privacy-policy)

Embed a batch by passing an array. Each `data[i].index` maps a vector back to its input.

```bash Batch embeddings theme={null}
curl https://api.zerogpu.ai/v1/embeddings \
  -H "content-type: application/json" \
  -H "x-api-key: YOUR_API_KEY" \
  -d '{
  "model": "bge-small-en-v1.5",
  "input": ["first text", "second text"]
}'
```

## Which one to pick

Start with `all-minilm-l6-v2` for general semantic similarity over short chunks. Move to `bge-small-en-v1.5` when the job is English retrieval, when your chunks run past 256 tokens, or when retrieval quality is the bottleneck. Both cost the same and return the same vector width, so switching is a one-line change plus a reindex.


# How ZeroGPU works
Source: https://docs.zerogpu.ai/docs/how-zerogpu-works

ZeroGPU turns idle compute into one programmable inference layer - specialized small models, an edge-powered network, and routing handled for you.

ZeroGPU turns idle compute into one programmable inference layer. You send a request; we run it on a specialized small or nano model, on the cheapest compute that can serve it well.

## Specialized small and nano models

ZeroGPU runs purpose-built ZLMs (ZeroGPU Language Models) for high-volume tasks like IAB classification and signal extraction, alongside a catalog of open small and nano models - DeBERTa, GLiNER, LFM2.5, Llama 3.1 8B. Small enough to run at the edge, good enough for production. See the [Model Catalog](/docs/model-catalog).

## An edge-powered network

Requests run across a hybrid of:

<Columns>
  <Card title="Edge devices" icon="mobile-screen">
    Phones, gaming PCs.
  </Card>

  <Card title="Optimized edge servers" icon="server">
    Mid-sized models, higher load.
  </Card>

  <Card title="Cloud fallback" icon="cloud">
    Consistent performance and burst capacity.
  </Card>
</Columns>

## Routing

For each request, ZeroGPU picks the right model and the right compute by capability, availability, and load - and routes geographically to cut latency. You call one endpoint; the orchestration is handled for you.

<Note>
  This is network-side routing (which model, which compute). It's distinct from the ZeroGPU Router in the skills/plugins, which decides - on the client side - which steps of an agent run to offload to ZeroGPU at all. See [Integrations → Claude Code Plugin](/integrations/claude-code-plugin).
</Note>


# Model Catalog
Source: https://docs.zerogpu.ai/docs/model-catalog

Browse ZeroGPU models, compare pricing, and pick the model ID to call.

The **Model Catalog** lets you browse every available ZeroGPU model and compare **pricing** across the tasks you care about. It's useful when you're selecting which `model` identifier to send to `POST /v1/responses`.

Sections on this page: [At a glance](#at-a-glance) (pricing table), [Detailed model cards](#detailed-model-cards), and [Model library by task](#model-library-by-task).

## At a glance

| Model | Input /1M | Output /1M | Cached input /1M | Task | Max tokens |
| - | -: | -: | -: | - | -: |
| <a href="/api-reference/models/deepseek-v4-1-flash"><img alt="deepseek-v4.1-flash" /> <code>deepseek-v4.1-flash</code></a> | \$0.30 | \$1.20 | \$0.003 | Text Generation | 1,048,576 |
| <a href="/api-reference/models/glm-5-3-flash"><img alt="glm-5.3-flash" /> <code>glm-5.3-flash</code></a> | \$0.10 | \$0.35 | \$0.015 | Text Generation | 1,048,576 |
| <a href="/api-reference/models/gpt-5-4-nano"><img alt="gpt-5.4-nano" /> <code>gpt-5.4-nano</code></a> | \$0.20 | \$1.25 | \$0.02 | Text Generation | 400,000 |
| <a href="/api-reference/models/gpt-5-6-luna"><img alt="gpt-5.6-luna" /> <code>gpt-5.6-luna</code></a> | \$0.20 | \$1.20 | \$0.20 | Text Generation | 272,000 |
| <a href="/api-reference/models/gpt-4-1-mini"><img alt="gpt-4.1-mini" /> <code>gpt-4.1-mini</code></a> | \$0.40 | \$1.60 | \$0.10 | Text Generation | 1,047,576 |
| <a href="/api-reference/models/gpt-oss-120b"><img alt="gpt-oss-120b" /> <code>gpt-oss-120b</code></a> | \$0.15 | \$0.60 | \$0.03 | Text Generation | 131,072 |
| <a href="/api-reference/models/qwen3-30b-a3b-fp8"><img alt="qwen3-30b-a3b-fp8" /> <code>qwen3-30b-a3b-fp8</code></a> | \$0.10 | \$0.45 | \$0.05 | Text Generation | 32,768 |
| <a href="/api-reference/models/llama-3-1-8b-instruct-fast"><img alt="llama-3.1-8b-instruct-fast" /> <code>llama-3.1-8b-instruct-fast</code></a> | \$0.15 | \$0.28 | \$0.025 | Text Generation | 131,072 |
| <a href="/api-reference/models/llama-guard-4-12b"><img alt="llama-guard-4-12b" /> <code>llama-guard-4-12b</code></a> | \$0.18 | \$0.18 | — | Text Generation | 163,840 |
| <a href="/api-reference/models/zlm-v2-iab-classify-edge-enriched"><img alt="zlm-v2-iab-classify-edge-enriched" /> <code>zlm-v2-iab-classify-edge-enriched</code></a> | \$0.05 | \$0.15 | — | Text Classification | 800 |
| <a href="/api-reference/models/zlm-v1-iab-classify-edge"><img alt="zlm-v1-iab-classify-edge" /> <code>zlm-v1-iab-classify-edge</code></a> | \$0.02 | \$0.05 | — | Text Classification | 400 |
| <a href="/api-reference/models/zlm-v1-iab-domain-classifier"><img alt="zlm-v1-iab-domain-classifier" /> <code>zlm-v1-iab-domain-classifier</code></a> | \$0.02 | \$0.05 | — | Text Classification | 100 |
| <a href="/api-reference/models/zlm-v1-moderation-edge"><img alt="zlm-v1-moderation-edge" /> <code>zlm-v1-moderation-edge</code></a> | \$0.10 | \$0.15 | — | Text Moderation | 800 |
| <a href="/api-reference/models/gliner2-5-base-v1"><img alt="gliner2.5-base-v1" /> <code>gliner2.5-base-v1</code></a> | \$0.02 | \$0.05 | — | Text Classification | — |
| <a href="/api-reference/models/gliner2-5-multi-v1"><img alt="gliner2.5-multi-v1" /> <code>gliner2.5-multi-v1</code></a> | \$0.02 | \$0.05 | — | Text Classification | — |
| <a href="/api-reference/models/deberta-v3-small"><img alt="deberta-v3-small" /> <code>deberta-v3-small</code></a> | \$0.02 | \$0.05 | — | Text Classification | 400 |
| <a href="/api-reference/models/lfm2-5-1-2b-thinking"><img alt="LFM2.5-1.2B-Thinking" /> <code>LFM2.5-1.2B-Thinking</code></a> | \$0.02 | \$0.05 | — | Text Generation | 32,768 |
| <a href="/api-reference/models/lfm2-5-1-2b-instruct"><img alt="LFM2.5-1.2B-Instruct" /> <code>LFM2.5-1.2B-Instruct</code></a> | \$0.02 | \$0.05 | — | Text Generation | 32,768 |
| <a href="/api-reference/models/qwen3-6-35b-a3b"><img alt="qwen3.6-35b-a3b" /> <code>qwen3.6-35b-a3b</code></a> | \$0.20 | \$1.50 | \$0.20 | Text Generation | — |
| <a href="/api-reference/models/zlm-v1-signal-extract"><img alt="zlm-v1-signal-extract" /> <code>zlm-v1-signal-extract</code></a> | \$0.04 | \$0.10 | — | Text Classification | 400 |
| <a href="/api-reference/models/gliner2-5-decide-v1"><img alt="gliner2.5-decide-v1" /> <code>gliner2.5-decide-v1</code></a> | \$0.004 | Not billed | — | Text Classification | — |
| <a href="/api-reference/models/all-minilm-l6-v2"><img alt="all-minilm-l6-v2" /> <code>all-minilm-l6-v2</code></a> | \$0.004 | Not billed | — | Text Embedding | 512 |
| <a href="/api-reference/models/bge-small-en-v1-5"><img alt="bge-small-en-v1.5" /> <code>bge-small-en-v1.5</code></a> | \$0.004 | Not billed | — | Text Embedding | 512 |

[Audio models](/docs/audio) are called on `POST /v1/audio/transcriptions` and `POST /v1/audio/speech` rather than `/v1/responses`, and are billed per minute of audio instead of per token, rounded up to the next whole minute with a one-minute minimum.

| Model | Price /min | Task | Limits |
| - | -: | - | - |
| <a href="/api-reference/models/whisper-tiny"><img alt="whisper-tiny" /> <code>whisper-tiny</code></a> | \$0.02 | Speech-to-Text | 25 MB, 10 minutes of audio |
| <a href="/api-reference/models/chatterbox-nano"><img alt="chatterbox-nano" /> <code>chatterbox-nano</code></a> | \$0.01 | Text-to-Speech | 2,000 characters |

## Detailed model cards

<CardGroup>
  <Card href="/api-reference/models/deepseek-v4-1-flash">
    <div>
      <img alt="deepseek-v4.1-flash" />

      <div>deepseek-v4.1-flash</div>
    </div>

    <div><span>1,048,576 context window</span><span>\$0.30 / 1M input</span><span>\$1.20 / 1M output</span><span>\$0.003 / 1M cached input</span></div>

    DeepSeek's DeepSeek-V4.1-Flash is an open-weight sparse Mixture-of-Experts model and the first built on DeepSeek's Causal Encoder-Decoder (CED) architecture, activating 8B parameters on input and 16B on output. It…
  </Card>

  <Card href="/api-reference/models/glm-5-3-flash">
    <div>
      <img alt="glm-5.3-flash" />

      <div>glm-5.3-flash</div>
    </div>

    <div><span>1,048,576 context window</span><span>\$0.10 / 1M input</span><span>\$0.35 / 1M output</span><span>\$0.015 / 1M cached input</span></div>

    Z.ai's GLM-5.3-Flash is an efficient open-weight model for coding and long-horizon agent tasks, served on ZeroGPU for general text generation. Its hybrid sparse and linear attention keeps…
  </Card>

  <Card href="/api-reference/models/gpt-5-4-nano">
    <div>
      <img alt="gpt-5.4-nano" />

      <div>gpt-5.4-nano</div>
    </div>

    <div><span>400,000 context window</span><span>\$0.20 / 1M input</span><span>\$1.25 / 1M output</span><span>\$0.02 / 1M cached input</span></div>

    OpenAI's GPT-5.4 nano is the most cost-efficient model in the GPT-5.4 family, served on ZeroGPU for high-volume and latency-sensitive workloads such as classification, extraction, routing, and…
  </Card>

  <Card href="/api-reference/models/gpt-5-6-luna">
    <div>
      <img alt="gpt-5.6-luna" />

      <div>gpt-5.6-luna</div>
    </div>

    <div><span>272,000 context window</span><span>\$0.20 / 1M input</span><span>\$1.20 / 1M output</span><span>\$0.20 / 1M cached input</span></div>

    OpenAI's GPT-5.6 Luna is the cost-optimized model of the GPT-5.6 family, served on ZeroGPU for cost-sensitive, high-volume workloads. It supports adjustable reasoning effort, function calling, and…
  </Card>

  <Card href="/api-reference/models/gpt-4-1-mini">
    <div>
      <img alt="gpt-4.1-mini" />

      <div>gpt-4.1-mini</div>
    </div>

    <div><span>1,047,576 context window</span><span>\$0.40 / 1M input</span><span>\$1.60 / 1M output</span><span>\$0.10 / 1M cached input</span></div>

    OpenAI's GPT-4.1 mini is the fast, cost-efficient model of the GPT-4.1 family, served on ZeroGPU for general text generation. It excels at instruction following and tool calling, supports function…
  </Card>

  <Card href="/api-reference/models/gpt-oss-120b">
    <div>
      <img alt="gpt-oss-120b" />

      <div>gpt-oss-120b</div>
    </div>

    <div><span>131,072 context window</span><span>\$0.15 / 1M input</span><span>\$0.60 / 1M output</span><span>\$0.03 / 1M cached input</span></div>

    OpenAI's gpt-oss-120b is an open-weight Mixture-of-Experts model with 117B total parameters (5.1B active per token), served on ZeroGPU for general text generation. It reasons through a problem…
  </Card>

  <Card href="/api-reference/models/qwen3-30b-a3b-fp8">
    <div>
      <img alt="qwen3-30b-a3b-fp8" />

      <div>qwen3-30b-a3b-fp8</div>
    </div>

    <div><span>32,768 context window</span><span>\$0.10 / 1M input</span><span>\$0.45 / 1M output</span><span>\$0.05 / 1M cached input</span></div>

    Alibaba's Qwen3-30B-A3B is an open-weight Mixture-of-Experts model with 30.5B total parameters (3.3B active per token), served on ZeroGPU as an FP8 build for efficient inference. It thinks through a problem…
  </Card>

  <Card href="/api-reference/models/llama-3-1-8b-instruct-fast">
    <div>
      <img alt="llama-3.1-8b-instruct-fast" />

      <div>llama-3.1-8b-instruct-fast</div>
    </div>

    <div><span>131,072 max tokens</span><span>\$0.15 / 1M input</span><span>\$0.28 / 1M output</span><span>\$0.025 / 1M cached input</span></div>

    Meta's Llama 3.1 Instruct, tuned for fast, low-cost summarization at scale on the ZeroGPU edge network. Its 128K-token context window takes in entire documents, long transcripts, and full email or…
  </Card>

  <Card href="/api-reference/models/llama-guard-4-12b">
    <div>
      <img alt="llama-guard-4-12b" />

      <div>llama-guard-4-12b</div>
    </div>

    <div><span>Text Generation</span><span>Safety classification</span><span>Text Moderation</span><span>Brand Safety</span><span>12B params</span><span>163,840 context window</span><span>\$0.18 / 1M input</span><span>\$0.18 / 1M output</span></div>

    Meta's Llama Guard 4 12B is a multimodal safety classification model for moderating text, images, and mixed text-image inputs. It evaluates both incoming prompts…
  </Card>

  <Card href="/api-reference/models/zlm-v2-iab-classify-edge-enriched">
    <div>
      <img alt="zlm-v2-iab-classify-edge-enriched" />

      <div>zlm-v2-iab-classify-edge-enriched</div>
    </div>

    <div><span>800 max tokens</span><span>\$0.05 / 1M input</span><span>\$0.15 / 1M output</span></div>

    The enriched variant of ZeroGPU's IAB classifier turns a single inference call into a full content-intelligence profile not just a label, but everything a contextual pipeline needs to act on. Each…
  </Card>

  <Card href="/api-reference/models/zlm-v1-iab-classify-edge">
    <div>
      <img alt="zlm-v1-iab-classify-edge" />

      <div>zlm-v1-iab-classify-edge</div>
    </div>

    <div><span>400 max tokens</span><span>\$0.02 / 1M input</span><span>\$0.05 / 1M output</span></div>

    ZeroGPU's IAB classifier maps any text to the industry-standard IAB Content Taxonomy in a single, fast inference call. Each call returns categories across both the 1.0 and 2.2 taxonomies plus matched…
  </Card>

  <Card href="/api-reference/models/zlm-v1-iab-domain-classifier">
    <div>
      <img alt="zlm-v1-iab-domain-classifier" />

      <div>zlm-v1-iab-domain-classifier</div>
    </div>

    <div><span>100 max tokens</span><span>\$0.02 / 1M input</span><span>\$0.05 / 1M output</span></div>

    ZeroGPU's Domain IAB Classifier maps a raw domain name straight to the IAB Content Taxonomy, returning content categories, topics, keywords, and user-intent signals. It needs only the domain as input, cutting payload size by up to 10x versus page-level…
  </Card>

  <Card href="/api-reference/models/zlm-v1-moderation-edge">
    <div>
      <img alt="zlm-v1-moderation-edge" />

      <div>zlm-v1-moderation-edge</div>
    </div>

    <div><span>800 max tokens</span><span>\$0.10 / 1M input</span><span>\$0.15 / 1M output</span></div>

    ZeroGPU's moderation model returns the complete OpenAI 13-category taxonomy — a flagged verdict, category booleans, and calibrated scores — as a drop-in for omni-moderation-latest. In head-to-head benchmarks it wins the binary safe/unsafe decision (0.899 vs 0.853 F1) and 9 of 13 harm categories, and returns verdicts 1.2–1.8× faster on production-range…
  </Card>

  <Card href="/api-reference/models/gliner2-5-base-v1">
    <div>
      <img alt="gliner2.5-base-v1" />

      <div>gliner2.5-base-v1</div>
    </div>

    <div><span>194M params</span><span>\$0.02 / 1M input</span><span>\$0.05 / 1M output</span></div>

    Fastino's GLiNER2.5 Base is an English schema-based information extraction model built on DeBERTa-v3-base. A single model covers entity extraction, text classification, structured record extraction…
  </Card>

  <Card href="/api-reference/models/gliner2-5-multi-v1">
    <div>
      <img alt="gliner2.5-multi-v1" />

      <div>gliner2.5-multi-v1</div>
    </div>

    <div><span>287M params</span><span>\$0.02 / 1M input</span><span>\$0.05 / 1M output</span></div>

    Fastino's GLiNER2.5 Multi is a multilingual information extraction model built on mDeBERTa-v3-base. It supports zero-shot named entity recognition, text classification, and structured JSON extraction…
  </Card>

  <Card href="/api-reference/models/deberta-v3-small">
    <div>
      <img alt="deberta-v3-small" />

      <div>deberta-v3-small</div>
    </div>

    <div><span>400 max tokens</span><span>\$0.02 / 1M input</span><span>\$0.05 / 1M output</span></div>

    Microsoft's DeBERTa-v3-small is a fast, lightweight zero-shot text classifier for high-volume routing, filtering, and tagging. Hand it any text alongside your own candidate labels and it returns a…
  </Card>

  <Card href="/api-reference/models/lfm2-5-1-2b-thinking">
    <div>
      <img alt="LFM2.5-1.2B-Thinking" />

      <div>LFM2.5-1.2B-Thinking</div>
    </div>

    <div><span>32,768 max tokens</span><span>\$0.02 / 1M input</span><span>\$0.05 / 1M output</span></div>

    Liquid AI's LFM2.5-1.2B-Thinking is a compact reasoning model that works through a problem step by step before it answers. Built by Liquid AI, it generates an explicit chain-of-thought trace so for…
  </Card>

  <Card href="/api-reference/models/lfm2-5-1-2b-instruct">
    <div>
      <img alt="LFM2.5-1.2B-Instruct" />

      <div>LFM2.5-1.2B-Instruct</div>
    </div>

    <div><span>32,768 max tokens</span><span>\$0.02 / 1M input</span><span>\$0.05 / 1M output</span></div>

    Liquid AI's LFM2.5-1.2B-Instruct is a hybrid architecture model purpose-built for on-device deployment, trained on 28 trillion tokens with multi-stage reinforcement learning. It delivers…
  </Card>

  <Card href="/api-reference/models/qwen3-6-35b-a3b">
    <div>
      <img alt="qwen3.6-35b-a3b" />

      <div>qwen3.6-35b-a3b</div>
    </div>

    <div><span>35B params</span><span>\$0.20 / 1M input</span><span>\$1.50 / 1M output</span><span>\$0.20 / 1M cached input</span></div>

    Alibaba's Qwen3.6-35B-A3B is a fast, open-weight multimodal model for applications that need to understand text, images, and video, served on ZeroGPU as an FP8 build. It suits visual question answering, document and screenshot analysis…
  </Card>

  <Card href="/api-reference/models/zlm-v1-signal-extract">
    <div>
      <img alt="zlm-v1-signal-extract" />

      <div>zlm-v1-signal-extract</div>
    </div>

    <div><span>Text Classification</span><span>Signal extraction</span><span>Ad Tech</span><span>Chat</span><span>80M params</span><span>400 context window</span><span>\$0.04 / 1M input</span><span>\$0.10 / 1M output</span></div>

    ZeroGPU's signal extractor turns unstructured text into structured signals that downstream systems can act on. One inference call returns topics, keywords, intent…
  </Card>

  <Card href="/api-reference/models/gliner2-5-decide-v1">
    <div>
      <img alt="gliner2.5-decide-v1" />

      <div>gliner2.5-decide-v1</div>
    </div>

    <div><span>486M params</span><span>\$0.004 / 1M input</span></div>

    Fastino's GLiNER2.5 Decide is a 486M-parameter zero-shot decision and classification model built for operational AI workflows. You supply the labels or decision options at request time to classify intent, route agents and tools, triage support requests, make moderation decisions…
  </Card>

  <Card href="/api-reference/models/all-minilm-l6-v2">
    <div>
      <img alt="all-minilm-l6-v2" />

      <div>all-minilm-l6-v2</div>
    </div>

    <div><span>512 max tokens</span><span>\$0.004 / 1M input</span></div>

    Sentence-Transformers' all-MiniLM-L6-v2 is the default workhorse of semantic search. It maps a sentence or short paragraph to a 384-dimensional vector, trained with contrastive learning on more than a billion sentence pairs…
  </Card>

  <Card href="/api-reference/models/bge-small-en-v1-5">
    <div>
      <img alt="bge-small-en-v1.5" />

      <div>bge-small-en-v1.5</div>
    </div>

    <div><span>512 max tokens</span><span>\$0.004 / 1M input</span></div>

    BAAI's BGE-small-en-v1.5 is a retrieval-first English embedding model and one of the strongest performers on the MTEB benchmark for its size. It produces the same 384-dimensional vectors, with a 512-token window…
  </Card>

  <Card href="/api-reference/models/whisper-tiny">
    <div>
      <img alt="whisper-tiny" />

      <div>whisper-tiny</div>
    </div>

    <div><span>Speech-to-Text</span><span>39M params</span><span>\$0.02 / minute</span></div>

    OpenAI's Whisper Tiny is the smallest model in the Whisper family of speech recognition models, at 39M parameters. It is the multilingual checkpoint: it transcribes speech in the language it was spoken…
  </Card>

  <Card href="/api-reference/models/chatterbox-nano">
    <div>
      <img alt="chatterbox-nano" />

      <div>chatterbox-nano</div>
    </div>

    <div><span>Text-to-Speech</span><span>Voice cloning</span><span>110M params</span><span>\$0.01 / minute</span></div>

    Resemble AI's Chatterbox Nano is a 110M-parameter English text-to-speech model built for fast, low-cost speech generation. It speaks your text in a built-in voice, or clones a speaker…
  </Card>
</CardGroup>

## Model library by task

<CardGroup>
  <Card title="Open Weight" icon="cubes" href="/docs/open-weight">
    7 models: `deepseek-v4.1-flash`, `glm-5.3-flash`, `qwen3-30b-a3b-fp8`, `qwen3.6-35b-a3b`, `gpt-oss-120b`, `llama-3.1-8b-instruct-fast`, `llama-guard-4-12b`
  </Card>

  <Card title="Text Generation" icon="wand-magic-sparkles" href="/docs/text-generation">
    12 models: `deepseek-v4.1-flash`, `glm-5.3-flash`, `gpt-5.4-nano`, `gpt-5.6-luna`, `gpt-4.1-mini`, `gpt-oss-120b`, `qwen3-30b-a3b-fp8`, `qwen3.6-35b-a3b`, `llama-3.1-8b-instruct-fast`, `llama-guard-4-12b`, `LFM2.5-1.2B-Thinking`, `LFM2.5-1.2B-Instruct`
  </Card>

  <Card title="Audio Models" icon="microphone" href="/docs/audio">
    2 models: `whisper-tiny`, `chatterbox-nano`
  </Card>

  <Card title="Text Classification" icon="tags" href="/docs/text-classification">
    8 models: `zlm-v2-iab-classify-edge-enriched`, `zlm-v1-iab-classify-edge`, `zlm-v1-iab-domain-classifier`, `deberta-v3-small`, `zlm-v1-signal-extract`, `gliner2.5-base-v1`, `gliner2.5-multi-v1`, `gliner2.5-decide-v1`
  </Card>

  <Card title="Moderation" icon="shield-halved" href="/docs/moderation">
    1 model: `zlm-v1-moderation-edge`
  </Card>

  <Card title="Text Embedding" icon="vector-square" href="/docs/embeddings">
    2 models: `all-minilm-l6-v2`, `bge-small-en-v1.5`
  </Card>

  <Card title="Ad Tech" icon="bullhorn" href="/docs/ad-tech">
    4 models: `zlm-v1-iab-classify-edge`, `zlm-v2-iab-classify-edge-enriched`, `zlm-v1-iab-domain-classifier`, `zlm-v1-signal-extract`
  </Card>
</CardGroup>


# Moderation
Source: https://docs.zerogpu.ai/docs/moderation

OpenAI-compatible content moderation, benchmarked against omni-moderation-latest.

Moderation models screen text for unsafe, harmful, or policy-sensitive content before it reaches your application or model workflow. Call the dedicated [Moderations API](/api-reference/moderations) (`POST /v1/moderations`) — a moderation model is routable only on this endpoint and returns OpenAI's native moderations envelope, so it drops into any pipeline written against `omni-moderation-latest`.

| Model | Input /1M | Output /1M | Max tokens |
| - | -: | -: | -: |
| <a href="/api-reference/models/zlm-v1-moderation-edge"><img alt="zlm-v1-moderation-edge" /> <code>zlm-v1-moderation-edge</code></a> | \$0.10 | \$0.15 | 800 |

## zlm-v1-moderation-edge

> ZeroGPU's moderation model screens text for unsafe, harmful, or policy-sensitive content and returns the complete OpenAI 13-category taxonomy (a `flagged` verdict, per-category booleans, and calibrated `category_scores`), so it drops into any pipeline written against `omni-moderation-latest`. Under the hood it's an 86M-parameter DeBERTa encoder with a shared trunk feeding one binary safe/unsafe head and 13 category heads, with per-category thresholds calibrated on held-out validation data. In [head-to-head benchmarks](https://zerogpu.ai/benchmarks/moderation-edge) against OpenAI omni-moderation it wins the binary safe/unsafe decision (0.899 vs 0.853 F1) and 9 of 13 harm categories, with the largest gains on graphic violence, illicit content, and self-harm, while returning verdicts 1.2–1.8× faster at the median on production-range inputs, because inference is co-located at the edge instead of a round trip to a central API. Moderation sits inline in front of every response your app serves; this is the model that's fast and accurate enough to live there.

**References:** [Moderation benchmark](https://zerogpu.ai/benchmarks/moderation-edge) • [Terms](https://zerogpu.ai/terms) • [Privacy](https://zerogpu.ai/privacy-policy)

Send the text to classify as `input` — a string, an array of strings (one verdict per element), or an array of `{ "type": "text", "text": "…" }` content parts.

<CodeGroup>
  ```bash Moderations API theme={null}
  curl https://api.zerogpu.ai/v1/moderations \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "model": "zlm-v1-moderation-edge",
    "input": "I am so angry at this person that I want to hurt them. They are worthless and should be scared of what I might do next."
  }'
  ```

  ```bash Chat Completions theme={null}
  curl https://api.zerogpu.ai/v1/chat/completions \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "model": "zlm-v1-moderation-edge",
    "messages": [
      {
        "role": "user",
        "content": "I am so angry at this person that I want to hurt them. They are worthless and should be scared of what I might do next."
      }
    ]
  }'
  ```
</CodeGroup>

On `/v1/moderations` the response is OpenAI's native moderations envelope — an `id`, the resolved `model`, and a `results` array (one entry per input). Each result carries a `flagged` verdict, per-category `categories` booleans, `category_scores` in `[0, 1]`, and `category_applied_input_types` (always `["text"]` for this text-only model), with all 13 categories present in OpenAI's order:

```json Response theme={null}
{
  "id": "modr-0a1b2c3d4e5f60718293a4b5c6d7e8f90",
  "model": "zlm-v1-moderation-edge",
  "results": [
    {
      "flagged": true,
      "categories": {
        "harassment": true,
        "harassment/threatening": true,
        "hate": false,
        "hate/threatening": false,
        "illicit": true,
        "illicit/violent": true,
        "self-harm": false,
        "self-harm/intent": false,
        "self-harm/instructions": false,
        "sexual": false,
        "sexual/minors": false,
        "violence": true,
        "violence/graphic": false
      },
      "category_scores": {
        "harassment": 0.82545,
        "harassment/threatening": 0.957703,
        "hate": 0.21574,
        "hate/threatening": 0.168273,
        "illicit": 0.732167,
        "illicit/violent": 0.780693,
        "self-harm": 0.015232,
        "self-harm/intent": 0.017179,
        "self-harm/instructions": 0.01807,
        "sexual": 0.035671,
        "sexual/minors": 0.094207,
        "violence": 0.665897,
        "violence/graphic": 0.198346
      },
      "category_applied_input_types": {
        "harassment": ["text"],
        "harassment/threatening": ["text"],
        "hate": ["text"],
        "hate/threatening": ["text"],
        "illicit": ["text"],
        "illicit/violent": ["text"],
        "self-harm": ["text"],
        "self-harm/intent": ["text"],
        "self-harm/instructions": ["text"],
        "sexual": ["text"],
        "sexual/minors": ["text"],
        "violence": ["text"],
        "violence/graphic": ["text"]
      }
    }
  ]
}
```


# Open Weight
Source: https://docs.zerogpu.ai/docs/open-weight

The open-weight frontier-class models served on ZeroGPU, with pricing and context windows.

Open-weight models ship with published weights and permissive licenses, so you can inspect them, self-host them, and avoid lock-in to a closed vendor. ZeroGPU serves seven of them — for reasoning, coding, agentic workflows, multimodal understanding, high-volume summarization, and safety classification — through the same OpenAI-compatible API as every other model in the [catalog](/docs/model-catalog).

Full request and response examples live on the [Text Generation](/docs/text-generation) page. Open-weight speech-to-text and text-to-speech models are on [Audio Models](/docs/audio).

## At a glance

| Model | Input /1M | Output /1M | Cached input /1M | Max tokens |
| - | -: | -: | -: | -: |
| <a href="/api-reference/models/deepseek-v4-1-flash"><img alt="deepseek-v4.1-flash" /> <code>deepseek-v4.1-flash</code></a> | \$0.30 | \$1.20 | \$0.003 | 1,048,576 |
| <a href="/api-reference/models/glm-5-3-flash"><img alt="glm-5.3-flash" /> <code>glm-5.3-flash</code></a> | \$0.10 | \$0.35 | \$0.015 | 1,048,576 |
| <a href="/api-reference/models/qwen3-30b-a3b-fp8"><img alt="qwen3-30b-a3b-fp8" /> <code>qwen3-30b-a3b-fp8</code></a> | \$0.10 | \$0.45 | \$0.05 | 32,768 |
| <a href="/api-reference/models/qwen3-6-35b-a3b"><img alt="qwen3.6-35b-a3b" /> <code>qwen3.6-35b-a3b</code></a> | \$0.20 | \$1.50 | \$0.20 | — |
| <a href="/api-reference/models/gpt-oss-120b"><img alt="gpt-oss-120b" /> <code>gpt-oss-120b</code></a> | \$0.15 | \$0.60 | \$0.03 | 131,072 |
| <a href="/api-reference/models/llama-3-1-8b-instruct-fast"><img alt="llama-3.1-8b-instruct-fast" /> <code>llama-3.1-8b-instruct-fast</code></a> | \$0.15 | \$0.28 | \$0.025 | 131,072 |
| <a href="/api-reference/models/llama-guard-4-12b"><img alt="llama-guard-4-12b" /> <code>llama-guard-4-12b</code></a> | \$0.18 | \$0.18 | — | 163,840 |

## Model cards

<CardGroup>
  <Card href="/api-reference/models/deepseek-v4-1-flash">
    <div>
      <img alt="deepseek-v4.1-flash" />

      <div>deepseek-v4.1-flash</div>
    </div>

    <div><span>1,048,576 context window</span><span>\$0.30 / 1M input</span><span>\$1.20 / 1M output</span><span>\$0.003 / 1M cached input</span></div>

    DeepSeek's DeepSeek-V4.1-Flash is an open-weight sparse Mixture-of-Experts model and the first built on DeepSeek's Causal Encoder-Decoder (CED) architecture, activating 8B parameters on input and 16B on output. It…
  </Card>

  <Card href="/api-reference/models/glm-5-3-flash">
    <div>
      <img alt="glm-5.3-flash" />

      <div>glm-5.3-flash</div>
    </div>

    <div><span>1,048,576 context window</span><span>\$0.10 / 1M input</span><span>\$0.35 / 1M output</span><span>\$0.015 / 1M cached input</span></div>

    Z.ai's GLM-5.3-Flash is an efficient open-weight model for coding and long-horizon agent tasks, served on ZeroGPU for general text generation. Its hybrid sparse and linear attention keeps…
  </Card>

  <Card href="/api-reference/models/qwen3-30b-a3b-fp8">
    <div>
      <img alt="qwen3-30b-a3b-fp8" />

      <div>qwen3-30b-a3b-fp8</div>
    </div>

    <div><span>32,768 context window</span><span>\$0.10 / 1M input</span><span>\$0.45 / 1M output</span><span>\$0.05 / 1M cached input</span></div>

    Alibaba's Qwen3-30B-A3B is an open-weight Mixture-of-Experts model with 30.5B total parameters (3.3B active per token), served on ZeroGPU as an FP8 build for efficient inference. It thinks through a problem…
  </Card>

  <Card href="/api-reference/models/qwen3-6-35b-a3b">
    <div>
      <img alt="qwen3.6-35b-a3b" />

      <div>qwen3.6-35b-a3b</div>
    </div>

    <div><span>35B params</span><span>\$0.20 / 1M input</span><span>\$1.50 / 1M output</span><span>\$0.20 / 1M cached input</span></div>

    Alibaba's Qwen3.6-35B-A3B is a fast, open-weight multimodal model for applications that need to understand text, images, and video, served on ZeroGPU as an FP8 build. It suits visual question answering, document and screenshot analysis…
  </Card>

  <Card href="/api-reference/models/gpt-oss-120b">
    <div>
      <img alt="gpt-oss-120b" />

      <div>gpt-oss-120b</div>
    </div>

    <div><span>131,072 context window</span><span>\$0.15 / 1M input</span><span>\$0.60 / 1M output</span><span>\$0.03 / 1M cached input</span></div>

    OpenAI's gpt-oss-120b is an open-weight Mixture-of-Experts model with 117B total parameters (5.1B active per token), served on ZeroGPU for general text generation. It reasons through a problem…
  </Card>

  <Card href="/api-reference/models/llama-3-1-8b-instruct-fast">
    <div>
      <img alt="llama-3.1-8b-instruct-fast" />

      <div>llama-3.1-8b-instruct-fast</div>
    </div>

    <div><span>131,072 max tokens</span><span>\$0.15 / 1M input</span><span>\$0.28 / 1M output</span><span>\$0.025 / 1M cached input</span></div>

    Meta's Llama 3.1 Instruct, tuned for fast, low-cost summarization at scale on the ZeroGPU edge network. Its 128K-token context window takes in entire documents, long transcripts, and full email or…
  </Card>

  <Card href="/api-reference/models/llama-guard-4-12b">
    <div>
      <img alt="llama-guard-4-12b" />

      <div>llama-guard-4-12b</div>
    </div>

    <div><span>Text Generation</span><span>Safety classification</span><span>Text Moderation</span><span>Brand Safety</span><span>12B params</span><span>163,840 context window</span><span>\$0.18 / 1M input</span><span>\$0.18 / 1M output</span></div>

    Meta's Llama Guard 4 12B is a multimodal safety classification model for moderating text, images, and mixed text-image inputs. It evaluates both incoming prompts…
  </Card>
</CardGroup>


# Production patterns
Source: https://docs.zerogpu.ai/docs/production-patterns

What to get right when you move ZeroGPU from a first call into production.

Keep API keys in environment variables, never in source or client-side code. Handle the common status codes explicitly: 401 (bad API key), 403 (bad project ID), and 429 (rate limit). Set sensible timeouts and retries with backoff on 429 and 5xx. For large, non-real-time jobs, use the Batch and Files API instead of the synchronous endpoint - streaming is not supported in batch mode. Already on the OpenAI SDK? The drop-in client shown in the [Introduction](/) is the recommended way to call ZeroGPU from application code.

## Keep secrets out of source

Read your `x-api-key` and `x-project-id` from the environment (or a secrets manager) and inject them at deploy time. Never commit them, never ship them in a browser bundle or mobile app - a key embedded in client-side code is a public key.

<Warning>
  ZeroGPU calls authenticate from your backend. If you need to call from a browser or mobile client, proxy the request through a server you control so the key stays server-side.
</Warning>

```bash theme={null}
# .env (git-ignored) - load with your process manager or dotenv
export ZEROGPU_API_KEY="zgpu-..."
export ZEROGPU_PROJECT_ID="..."
```

## Handle status codes explicitly

Branch on the status code. Authentication and authorization errors are permanent - retrying them just burns time and quota. Rate limits and server errors are transient - those are the ones to retry.

| Status | Meaning | What to do |
| - | - | - |
| `200` | Success | Parse and use the response. |
| `400` | Bad request | Fix the request body. Do **not** retry. |
| `401` | Bad API key | Check `x-api-key`. Do **not** retry. |
| `403` | Bad project ID | Check `x-project-id` and permissions. Do **not** retry. |
| `420` | Input over token limit | Shorten the input. Do **not** retry unchanged. |
| `429` | Rate limit reached | Wait at least `Retry-After` seconds, then retry. See [Rate limits](#rate-limits). |
| `5xx` | Server error | Retry with exponential backoff. |

<Note>
  Treat `408` (request timeout) and `409` (conflict) the same as `5xx` for retry purposes. Network errors and client-side timeouts are retriable too.
</Note>

## Rate limits

Your organization has limits for each model, shared by all of its API keys:

* **Requests per minute**
* **Tokens per minute** - counted when a request is accepted: the estimated input tokens plus `max_tokens` (`max_completion_tokens`, or `max_output_tokens` on the Responses API) for each of `n` choices. Tokens a response doesn't use are not given back, so set `max_tokens` close to the size of the response you expect.
* **Concurrent requests** - requests still in progress, including open streams.

Counts reset at the start of each minute. Audio endpoints have request and concurrency limits, but no token limit.

Responses carry your limits in headers. A header is left out when that limit doesn't apply.

| Header | Example | Meaning |
| - | - | - |
| `x-ratelimit-limit-requests` | `300` | Requests allowed per minute |
| `x-ratelimit-remaining-requests` | `295` | Requests left this minute |
| `x-ratelimit-reset-requests` | `37s` | Time until the request count resets |
| `x-ratelimit-limit-tokens` | `500000` | Tokens allowed per minute |
| `x-ratelimit-remaining-tokens` | `499000` | Tokens left this minute |
| `x-ratelimit-reset-tokens` | `37s` | Time until the token count resets |
| `x-ratelimit-limit-concurrent-requests` | `16` | Requests allowed in progress at once |
| `x-ratelimit-remaining-concurrent-requests` | `15` | Requests you can start right now |

A request over a limit is rejected with `429` before it runs, and `error.code` says which limit it hit:

```json theme={null}
{
  "error": {
    "message": "Rate limit reached for deepseek-v4.1-flash in organization 3f2c... on requests per min (RPM): Limit 300, Used 300, Requested 1. Please try again in 37s.",
    "type": "rate_limit_error",
    "param": null,
    "code": "requests_per_minute_exceeded"
  }
}
```

| `error.code` | What happened | What to do |
| - | - | - |
| `requests_per_minute_exceeded` | Too many requests this minute | Wait for `Retry-After`, then retry. |
| `tokens_per_minute_exceeded` | Too many tokens this minute | Wait for `Retry-After`, then retry. If there is no `Retry-After`, this request alone is over your limit: reduce the input or `max_tokens`. |
| `concurrency_limit_exceeded` | Too many requests in progress | Retry after `Retry-After`, or when one of your requests finishes. |

<Note>
  Rejected requests still count toward your requests per minute, so retrying in a tight loop keeps you limited. Batch jobs use the same limits as your synchronous calls. To raise a limit, contact support.
</Note>

## Set timeouts and retries

Three rules cover almost every case:

1. **Set a per-request timeout** so a stalled connection can't hang your worker.
2. **Retry only the transient codes** (`408`, `429`, `5xx`) and network failures - never `401`, `403`, or `400`.
3. **Back off exponentially with jitter**, cap the delay, cap the attempts, and honor the `Retry-After` header on `429`.

### Using the OpenAI SDK (recommended)

If you call ZeroGPU through the [drop-in OpenAI client](/), timeouts and retries are built in - set `timeout` and `max_retries` once on the client. The SDK retries `408`, `409`, `429`, and `5xx` with exponential backoff and respects `Retry-After` automatically.

<CodeGroup>
  ```python Python wrap theme={null}
  import os
  from openai import OpenAI

  client = OpenAI(
      base_url="https://api.zerogpu.ai/v1",
      api_key="unused",  # ZeroGPU authenticates via the headers below
      default_headers={
          "x-api-key": os.environ["ZEROGPU_API_KEY"],
          "x-project-id": os.environ["ZEROGPU_PROJECT_ID"],
      },
      timeout=30.0,    # seconds, per request
      max_retries=5,   # exponential backoff on 408 / 409 / 429 / 5xx
  )

  resp = client.responses.create(
      model="llama-3.1-8b-instruct-fast",
      input="Your input text here...",
  )
  print(resp.output)
  ```

  ```javascript JavaScript wrap theme={null}
  import OpenAI from 'openai';

  const client = new OpenAI({
    baseURL: 'https://api.zerogpu.ai/v1',
    apiKey: 'unused', // ZeroGPU authenticates via the headers below
    defaultHeaders: {
      'x-api-key': process.env.ZEROGPU_API_KEY,
      'x-project-id': process.env.ZEROGPU_PROJECT_ID,
    },
    timeout: 30_000, // ms, per request
    maxRetries: 5,   // exponential backoff on 408 / 409 / 429 / 5xx
  });

  const resp = await client.responses.create({
    model: 'llama-3.1-8b-instruct-fast',
    input: 'Your input text here...',
  });
  console.log(resp.output);
  ```
</CodeGroup>

You can override either value per request, for example a longer timeout on a heavy call: `client.responses.create(..., timeout=60.0)`.

### Rolling your own

When you call the HTTP API directly, implement the loop yourself: a per-request timeout, a retriable-status check, and exponential backoff with jitter that honors `Retry-After`.

<CodeGroup>
  ```bash cURL wrap theme={null}
  # curl's built-in --retry handles 408/429/500/502/503/504 with exponential
  # backoff and honors Retry-After. It does NOT retry 401/403/400 - exactly right.
  curl --retry 5 --retry-delay 1 --max-time 30 \
    https://api.zerogpu.ai/v1/responses \
    -H "content-type: application/json" \
    -H "x-api-key: $ZEROGPU_API_KEY" \
    -H "x-project-id: $ZEROGPU_PROJECT_ID" \
    -d '{
      "model": "llama-3.1-8b-instruct-fast",
      "input": "Your input text here..."
    }'
  ```

  ```python Python wrap theme={null}
  import os
  import random
  import time
  import requests

  URL = "https://api.zerogpu.ai/v1/responses"
  RETRIABLE = {408, 429, 500, 502, 503, 504}
  HEADERS = {
      "content-type": "application/json",
      "x-api-key": os.environ["ZEROGPU_API_KEY"],
      "x-project-id": os.environ["ZEROGPU_PROJECT_ID"],
  }


  def create_response(payload, max_retries=5, timeout=30):
      for attempt in range(max_retries + 1):
          retry_after = None
          try:
              resp = requests.post(URL, json=payload, headers=HEADERS, timeout=timeout)
              if resp.status_code not in RETRIABLE:
                  resp.raise_for_status()  # raises on 401 / 403 / 400 - no retry
                  return resp.json()
              if attempt == max_retries:
                  resp.raise_for_status()
              retry_after = resp.headers.get("retry-after")
          except requests.exceptions.RequestException:
              if attempt == max_retries:
                  raise

          # exponential backoff with jitter; honor Retry-After on 429
          delay = float(retry_after) if retry_after else min(2 ** attempt, 30) + random.random()
          time.sleep(delay)
  ```

  ```javascript JavaScript wrap theme={null}
  const URL = 'https://api.zerogpu.ai/v1/responses';
  const RETRIABLE = new Set([408, 429, 500, 502, 503, 504]);
  const sleep = (ms) => new Promise((r) => setTimeout(r, ms));

  async function createResponse(payload, { maxRetries = 5, timeoutMs = 30_000 } = {}) {
    const headers = {
      'content-type': 'application/json',
      'x-api-key': process.env.ZEROGPU_API_KEY,
      'x-project-id': process.env.ZEROGPU_PROJECT_ID,
    };

    for (let attempt = 0; attempt <= maxRetries; attempt++) {
      const controller = new AbortController();
      const timer = setTimeout(() => controller.abort(), timeoutMs);
      try {
        const res = await fetch(URL, {
          method: 'POST',
          headers,
          body: JSON.stringify(payload),
          signal: controller.signal,
        });

        if (!RETRIABLE.has(res.status)) {
          if (!res.ok) throw new Error(`ZeroGPU ${res.status}: ${await res.text()}`); // 401 / 403 / 400
          return res.json();
        }
        if (attempt === maxRetries) throw new Error(`ZeroGPU ${res.status} after ${maxRetries} retries`);

        // exponential backoff with jitter; honor Retry-After on 429
        const retryAfter = res.headers.get('retry-after');
        const delay = retryAfter
          ? Number(retryAfter) * 1000
          : Math.min(2 ** attempt * 1000, 30_000) + Math.random() * 1000;
        await sleep(delay);
      } catch (err) {
        if (attempt === maxRetries) throw err;
        await sleep(Math.min(2 ** attempt * 1000, 30_000) + Math.random() * 1000);
      } finally {
        clearTimeout(timer);
      }
    }
  }
  ```

  ```rust Rust wrap theme={null}
  use reqwest::Client;
  use serde_json::Value;
  use std::time::Duration;
  use tokio::time::sleep;

  const RETRIABLE: [u16; 6] = [408, 429, 500, 502, 503, 504];

  async fn create_response(payload: Value, max_retries: u32) -> Result<Value, Box<dyn std::error::Error>> {
      let client = Client::builder().timeout(Duration::from_secs(30)).build()?;
      let api_key = std::env::var("ZEROGPU_API_KEY")?;
      let project_id = std::env::var("ZEROGPU_PROJECT_ID")?;

      for attempt in 0..=max_retries {
          let mut retry_after: Option<u64> = None;

          match client
              .post("https://api.zerogpu.ai/v1/responses")
              .header("content-type", "application/json")
              .header("x-api-key", &api_key)
              .header("x-project-id", &project_id)
              .json(&payload)
              .send()
              .await
          {
              Ok(resp) => {
                  let status = resp.status();
                  if !RETRIABLE.contains(&status.as_u16()) {
                      if !status.is_success() {
                          // 401 / 403 / 400 - permanent, do not retry
                          return Err(format!("zerogpu {}: {}", status, resp.text().await?).into());
                      }
                      return Ok(resp.json().await?);
                  }
                  if attempt == max_retries {
                      return Err(format!("zerogpu {} after {} retries", status, max_retries).into());
                  }
                  retry_after = resp
                      .headers()
                      .get("retry-after")
                      .and_then(|v| v.to_str().ok())
                      .and_then(|v| v.parse().ok());
              }
              Err(err) => {
                  if attempt == max_retries {
                      return Err(err.into());
                  }
              }
          }

          // exponential backoff; honor Retry-After on 429
          let backoff = retry_after.unwrap_or_else(|| 2u64.pow(attempt).min(30));
          sleep(Duration::from_secs(backoff)).await;
      }
      unreachable!()
  }
  ```

  ```go Go wrap theme={null}
  package main

  import (
  	"bytes"
  	"encoding/json"
  	"fmt"
  	"io"
  	"math"
  	"math/rand"
  	"net/http"
  	"os"
  	"strconv"
  	"time"
  )

  var retriable = map[int]bool{408: true, 429: true, 500: true, 502: true, 503: true, 504: true}

  func createResponse(payload map[string]any, maxRetries int) ([]byte, error) {
  	body, _ := json.Marshal(payload)
  	client := &http.Client{Timeout: 30 * time.Second}

  	for attempt := 0; ; attempt++ {
  		req, _ := http.NewRequest("POST", "https://api.zerogpu.ai/v1/responses", bytes.NewReader(body))
  		req.Header.Set("content-type", "application/json")
  		req.Header.Set("x-api-key", os.Getenv("ZEROGPU_API_KEY"))
  		req.Header.Set("x-project-id", os.Getenv("ZEROGPU_PROJECT_ID"))

  		res, err := client.Do(req)
  		if err == nil && !retriable[res.StatusCode] {
  			defer res.Body.Close()
  			data, _ := io.ReadAll(res.Body)
  			if res.StatusCode >= 400 { // 401 / 403 / 400 - permanent
  				return nil, fmt.Errorf("zerogpu %d: %s", res.StatusCode, data)
  			}
  			return data, nil
  		}
  		if attempt >= maxRetries {
  			if err != nil {
  				return nil, err
  			}
  			return nil, fmt.Errorf("zerogpu %d after %d retries", res.StatusCode, maxRetries)
  		}

  		// exponential backoff with jitter; honor Retry-After on 429
  		delay := time.Duration(math.Min(math.Pow(2, float64(attempt)), 30))*time.Second +
  			time.Duration(rand.Intn(1000))*time.Millisecond
  		if err == nil {
  			if ra := res.Header.Get("Retry-After"); ra != "" {
  				if secs, e := strconv.Atoi(ra); e == nil {
  					delay = time.Duration(secs) * time.Second
  				}
  			}
  			res.Body.Close()
  		}
  		time.Sleep(delay)
  	}
  }
  ```

  ```ruby Ruby wrap theme={null}
  require 'net/http'
  require 'uri'
  require 'json'

  ENDPOINT = URI('https://api.zerogpu.ai/v1/responses')
  RETRIABLE = [408, 429, 500, 502, 503, 504].freeze

  def create_response(payload, max_retries: 5, timeout: 30)
    http = Net::HTTP.new(ENDPOINT.host, ENDPOINT.port)
    http.use_ssl = true
    http.open_timeout = timeout
    http.read_timeout = timeout

    request = Net::HTTP::Post.new(ENDPOINT.request_uri)
    request['content-type'] = 'application/json'
    request['x-api-key'] = ENV.fetch('ZEROGPU_API_KEY')
    request['x-project-id'] = ENV.fetch('ZEROGPU_PROJECT_ID')
    request.body = payload.to_json

    (0..max_retries).each do |attempt|
      delay =
        begin
          response = http.request(request)
          status = response.code.to_i
          unless RETRIABLE.include?(status)
            raise "ZeroGPU #{status}: #{response.body}" if status >= 400 # 401 / 403 / 400

            return JSON.parse(response.body)
          end
          raise "ZeroGPU #{status} after #{max_retries} retries" if attempt == max_retries

          # exponential backoff with jitter; honor Retry-After on 429
          response['retry-after']&.to_f || [2**attempt, 30].min + rand
        rescue Net::OpenTimeout, Net::ReadTimeout
          raise if attempt == max_retries

          [2**attempt, 30].min + rand
        end

      sleep(delay)
    end
  end
  ```
</CodeGroup>

## Use the Batch API for large jobs

For large, non-real-time workloads, the [Batch and Files API](/docs/batch) is the right tool instead of looping over the synchronous endpoint. It processes up to 50,000 requests within a 24-hour window at a discounted rate and sidesteps per-request rate limits entirely - so you don't need a retry loop at all.

| You need… | Use |
| - | - |
| A single immediate response | The synchronous endpoint (`POST /v1/responses`) with the retry loop above |
| Thousands of completions, can wait minutes-to-hours | The [Batch API](/docs/batch) |
| To avoid per-second rate limits during a backfill | The [Batch API](/docs/batch) |
| Streaming responses | The synchronous endpoint - streaming is **not** supported in batch mode |

## Next steps

<Columns>
  <Card title="Batch & Files API" icon="layer-group" href="/docs/batch" />

  <Card title="Quickstart" icon="key" href="/docs/quickstart" />

  <Card title="API Reference" icon="terminal" href="/api-reference/responses" />
</Columns>


# Quickstart
Source: https://docs.zerogpu.ai/docs/quickstart

First API call in minutes.

## 1. Grab your credentials

Log into the [ZeroGPU dashboard](https://platform.zerogpu.ai/dashboard) and copy these three values:

| Credential | Where to find it |
| - | - |
| **API Key** | Dashboard → API Keys |
| **Project ID** (optional) | Dashboard → Project Settings - scopes requests to a project |
| **Model** | Dashboard → Model selector, or pick one from the [Model Catalog](/docs/model-catalog) |

## 2. Send a request

<CodeGroup>
  ```bash cURL wrap theme={null}
  curl https://api.zerogpu.ai/v1/responses \
    -H "content-type: application/json" \
    -H "x-api-key: $ZEROGPU_API_KEY" \
    -H "x-project-id: $ZEROGPU_PROJECT_ID" \
    -d '{
      "model": "YOUR_MODEL",
      "input": "Your input text here..."
    }'
  ```

  ```python Python wrap theme={null}
  # pip install zerogpu-api
  import os
  from zerogpu import ZerogpuApi

  client = ZerogpuApi(
      api_key=os.environ["ZEROGPU_API_KEY"],
      project_id=os.environ["ZEROGPU_PROJECT_ID"],
  )

  response = client.responses.create_response(
      model="YOUR_MODEL",
      input="Your input text here...",
  )
  print(response)
  ```

  ```javascript JavaScript wrap theme={null}
  // npm install zerogpu-api
  import { ZerogpuApiClient } from 'zerogpu-api';

  const client = new ZerogpuApiClient({
    apiKey: process.env.ZEROGPU_API_KEY,
    projectId: process.env.ZEROGPU_PROJECT_ID,
  });

  const response = await client.responses.createResponse({
    model: 'YOUR_MODEL',
    input: 'Your input text here...',
  });

  console.log(response.output);
  ```

  ```rust Rust wrap theme={null}
  use reqwest::Client;
  use serde_json::json;

  #[tokio::main]
  async fn main() -> Result<(), Box<dyn std::error::Error>> {
      let client = Client::new();

      let payload = json!({
          "model": "YOUR_MODEL",
          "input": "Your input text here..."
      });

      let response = client
          .post("https://api.zerogpu.ai/v1/responses")
          .header("content-type", "application/json")
          .header("x-api-key", "YOUR_API_KEY")
          .header("x-project-id", "YOUR_PROJECT_ID")
          .json(&payload)
          .send()
          .await?;

      let body = response.text().await?;
      println!("{}", body);
      Ok(())
  }
  ```

  ```go Go wrap theme={null}
  package main

  import (
    "bytes"
    "encoding/json"
    "fmt"
    "io"
    "net/http"
  )

  func main() {
    url := "https://api.zerogpu.ai/v1/responses"

    payload := map[string]any{
      "model": "YOUR_MODEL",
      "input": "Your input text here...",
    }

    payloadBytes, err := json.Marshal(payload)
    if err != nil {
      panic(err)
    }

    req, err := http.NewRequest("POST", url, bytes.NewBuffer(payloadBytes))
    if err != nil {
      panic(err)
    }

    req.Header.Set("content-type", "application/json")
    req.Header.Set("x-api-key", "YOUR_API_KEY")
    req.Header.Set("x-project-id", "YOUR_PROJECT_ID")

    client := &http.Client{}
    res, err := client.Do(req)
    if err != nil {
      panic(err)
    }
    defer res.Body.Close()

    body, _ := io.ReadAll(res.Body)
    fmt.Println(string(body))
  }
  ```

  ```ruby Ruby wrap theme={null}
  require 'net/http'
  require 'uri'
  require 'json'

  uri = URI.parse('https://api.zerogpu.ai/v1/responses')
  http = Net::HTTP.new(uri.host, uri.port)
  http.use_ssl = (uri.scheme == 'https')

  request = Net::HTTP::Post.new(uri.request_uri)
  request['content-type'] = 'application/json'
  request['x-api-key'] = 'YOUR_API_KEY'
  request['x-project-id'] = 'YOUR_PROJECT_ID'

  request.body = {
    model: 'YOUR_MODEL',
    input: 'Your input text here...'
  }.to_json

  response = http.request(request)
  puts response.body
  ```
</CodeGroup>

<Warning>
  Store your API key in environment variables. Never commit it to version control or expose it in client-side code.
</Warning>

## 3. See a complete example

A full request - here the `llama-3.1-8b-instruct-fast` model summarizing a passage:

<CodeGroup>
  ```bash cURL wrap theme={null}
  curl https://api.zerogpu.ai/v1/responses \
    -H "content-type: application/json" \
    -H "x-api-key: $ZEROGPU_API_KEY" \
    -H "x-project-id: $ZEROGPU_PROJECT_ID" \
    -d '{
      "model": "llama-3.1-8b-instruct-fast",
      "input": "NASA announced that its Artemis III mission is now scheduled for late 2026, marking the first time astronauts will land on the lunar surface since Apollo 17 in 1972. The mission will send a crew of four to the Moon aboard the Orion spacecraft, with two astronauts descending to the south pole using SpaceX Starship as a lunar lander. Scientists are particularly excited about exploring permanently shadowed craters that may contain water ice, which could be critical for sustaining long-term human presence on the Moon."
    }'
  ```

  ```python Python wrap theme={null}
  # pip install zerogpu-api
  import os
  from zerogpu import ZerogpuApi

  client = ZerogpuApi(
      api_key=os.environ["ZEROGPU_API_KEY"],
      project_id=os.environ["ZEROGPU_PROJECT_ID"],
  )

  response = client.responses.create_response(
      model="llama-3.1-8b-instruct-fast",
      input="NASA announced that its Artemis III mission is now scheduled for late 2026, marking the first time astronauts will land on the lunar surface since Apollo 17 in 1972. The mission will send a crew of four to the Moon aboard the Orion spacecraft, with two astronauts descending to the south pole using SpaceX Starship as a lunar lander. Scientists are particularly excited about exploring permanently shadowed craters that may contain water ice, which could be critical for sustaining long-term human presence on the Moon.",
  )
  print(response)
  ```

  ```javascript JavaScript wrap theme={null}
  // npm install zerogpu-api
  import { ZerogpuApiClient } from 'zerogpu-api';

  const client = new ZerogpuApiClient({
    apiKey: process.env.ZEROGPU_API_KEY,
    projectId: process.env.ZEROGPU_PROJECT_ID,
  });

  const response = await client.responses.createResponse({
    model: 'llama-3.1-8b-instruct-fast',
    input: 'NASA announced that its Artemis III mission is now scheduled for late 2026, marking the first time astronauts will land on the lunar surface since Apollo 17 in 1972. The mission will send a crew of four to the Moon aboard the Orion spacecraft, with two astronauts descending to the south pole using SpaceX Starship as a lunar lander. Scientists are particularly excited about exploring permanently shadowed craters that may contain water ice, which could be critical for sustaining long-term human presence on the Moon.',
  });

  console.log(response.output);
  ```

  ```rust Rust wrap theme={null}
  use reqwest::Client;
  use serde_json::json;

  #[tokio::main]
  async fn main() -> Result<(), Box<dyn std::error::Error>> {
      let client = Client::new();

      let payload = json!({
          "model": "llama-3.1-8b-instruct-fast",
          "input": "NASA announced that its Artemis III mission is now scheduled for late 2026, marking the first time astronauts will land on the lunar surface since Apollo 17 in 1972. The mission will send a crew of four to the Moon aboard the Orion spacecraft, with two astronauts descending to the south pole using SpaceX Starship as a lunar lander. Scientists are particularly excited about exploring permanently shadowed craters that may contain water ice, which could be critical for sustaining long-term human presence on the Moon."
      });

      let response = client
          .post("https://api.zerogpu.ai/v1/responses")
          .header("content-type", "application/json")
          .header("x-api-key", "YOUR_API_KEY")
          .header("x-project-id", "YOUR_PROJECT_ID")
          .json(&payload)
          .send()
          .await?;

      let body = response.text().await?;
      println!("{}", body);
      Ok(())
  }
  ```

  ```go Go wrap theme={null}
  package main

  import (
    "bytes"
    "encoding/json"
    "fmt"
    "io"
    "net/http"
  )

  func main() {
    url := "https://api.zerogpu.ai/v1/responses"

    payload := map[string]any{
      "model": "llama-3.1-8b-instruct-fast",
      "input": "NASA announced that its Artemis III mission is now scheduled for late 2026, marking the first time astronauts will land on the lunar surface since Apollo 17 in 1972. The mission will send a crew of four to the Moon aboard the Orion spacecraft, with two astronauts descending to the south pole using SpaceX Starship as a lunar lander. Scientists are particularly excited about exploring permanently shadowed craters that may contain water ice, which could be critical for sustaining long-term human presence on the Moon.",
    }

    payloadBytes, err := json.Marshal(payload)
    if err != nil {
      panic(err)
    }

    req, err := http.NewRequest("POST", url, bytes.NewBuffer(payloadBytes))
    if err != nil {
      panic(err)
    }

    req.Header.Set("content-type", "application/json")
    req.Header.Set("x-api-key", "YOUR_API_KEY")
    req.Header.Set("x-project-id", "YOUR_PROJECT_ID")

    client := &http.Client{}
    res, err := client.Do(req)
    if err != nil {
      panic(err)
    }
    defer res.Body.Close()

    body, _ := io.ReadAll(res.Body)
    fmt.Println(string(body))
  }
  ```

  ```ruby Ruby wrap theme={null}
  require 'net/http'
  require 'uri'
  require 'json'

  uri = URI.parse('https://api.zerogpu.ai/v1/responses')
  http = Net::HTTP.new(uri.host, uri.port)
  http.use_ssl = (uri.scheme == 'https')

  request = Net::HTTP::Post.new(uri.request_uri)
  request['content-type'] = 'application/json'
  request['x-api-key'] = 'YOUR_API_KEY'
  request['x-project-id'] = 'YOUR_PROJECT_ID'

  request.body = {
    model: 'llama-3.1-8b-instruct-fast',
    input: 'NASA announced that its Artemis III mission is now scheduled for late 2026, marking the first time astronauts will land on the lunar surface since Apollo 17 in 1972. The mission will send a crew of four to the Moon aboard the Orion spacecraft, with two astronauts descending to the south pole using SpaceX Starship as a lunar lander. Scientists are particularly excited about exploring permanently shadowed craters that may contain water ice, which could be critical for sustaining long-term human presence on the Moon.'
  }.to_json

  response = http.request(request)
  puts response.body
  ```
</CodeGroup>

🎉 The model returns the summary:

```text wrap theme={null}
NASA's Artemis III mission is now scheduled for late 2026, marking the first lunar landing since Apollo 17 in 1972, with a crew of four astronauts aboard the Orion spacecraft, including two descending to the Moon's south pole using SpaceX Starship as a lunar lander to explore permanently shadowed craters potentially containing water ice crucial for sustaining a long-term human presence on the Moon.
```

## 4. Verify it worked

Check the dashboard:

* **Logs**: your request appears with model, status, and latency.
* **Usage**: token counts update in real time.

If you get a `401`, your API key is wrong. A `403` means a bad project ID.

## Next steps

<Columns>
  <Card title="API Reference" icon="terminal" href="/api-reference/responses" />

  <Card title="How ZeroGPU works" icon="diagram-project" href="/docs/how-zerogpu-works" />
</Columns>


# Text Classification
Source: https://docs.zerogpu.ai/docs/text-classification

Edge models that map text to labels, IAB taxonomy, and audience signals.

Text classification models map a passage to predefined labels or categories. ZeroGPU offers IAB taxonomy classifiers tuned for ad tech, a general zero-shot classifier where you supply the candidate labels, and the schema-based GLiNER2.5 models, which classify, extract entities, fill a JSON schema, and make dynamic-label decisions such as intent and routing from the same request. Each request is shown for both the [Responses API](/api-reference/responses) and the OpenAI-compatible [Chat Completions API](/api-reference/chat-completions).

| Model | Input /1M | Output /1M | Max tokens |
| - | -: | -: | -: |
| <a href="/api-reference/models/zlm-v1-iab-classify-edge"><img alt="zlm-v1-iab-classify-edge" /> <code>zlm-v1-iab-classify-edge</code></a> | \$0.02 | \$0.05 | 400 |
| <a href="/api-reference/models/zlm-v2-iab-classify-edge-enriched"><img alt="zlm-v2-iab-classify-edge-enriched" /> <code>zlm-v2-iab-classify-edge-enriched</code></a> | \$0.05 | \$0.15 | 800 |
| <a href="/api-reference/models/zlm-v1-iab-domain-classifier"><img alt="zlm-v1-iab-domain-classifier" /> <code>zlm-v1-iab-domain-classifier</code></a> | \$0.02 | \$0.05 | 100 |
| <a href="/api-reference/models/deberta-v3-small"><img alt="deberta-v3-small" /> <code>deberta-v3-small</code></a> | \$0.02 | \$0.05 | 400 |
| <a href="/api-reference/models/zlm-v1-signal-extract"><img alt="zlm-v1-signal-extract" /> <code>zlm-v1-signal-extract</code></a> | \$0.04 | \$0.10 | 400 |
| <a href="/api-reference/models/gliner2-5-base-v1"><img alt="gliner2.5-base-v1" /> <code>gliner2.5-base-v1</code></a> | \$0.02 | \$0.05 | — |
| <a href="/api-reference/models/gliner2-5-multi-v1"><img alt="gliner2.5-multi-v1" /> <code>gliner2.5-multi-v1</code></a> | \$0.02 | \$0.05 | — |
| <a href="/api-reference/models/gliner2-5-decide-v1"><img alt="gliner2.5-decide-v1" /> <code>gliner2.5-decide-v1</code></a> | \$0.004 | Not billed | — |

## zlm-v1-iab-classify-edge

> ZeroGPU's IAB classifier maps any text to the industry-standard IAB Content Taxonomy in a single, fast inference call. Each call returns categories across both the 1.0 and 2.2 taxonomies plus matched audience segments, every result scored for confidence. With support for 50+ languages, it classifies multilingual content without a translation step. At 90M parameters on ONNX, it's built for the high-volume, sub-millisecond classification that ad tech and content platforms demand, right at the edge. When you need clean category and audience signals on every request, this is the model — and when you need the full profile, reach for the enriched variant.

**References:** [Model docs](https://docs.zerogpu.ai/) • [Terms](https://zerogpu.ai/terms) • [Privacy](https://zerogpu.ai/privacy-policy)

<CodeGroup>
  ```bash Responses API theme={null}
  curl https://api.zerogpu.ai/v1/responses \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "input": "Technology has quietly reshaped the rhythm of everyday life, weaving itself into routines so seamlessly that it often goes unnoticed. From the moment a smartphone alarm wakes someone in the morning to the final glance at a glowing screen before sleep, digital systems guide communication, navigation, work, and entertainment. This transformation did not happen overnight. It emerged through decades of incremental innovation, each new tool building upon the last, until the modern world became deeply interconnected.One of the most significant changes has been the speed at which information travels.",
    "model": "zlm-v1-iab-classify-edge"
  }'
  ```

  ```bash Chat Completions theme={null}
  curl https://api.zerogpu.ai/v1/chat/completions \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "model": "zlm-v1-iab-classify-edge",
    "messages": [
      {
        "role": "user",
        "content": "What are the basics of exercise and physical fitness?Exercise is anything that gets your body moving. Regular exercise is one of the best things you can do for your health. It has many benefits, including improving your overall health and fitness, and reducing your risk for many chronic (long-term) diseases.Every physical fitness routine is built on a few simple ideas. These include:Make exercise a habit, as your body adapts to the type of activity you do most often. Regular practice will help you improve.Build up your activity level slowly to help you continue to get stronger, faster, or more flexible without pushing too hard all at once.Challenge yourself by lifting slightly heavier weights, adding a few more minutes to your walk, or increasing your pace."
      }
    ]
  }'
  ```
</CodeGroup>

Returns IAB `audience` categories plus `content` matches across taxonomy versions, each with a confidence score.

```json Response theme={null}
{
  "audience": [
    { "name": "Technology & Computing", "score": 0.7686 },
    { "name": "Consumer Electronics", "score": 0.7206 },
    { "name": "65-69", "score": 0.5950 }
  ],
  "content": {
    "iab_1_0": [
      { "name": "Cell Phones", "score": 0.7597 },
      { "name": "Technology & Computing", "score": 0.7387 }
    ],
    "iab_2_2": [
      { "name": "Smartphones", "score": 0.7597 },
      { "name": "Wearable Technology", "score": 0.7387 },
      { "name": "Technology & Computing", "score": 0.7109 }
    ]
  }
}
```

## zlm-v2-iab-classify-edge-enriched

> The enriched variant of ZeroGPU's IAB classifier turns a single inference call into a full content-intelligence profile not just a label, but everything a contextual pipeline needs to act on. Each call returns IAB categories across both the 1.0 and 2.2 taxonomies (down to tier-3), audience and interest segments, topics, keywords, and user-intent classification — all with confidence scores with edge-native, sub-millisecond speeds. With support for 50+ languages, it delivers the same full profile on multilingual content without a translation step. Built for contextual ad targeting, brand-safety scoring, publisher content categorization, and signal pipelines that need full metadata on every request.

**References:** [Model docs](https://docs.zerogpu.ai/) • [Terms](https://zerogpu.ai/terms) • [Privacy](https://zerogpu.ai/privacy-policy)

<CodeGroup>
  ```bash Responses API theme={null}
  curl https://api.zerogpu.ai/v1/responses \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "input": "Technology has quietly reshaped the rhythm of everyday life, weaving itself into routines so seamlessly that it often goes unnoticed. From the moment a smartphone alarm wakes someone in the morning to the final glance at a glowing screen before sleep, digital systems guide communication, navigation, work, and entertainment. This transformation did not happen overnight. It emerged through decades of incremental innovation, each new tool building upon the last, until the modern world became deeply interconnected.One of the most significant changes has been the speed at which information travels.",
    "model": "zlm-v2-iab-classify-edge-enriched"
  }'
  ```

  ```bash Chat Completions theme={null}
  curl https://api.zerogpu.ai/v1/chat/completions \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "model": "zlm-v2-iab-classify-edge-enriched",
    "messages": [
      {
        "role": "user",
        "content": "What are the basics of exercise and physical fitness?Exercise is anything that gets your body moving. Regular exercise is one of the best things you can do for your health. It has many benefits, including improving your overall health and fitness, and reducing your risk for many chronic (long-term) diseases.Every physical fitness routine is built on a few simple ideas. These include:Make exercise a habit, as your body adapts to the type of activity you do most often. Regular practice will help you improve.Build up your activity level slowly to help you continue to get stronger, faster, or more flexible without pushing too hard all at once.Challenge yourself by lifting slightly heavier weights, adding a few more minutes to your walk, or increasing your pace."
      }
    ]
  }'
  ```
</CodeGroup>

In addition to scored `audience` and `content` categories, the enriched model returns taxonomy codes and parent IDs, plus `topics`, `keywords`, and an inferred `user_intent`.

```json Response theme={null}
{
  "audience": [
    {
      "id": 687,
      "parent_id": 206,
      "name": "Technology & Computing",
      "tier1_name": "Interest",
      "tier2_name": "Technology & Computing",
      "score": 0.7686
    },
    {
      "id": 703,
      "parent_id": 687,
      "name": "Consumer Electronics",
      "tier1_name": "Interest",
      "tier2_name": "Technology & Computing",
      "tier3_name": "Consumer Electronics",
      "score": 0.7206
    }
  ],
  "content": {
    "iab_1_0": [
      { "code": "IAB19-6", "name": "Cell Phones", "tier": 2, "parent_code": "IAB19", "score": 0.7597 },
      { "code": "IAB19", "name": "Technology & Computing", "tier": 1, "parent_code": null, "score": 0.7387 }
    ]
  },
  "topics": [
    { "name": "smartphones", "score": 0.7597 },
    { "name": "wearable technology", "score": 0.7387 },
    { "name": "technology & computing", "score": 0.7109 }
  ],
  "keywords": ["technology", "reshaped", "everyday", "routines", "quietly", "weaving", "rhythm", "itself", "life", "into"],
  "user_intent": {
    "name": "mastering technology quietly reshaped rhythm everyday life",
    "category": "informational",
    "score": 0.5
  }
}
```

## zlm-v1-iab-domain-classifier

> ZeroGPU's Domain IAB Classifier maps a raw domain name straight to the IAB Content Taxonomy in a single, fast inference call, returning content categories, topics, keywords, and user-intent signals. Where the page-level IAB classifiers take the full body of an article as input, this model takes only the domain (for example `nytimes.com`), reducing payload size by up to 10x while still surfacing the contextual signals that bidstream enrichment, contextual targeting, and domain-level intelligence pipelines run on. At 149M parameters on ONNX, it's built for the high-volume, sub-millisecond classification that ad tech and content platforms demand, right at the edge.

**References:** [Model docs](https://docs.zerogpu.ai/) • [Terms](https://zerogpu.ai/terms) • [Privacy](https://zerogpu.ai/privacy-policy)

<CodeGroup>
  ```bash Responses API theme={null}
  curl https://api.zerogpu.ai/v1/responses \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "input": "nytimes.com",
    "model": "zlm-v1-iab-domain-classifier"
  }'
  ```

  ```bash Chat Completions theme={null}
  curl https://api.zerogpu.ai/v1/chat/completions \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "model": "zlm-v1-iab-domain-classifier",
    "messages": [
      {
        "role": "user",
        "content": "nytimes.com"
      }
    ]
  }'
  ```
</CodeGroup>

Returns scored `content` categories across taxonomy versions plus `topics`, `keywords`, and an inferred `user_intent` for the domain.

```json Response theme={null}
{
  "content": {
    "iab_1_0": [
      { "name": "News", "score": 0.8124 },
      { "name": "Politics", "score": 0.6533 }
    ],
    "iab_2_2": [
      { "name": "News and Politics", "score": 0.8124 },
      { "name": "Business and Finance", "score": 0.5980 }
    ]
  },
  "topics": [
    { "name": "news", "score": 0.8124 },
    { "name": "journalism", "score": 0.6740 },
    { "name": "current events", "score": 0.6533 }
  ],
  "keywords": ["news", "politics", "world", "business", "opinion", "media", "journalism"],
  "user_intent": {
    "name": "reading news and current events",
    "category": "informational",
    "score": 0.5
  }
}
```

## deberta-v3-small

> Microsoft's DeBERTa-v3-small is a fast, lightweight zero-shot text classifier for high-volume routing, filtering, and tagging. Hand it any text alongside your own candidate labels and it returns a confidence score for each. No training data and no fixed taxonomy, just the categories you define at request time. Because it scores every label rather than forcing one choice, you get soft probabilities to threshold, rank, or route on however your pipeline needs. Purpose-built and CPU-optimized, it runs faster and cheaper than asking a general-purpose frontier model to classify.

**References:** [Model docs](https://github.com/microsoft/DeBERTa/tree/master/docs) • [Terms](https://github.com/microsoft/DeBERTa/blob/master/SECURITY.md) • [Privacy](https://github.com/microsoft/DeBERTa/blob/master/LICENSE)

<CodeGroup>
  ```bash Responses API theme={null}
  curl https://api.zerogpu.ai/v1/responses \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "model": "deberta-v3-small",
    "input": "Apple is expected to unveil its next-generation M5 chip at WWDC this June, promising a 40% boost in GPU performance and a new dedicated AI core for on-device machine learning tasks.",
    "instructions": "[sports, finance, politics, technology]"
  }'
  ```

  ```bash Chat Completions theme={null}
  curl https://api.zerogpu.ai/v1/chat/completions \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "model": "deberta-v3-small",
    "messages": [
      {
        "role": "system",
        "content": "[sports, finance, politics, technology]"
      },
      {
        "role": "user",
        "content": "Apple is expected to unveil its next-generation M5 chip at WWDC this June, promising a 40% boost in GPU performance and a new dedicated AI core for on-device machine learning tasks."
      }
    ]
  }'
  ```
</CodeGroup>

Returns each candidate label with a probability score.

```json Response theme={null}
{
  "technology": 0.971784,
  "politics": 0.011858,
  "sports": 0.01175,
  "finance": 0.004608
}
```

## zlm-v1-signal-extract

> ZeroGPU's signal extractor turns unstructured text into structured signals that downstream systems can act on. One inference call returns topics, keywords, intent, and other contextual attributes as structured output, so content enrichment, contextual intelligence, ad targeting, agent routing, recommendation systems, and analytics pipelines all read from the same pass. At 80M parameters it is built for high-volume workloads where a general-purpose LLM is more machinery than the job needs.

**References:** [Terms](https://zerogpu.ai/terms) • [Privacy](https://zerogpu.ai/privacy-policy)

Pass the signal categories you want in `categories`; `max_keywords` caps how many keywords come back.

<CodeGroup>
  ```bash Responses API theme={null}
  curl https://api.zerogpu.ai/v1/responses \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "model": "zlm-v1-signal-extract",
    "input": "How can I pay using my credit card?",
    "categories": [
      "signals"
    ]
  }'
  ```

  ```bash Chat Completions theme={null}
  curl https://api.zerogpu.ai/v1/chat/completions \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "model": "zlm-v1-signal-extract",
    "messages": [
      {
        "role": "user",
        "content": "How can I pay using my credit card?"
      }
    ],
    "categories": [
      "signals"
    ],
    "max_keywords": 6
  }'
  ```
</CodeGroup>

The signals arrive as a JSON string in the output text. Parsed:

```json Response theme={null}
{
  "keywords": [
    "credit card",
    "login",
    "official website",
    "account access"
  ],
  "user_intent": {
    "name": "pay using credit card",
    "category": "navigational",
    "score": 0.95
  }
}
```

## gliner2.5-base-v1

> Fastino's GLiNER2.5 Base is an English schema-based information extraction model built on DeBERTa-v3-base. A single model covers entity extraction, text classification, structured record extraction, relation extraction, and span-level attributes — you supply a label set or a schema at inference time instead of fine-tuning one model per task. It fits GTM workflows such as parsing inbound leads, extracting company and contact details, classifying sales conversations, identifying relationships between people and companies, structuring CRM notes, and enriching customer or account data.

**References:** [Model docs](https://huggingface.co/fastino/gliner2.5-base-v1) • [Terms](https://github.com/fastino-ai/GLiNER2/blob/main/LICENSE) • [Privacy](https://github.com/fastino-ai/GLiNER2/blob/main/LICENSE)

### Entity Extraction

Set `metadata.usecase` to `ner` and pass your own `labels` to extract named entities. `threshold` sets the minimum confidence for a span to be returned.

<CodeGroup>
  ```bash Responses API theme={null}
  curl https://api.zerogpu.ai/v1/responses \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "input": "The application is built with Python 3.11 and uses PostgreSQL 15 for storage. It runs on Kubernetes with Docker containers and communicates via gRPC.",
    "model": "gliner2.5-base-v1",
    "metadata": {
      "labels": [
        "programming language",
        "database",
        "technology",
        "protocol"
      ],
      "usecase": "ner",
      "threshold": 0.3
    }
  }'
  ```

  ```bash Chat Completions theme={null}
  curl https://api.zerogpu.ai/v1/chat/completions \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "model": "gliner2.5-base-v1",
    "messages": [
      {
        "role": "user",
        "content": "The application is built with Python 3.11 and uses PostgreSQL 15 for storage. It runs on Kubernetes with Docker containers and communicates via gRPC."
      }
    ],
    "metadata": {
      "labels": [
        "programming language",
        "database",
        "technology",
        "protocol"
      ],
      "usecase": "ner",
      "threshold": 0.3
    }
  }'
  ```
</CodeGroup>

### Structured Data Extraction

Set `metadata.usecase` to `json` and define a `schema` of `field::type::description` entries to pull structured records out of free text.

<CodeGroup>
  ```bash Responses API theme={null}
  curl https://api.zerogpu.ai/v1/responses \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "input": "Best regards, John Smith, Senior Software Engineer at Acme Corp. Phone: (555) 123-4567, Email: john.smith@acme.com, Office: 123 Main Street, Suite 400, San Francisco, CA 94105",
    "model": "gliner2.5-base-v1",
    "metadata": {
      "schema": {
        "contact": [
          "name::str::Full name",
          "title::str::Job title",
          "company::str::Company name",
          "phone::str::Phone number",
          "email::str::Email address",
          "address::str::Office address"
        ]
      },
      "usecase": "json"
    }
  }'
  ```

  ```bash Chat Completions theme={null}
  curl https://api.zerogpu.ai/v1/chat/completions \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "model": "gliner2.5-base-v1",
    "messages": [
      {
        "role": "user",
        "content": "Best regards, John Smith, Senior Software Engineer at Acme Corp. Phone: (555) 123-4567, Email: john.smith@acme.com, Office: 123 Main Street, Suite 400, San Francisco, CA 94105"
      }
    ],
    "metadata": {
      "schema": {
        "contact": [
          "name::str::Full name",
          "title::str::Job title",
          "company::str::Company name",
          "phone::str::Phone number",
          "email::str::Email address",
          "address::str::Office address"
        ]
      },
      "usecase": "json"
    }
  }'
  ```
</CodeGroup>

### Text Classification

Set `metadata.usecase` to `classification` and supply a `schema` mapping each axis to its allowed labels.

<CodeGroup>
  ```bash Responses API theme={null}
  curl https://api.zerogpu.ai/v1/responses \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "input": "I absolutely love this product! The quality is outstanding and the customer service was incredibly helpful.",
    "model": "gliner2.5-base-v1",
    "metadata": {
      "schema": {
        "sentiment": [
          "positive",
          "negative",
          "neutral"
        ]
      },
      "usecase": "classification"
    }
  }'
  ```

  ```bash Chat Completions theme={null}
  curl https://api.zerogpu.ai/v1/chat/completions \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "model": "gliner2.5-base-v1",
    "messages": [
      {
        "role": "user",
        "content": "I absolutely love this product! The quality is outstanding and the customer service was incredibly helpful."
      }
    ],
    "metadata": {
      "schema": {
        "sentiment": [
          "positive",
          "negative",
          "neutral"
        ]
      },
      "usecase": "classification"
    }
  }'
  ```
</CodeGroup>

## gliner2.5-multi-v1

> Fastino's GLiNER2.5 Multi is a 287M-parameter multilingual information extraction model built on mDeBERTa-v3-base. It supports zero-shot named entity recognition, text classification, and structured JSON extraction across multiple languages, so it can pull people, organizations, locations, attributes, and other custom entities out of text without task-specific retraining.

**References:** [Model docs](https://huggingface.co/fastino/gliner2.5-multi-v1) • [Terms](https://github.com/fastino-ai/GLiNER2/blob/main/LICENSE) • [Privacy](https://github.com/fastino-ai/GLiNER2/blob/main/LICENSE)

### Entity Extraction

Set `metadata.usecase` to `ner` and pass your own `labels` to extract named entities. `threshold` sets the minimum confidence for a span to be returned.

<CodeGroup>
  ```bash Responses API theme={null}
  curl https://api.zerogpu.ai/v1/responses \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "input": "The application is built with Python 3.11 and uses PostgreSQL 15 for storage. It runs on Kubernetes with Docker containers and communicates via gRPC.",
    "model": "gliner2.5-multi-v1",
    "metadata": {
      "labels": [
        "programming language",
        "database",
        "technology",
        "protocol"
      ],
      "usecase": "ner",
      "threshold": 0.3
    }
  }'
  ```

  ```bash Chat Completions theme={null}
  curl https://api.zerogpu.ai/v1/chat/completions \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "model": "gliner2.5-multi-v1",
    "messages": [
      {
        "role": "user",
        "content": "The application is built with Python 3.11 and uses PostgreSQL 15 for storage. It runs on Kubernetes with Docker containers and communicates via gRPC."
      }
    ],
    "metadata": {
      "labels": [
        "programming language",
        "database",
        "technology",
        "protocol"
      ],
      "usecase": "ner",
      "threshold": 0.3
    }
  }'
  ```
</CodeGroup>

### Structured Data Extraction

Set `metadata.usecase` to `json` and define a `schema` of `field::type::description` entries to pull structured records out of free text.

<CodeGroup>
  ```bash Responses API theme={null}
  curl https://api.zerogpu.ai/v1/responses \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "input": "Best regards, John Smith, Senior Software Engineer at Acme Corp. Phone: (555) 123-4567, Email: john.smith@acme.com, Office: 123 Main Street, Suite 400, San Francisco, CA 94105",
    "model": "gliner2.5-multi-v1",
    "metadata": {
      "schema": {
        "contact": [
          "name::str::Full name",
          "title::str::Job title",
          "company::str::Company name",
          "phone::str::Phone number",
          "email::str::Email address",
          "address::str::Office address"
        ]
      },
      "usecase": "json"
    }
  }'
  ```

  ```bash Chat Completions theme={null}
  curl https://api.zerogpu.ai/v1/chat/completions \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "model": "gliner2.5-multi-v1",
    "messages": [
      {
        "role": "user",
        "content": "Best regards, John Smith, Senior Software Engineer at Acme Corp. Phone: (555) 123-4567, Email: john.smith@acme.com, Office: 123 Main Street, Suite 400, San Francisco, CA 94105"
      }
    ],
    "metadata": {
      "schema": {
        "contact": [
          "name::str::Full name",
          "title::str::Job title",
          "company::str::Company name",
          "phone::str::Phone number",
          "email::str::Email address",
          "address::str::Office address"
        ]
      },
      "usecase": "json"
    }
  }'
  ```
</CodeGroup>

### Text Classification

Set `metadata.usecase` to `classification` and supply a `schema` mapping each axis to its allowed labels.

<CodeGroup>
  ```bash Responses API theme={null}
  curl https://api.zerogpu.ai/v1/responses \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "input": "I absolutely love this product! The quality is outstanding and the customer service was incredibly helpful.",
    "model": "gliner2.5-multi-v1",
    "metadata": {
      "schema": {
        "sentiment": [
          "positive",
          "negative",
          "neutral"
        ]
      },
      "usecase": "classification"
    }
  }'
  ```

  ```bash Chat Completions theme={null}
  curl https://api.zerogpu.ai/v1/chat/completions \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "model": "gliner2.5-multi-v1",
    "messages": [
      {
        "role": "user",
        "content": "I absolutely love this product! The quality is outstanding and the customer service was incredibly helpful."
      }
    ],
    "metadata": {
      "schema": {
        "sentiment": [
          "positive",
          "negative",
          "neutral"
        ]
      },
      "usecase": "classification"
    }
  }'
  ```
</CodeGroup>

## gliner2.5-decide-v1

> Fastino's GLiNER2.5 Decide is a 486M-parameter zero-shot decision and classification model built for operational AI workflows. You supply the labels or decision options at request time to classify intent, route agents and tools, triage support requests, make moderation decisions, detect completion, and determine when a request needs human handoff. It handles single-label, multi-label, and several decision heads in one call, which suits high-volume routing and decision workloads where speed and structured outputs matter.

**References:** [Model docs](https://huggingface.co/fastino/GLiNER2.5-Decide) • [Terms](https://github.com/fastino-ai/GLiNER2/blob/main/LICENSE) • [Privacy](https://github.com/fastino-ai/GLiNER2/blob/main/LICENSE)

### Decision / Routing

Set `metadata.usecase` to `decide` and pass a `labels` map of decision heads to their allowed options. Every head is answered in the same call, so one request can return an intent and a handoff verdict together.

<CodeGroup>
  ```bash Responses API theme={null}
  curl https://api.zerogpu.ai/v1/responses \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "input": "The customer says their payment failed and they want someone to call them.",
    "model": "gliner2.5-decide-v1",
    "metadata": {
      "labels": {
        "intent": [
          "payment_issue",
          "refund",
          "account_access",
          "general_question"
        ],
        "handoff": [
          "yes",
          "no"
        ]
      },
      "usecase": "decide"
    }
  }'
  ```

  ```bash Chat Completions theme={null}
  curl https://api.zerogpu.ai/v1/chat/completions \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "model": "gliner2.5-decide-v1",
    "messages": [
      {
        "role": "user",
        "content": "The customer says their payment failed and they want someone to call them."
      }
    ],
    "metadata": {
      "labels": {
        "intent": [
          "payment_issue",
          "refund",
          "account_access",
          "general_question"
        ],
        "handoff": [
          "yes",
          "no"
        ]
      },
      "usecase": "decide"
    }
  }'
  ```
</CodeGroup>

### Text Classification

Set `metadata.usecase` to `classification` and supply a `schema` mapping each axis to its allowed labels.

<CodeGroup>
  ```bash Responses API theme={null}
  curl https://api.zerogpu.ai/v1/responses \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "input": "I absolutely love this product! The quality is outstanding and the customer service was incredibly helpful.",
    "model": "gliner2.5-decide-v1",
    "metadata": {
      "schema": {
        "sentiment": [
          "positive",
          "negative",
          "neutral"
        ]
      },
      "usecase": "classification"
    }
  }'
  ```

  ```bash Chat Completions theme={null}
  curl https://api.zerogpu.ai/v1/chat/completions \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "model": "gliner2.5-decide-v1",
    "messages": [
      {
        "role": "user",
        "content": "I absolutely love this product! The quality is outstanding and the customer service was incredibly helpful."
      }
    ],
    "metadata": {
      "schema": {
        "sentiment": [
          "positive",
          "negative",
          "neutral"
        ]
      },
      "usecase": "classification"
    }
  }'
  ```
</CodeGroup>


# Text Generation
Source: https://docs.zerogpu.ai/docs/text-generation

Edge generative models for conversation and reasoning.

Text generation models produce new text from an input prompt. ZeroGPU offers open-weight heavyweights — DeepSeek's deepseek-v4.1-flash, OpenAI's gpt-oss-120b, Alibaba's qwen3-30b-a3b-fp8 and multimodal qwen3.6-35b-a3b, and Z.ai's glm-5.3-flash — for reasoning and tool use, OpenAI's gpt-5.4-nano, gpt-5.6-luna, and gpt-4.1-mini for cost-efficient high-volume work, plus conversational and reasoning models from Liquid AI, Meta's llama-3.1-8b-instruct-fast for high-volume summarization, and Meta's llama-guard-4-12b for safety classification. Each request is shown for both the [Responses API](/api-reference/responses) and the OpenAI-compatible [Chat Completions API](/api-reference/chat-completions) — except `qwen3-30b-a3b-fp8`, which is Chat Completions only.

| Model | Input /1M | Output /1M | Cached input /1M | Max tokens |
| - | -: | -: | -: | -: |
| <a href="/api-reference/models/gpt-oss-120b"><img alt="gpt-oss-120b" /> <code>gpt-oss-120b</code></a> | \$0.15 | \$0.60 | \$0.03 | 131,072 |
| <a href="/api-reference/models/qwen3-30b-a3b-fp8"><img alt="qwen3-30b-a3b-fp8" /> <code>qwen3-30b-a3b-fp8</code></a> | \$0.10 | \$0.45 | \$0.05 | 32,768 |
| <a href="/api-reference/models/deepseek-v4-1-flash"><img alt="deepseek-v4.1-flash" /> <code>deepseek-v4.1-flash</code></a> | \$0.30 | \$1.20 | \$0.003 | 1,048,576 |
| <a href="/api-reference/models/glm-5-3-flash"><img alt="glm-5.3-flash" /> <code>glm-5.3-flash</code></a> | \$0.10 | \$0.35 | \$0.015 | 1,048,576 |
| <a href="/api-reference/models/gpt-5-4-nano"><img alt="gpt-5.4-nano" /> <code>gpt-5.4-nano</code></a> | \$0.20 | \$1.25 | \$0.02 | 400,000 |
| <a href="/api-reference/models/gpt-5-6-luna"><img alt="gpt-5.6-luna" /> <code>gpt-5.6-luna</code></a> | \$0.20 | \$1.20 | \$0.20 | 272,000 |
| <a href="/api-reference/models/gpt-4-1-mini"><img alt="gpt-4.1-mini" /> <code>gpt-4.1-mini</code></a> | \$0.40 | \$1.60 | \$0.10 | 1,047,576 |
| <a href="/api-reference/models/llama-3-1-8b-instruct-fast"><img alt="llama-3.1-8b-instruct-fast" /> <code>llama-3.1-8b-instruct-fast</code></a> | \$0.15 | \$0.28 | \$0.025 | 131,072 |
| <a href="/api-reference/models/llama-guard-4-12b"><img alt="llama-guard-4-12b" /> <code>llama-guard-4-12b</code></a> | \$0.18 | \$0.18 | — | 163,840 |
| <a href="/api-reference/models/lfm2-5-1-2b-instruct"><img alt="LFM2.5-1.2B-Instruct" /> <code>LFM2.5-1.2B-Instruct</code></a> | \$0.02 | \$0.05 | — | 32,768 |
| <a href="/api-reference/models/lfm2-5-1-2b-thinking"><img alt="LFM2.5-1.2B-Thinking" /> <code>LFM2.5-1.2B-Thinking</code></a> | \$0.02 | \$0.05 | — | 32,768 |
| <a href="/api-reference/models/qwen3-6-35b-a3b"><img alt="qwen3.6-35b-a3b" /> <code>qwen3.6-35b-a3b</code></a> | \$0.20 | \$1.50 | \$0.20 | — |

## gpt-oss-120b

> OpenAI's gpt-oss-120b is an open-weight Mixture-of-Experts model with 117B total parameters (5.1B active per token), served on ZeroGPU for general text generation. It reasons through a problem before answering and returns that reasoning trace alongside the final answer, and it supports function calling, batch tasks, and a 131,072-token context window. When a task needs frontier-level reasoning or tool use beyond what the smaller edge models cover, this is the model.

**References:** [Model docs](https://huggingface.co/openai/gpt-oss-120b) • [Terms](https://zerogpu.ai/terms) • [Privacy](https://zerogpu.ai/privacy-policy)

<CodeGroup>
  ```bash Responses API theme={null}
  curl https://api.zerogpu.ai/v1/responses \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "input": "Name my WiFi network something that will make my neighbors laugh. Give me your top 3 with a one-line reason each.",
    "model": "gpt-oss-120b",
    "max_tokens": 800,
    "instructions": "You are a witty naming consultant. Be brief."
  }'
  ```

  ```bash Chat Completions theme={null}
  curl https://api.zerogpu.ai/v1/chat/completions \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "model": "gpt-oss-120b",
    "messages": [
      {
        "role": "system",
        "content": "You are a witty naming consultant. Be brief."
      },
      {
        "role": "user",
        "content": "Name my WiFi network something that will make my neighbors laugh. Give me your top 3 with a one-line reason each."
      }
    ],
    "max_tokens": 800
  }'
  ```
</CodeGroup>

The model reasons before it answers and returns the trace alongside the final text — a `reasoning` output item in Responses, and a `reasoning` field on the message in Chat Completions.

<CodeGroup>
  ```json Responses API theme={null}
  {
    "id": "id-1784318124857",
    "object": "response",
    "created_at": 1784318124,
    "model": "gpt-oss-120b",
    "status": "completed",
    "instructions": "You are a witty naming consultant. Be brief.",
    "output": [
      {
        "id": "rs_8d4dc6f3f33d0d28",
        "type": "reasoning",
        "content": [
          {
            "type": "reasoning_text",
            "text": "We need to respond with a top 3 WiFi names that will make neighbors laugh, each with a one-line reason. Be witty. Keep brief."
          }
        ]
      },
      {
        "id": "msg_9c3dfbf63445d80a",
        "type": "message",
        "role": "assistant",
        "status": "completed",
        "content": [
          {
            "type": "output_text",
            "text": "**1. “FBI Surveillance Van #42”** – Neighbors will double-check the street before they even think about streaming.\n\n**2. “Drop It Like It’s Hotspot”** – A pun that’s so cheesy it’ll have the whole block groaning and giggling.\n\n**3. “Tell My Wi-Fi I Love You”** – Because nothing says “I’m caring” like a reminder that even your router needs affection."
          }
        ]
      }
    ],
    "usage": {
      "input_tokens": 0,
      "output_tokens": 0,
      "total_tokens": 0
    }
  }
  ```

  ```json Chat Completions theme={null}
  {
    "id": "id-1784318142640",
    "object": "chat.completion",
    "created": 1784318142,
    "model": "gpt-oss-120b",
    "choices": [
      {
        "index": 0,
        "finish_reason": "stop",
        "message": {
          "role": "assistant",
          "content": "**1. “Drop It Like It’s Hotspot”** – A cheeky nod to the song that’ll make anyone pause and grin while they log in.\n\n**2. “FBI Surveillance Van #42”** – The classic prank that’ll have neighbors double-checking their curtains.\n\n**3. “Tell My Wi-Fi I’m Sorry”** – Because we all know the neighbor who borrows bandwidth—now they’ll feel the love (and the guilt).",
          "reasoning": "We need to give top 3 witty WiFi network names that will make neighbors laugh, with one-line reason each. Be brief.",
          "tool_calls": []
        }
      }
    ],
    "usage": {
      "prompt_tokens": 107,
      "completion_tokens": 160,
      "total_tokens": 267
    }
  }
  ```
</CodeGroup>

## qwen3-30b-a3b-fp8

<Note>
  This model supports the Chat Completions API only — the Responses endpoint
  (`/v1/responses`) is not available for this model.
</Note>

> Alibaba's Qwen3-30B-A3B is an open-weight Mixture-of-Experts model with 30.5B total parameters (3.3B active per token), served on ZeroGPU as an FP8 build for efficient inference. It thinks through a problem before answering and returns that reasoning alongside the final text, and it supports function calling, streaming, batch tasks, and 100+ languages with a 32,768-token context window. When you want frontier-style reasoning and tool use in a lighter, multilingual package, this is the model.

**References:** [Model docs](https://huggingface.co/Qwen/Qwen3-30B-A3B-FP8) • [Terms](https://zerogpu.ai/terms) • [Privacy](https://zerogpu.ai/privacy-policy)

```bash Chat Completions theme={null}
curl https://api.zerogpu.ai/v1/chat/completions \
  -H "content-type: application/json" \
  -H "x-api-key: YOUR_API_KEY" \
  -d '{
  "model": "qwen3-30b-a3b-fp8",
  "messages": [
    {
      "role": "system",
      "content": "You are a pragmatic senior engineer. Be brief."
    },
    {
      "role": "user",
      "content": "My API is getting rate-limited by a third party. Give me 3 battle-tested strategies to handle it, one line each."
    }
  ],
  "max_tokens": 800
}'
```

The model reasons before it answers and returns the trace alongside the final text as a `reasoning` field on the message.

```json Response theme={null}
{
  "id": "id-1784324156320",
  "object": "chat.completion",
  "created": 1784324156,
  "model": "qwen3-30b-a3b-fp8",
  "choices": [
    {
      "index": 0,
      "finish_reason": "stop",
      "message": {
        "role": "assistant",
        "content": "**1. Exponential backoff with jitter** – Retry on 429s with doubling delays plus randomness so your retries don't stampede in sync.\n\n**2. Client-side token bucket** – Throttle your own outbound rate just below the provider's limit so you never hit the wall in the first place.\n\n**3. Cache and coalesce** – Cache hot responses and merge duplicate in-flight requests so repeat calls never reach the third party at all.",
        "reasoning": "The user's API is being rate-limited by a third party and wants 3 battle-tested strategies, one line each. The classic answers: exponential backoff with jitter, client-side throttling below the limit, and caching plus request coalescing. Keep each to one line.",
        "tool_calls": []
      }
    }
  ],
  "usage": {
    "prompt_tokens": 52,
    "completion_tokens": 143,
    "total_tokens": 195
  }
}
```

## deepseek-v4.1-flash

> DeepSeek's DeepSeek-V4.1-Flash is an open-weight sparse Mixture-of-Experts model and the first built on DeepSeek's Causal Encoder-Decoder (CED) architecture, activating 8B parameters on input and 16B on output, served on ZeroGPU for general text generation. It keeps the 1,048,576-token (1M) context window of the V4 Flash line, which suits large codebases, long documents, extended conversations, and multi-step agent tasks, and supports function calling alongside both fast non-thinking replies and higher-effort reasoning.

**References:** [Model docs](https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash) • [Terms](https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash/blob/main/LICENSE) • [Privacy](https://zerogpu.ai/privacy-policy)

<CodeGroup>
  ```bash Responses API theme={null}
  curl https://api.zerogpu.ai/v1/responses \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "input": "Hello!",
    "model": "deepseek-v4.1-flash"
  }'
  ```

  ```bash Chat Completions theme={null}
  curl https://api.zerogpu.ai/v1/chat/completions \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "model": "deepseek-v4.1-flash",
    "messages": [
      {
        "role": "system",
        "content": "You are a support triage analyst. Be concise."
      },
      {
        "role": "user",
        "content": "Classify this ticket by urgency (low/medium/high) and category, then summarize it in one line: '"'"'Our checkout page has been throwing intermittent 502 errors since this morning'"'"'s deploy. Roughly 1 in 5 customers can'"'"'t complete payment, and abandoned carts are spiking. Rolling back didn'"'"'t help."
      }
    ],
    "max_tokens": 800
  }'
  ```
</CodeGroup>

The model returns its answer as the assistant message — an `output_text` item in Responses, the `message.content` field in Chat Completions — with `usage` token counts alongside it.

## glm-5.3-flash

> Z.ai's GLM-5.3-Flash is an efficient open-weight model for coding and long-horizon agent tasks, served on ZeroGPU for general text generation. Its hybrid sparse and linear attention keeps long-context behaviour accurate across a 1,048,576-token (1M) window while reducing compute, and it supports function calling and adjustable reasoning effort.

**References:** [Model docs](https://huggingface.co/zai-org/GLM-5.3-Flash) • [Terms](https://huggingface.co/zai-org/GLM-5.3-Flash/blob/main/LICENSE) • [Privacy](https://zerogpu.ai/privacy-policy)

<CodeGroup>
  ```bash Responses API theme={null}
  curl https://api.zerogpu.ai/v1/responses \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "input": "Hello!",
    "model": "glm-5.3-flash"
  }'
  ```

  ```bash Chat Completions theme={null}
  curl https://api.zerogpu.ai/v1/chat/completions \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "model": "glm-5.3-flash",
    "messages": [
      {
        "role": "user",
        "content": "Hello!"
      }
    ]
  }'
  ```
</CodeGroup>

The model returns its answer as the assistant message — an `output_text` item in Responses, the `message.content` field in Chat Completions — with `usage` token counts alongside it.

## gpt-5.4-nano

> OpenAI's GPT-5.4 nano is the most cost-efficient model in the GPT-5.4 family, served on ZeroGPU for high-volume and latency-sensitive workloads such as classification, extraction, routing, and sub-agent tasks. It supports function calling and structured outputs across a 400,000-token context window.

**References:** [Terms](https://openai.com/policies/services-agreement/) • [Privacy](https://zerogpu.ai/privacy-policy)

<CodeGroup>
  ```bash Responses API theme={null}
  curl https://api.zerogpu.ai/v1/responses \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "input": "Hello!",
    "model": "gpt-5.4-nano"
  }'
  ```

  ```bash Chat Completions theme={null}
  curl https://api.zerogpu.ai/v1/chat/completions \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "model": "gpt-5.4-nano",
    "messages": [
      {
        "role": "user",
        "content": "Hello!"
      }
    ]
  }'
  ```
</CodeGroup>

The model returns its answer as the assistant message — an `output_text` item in Responses, the `message.content` field in Chat Completions — with `usage` token counts alongside it.

## gpt-5.6-luna

> OpenAI's GPT-5.6 Luna is the cost-optimized model of the GPT-5.6 family, served on ZeroGPU for cost-sensitive, high-volume workloads. It supports adjustable reasoning effort, function calling, and structured outputs across a 272,000-token context window.

**References:** [Terms](https://openai.com/policies/services-agreement/) • [Privacy](https://zerogpu.ai/privacy-policy)

<CodeGroup>
  ```bash Responses API theme={null}
  curl https://api.zerogpu.ai/v1/responses \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "input": "Hello!",
    "model": "gpt-5.6-luna"
  }'
  ```

  ```bash Chat Completions theme={null}
  curl https://api.zerogpu.ai/v1/chat/completions \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "model": "gpt-5.6-luna",
    "messages": [
      {
        "role": "user",
        "content": "Hello!"
      }
    ]
  }'
  ```
</CodeGroup>

The model returns its answer as the assistant message — an `output_text` item in Responses, the `message.content` field in Chat Completions — with `usage` token counts alongside it.

## gpt-4.1-mini

> OpenAI's GPT-4.1 mini is the fast, cost-efficient model of the GPT-4.1 family, served on ZeroGPU for general text generation. It excels at instruction following and tool calling, supports function calling and structured outputs, and works across a 1,047,576-token (1M) context window with low latency.

**References:** [Terms](https://openai.com/policies/services-agreement/) • [Privacy](https://zerogpu.ai/privacy-policy)

<CodeGroup>
  ```bash Responses API theme={null}
  curl https://api.zerogpu.ai/v1/responses \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "input": "Hello!",
    "model": "gpt-4.1-mini"
  }'
  ```

  ```bash Chat Completions theme={null}
  curl https://api.zerogpu.ai/v1/chat/completions \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "model": "gpt-4.1-mini",
    "messages": [
      {
        "role": "user",
        "content": "Hello!"
      }
    ]
  }'
  ```
</CodeGroup>

The model returns its answer as the assistant message — an `output_text` item in Responses, the `message.content` field in Chat Completions — with `usage` token counts alongside it.

## llama-3.1-8b-instruct-fast

> Meta's Llama 3.1 Instruct, tuned for fast, low-cost summarization at scale on the ZeroGPU edge network. Its 128K-token context window takes in entire documents, long transcripts, and full email or chat threads in a single pass. No chunking, no stitching partial summaries back together. Built for the high-volume summarization that runs around your core product: condensing long reports and transcripts, summarizing email and chat threads, and processing articles in bulk. When context length matters and you're summarizing on every request, this is the model that keeps it fast and affordable.

**References:** [Model docs](https://github.com/meta-llama/llama-models/tree/main/models/llama3_1) • [Terms](https://github.com/meta-llama/llama-models/blob/main/models/llama3_1/LICENSE) • [Privacy](https://github.com/meta-llama/llama-models/blob/main/models/llama3_1/USE_POLICY.md)

<CodeGroup>
  ```bash Responses API theme={null}
  curl https://api.zerogpu.ai/v1/responses \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "input": "The global semiconductor industry is undergoing one of its most significant structural shifts in decades, driven by a combination of geopolitical tensions, surging demand from artificial intelligence workloads, and a wave of government-backed industrial policy across the United States, Europe, and Asia. At the center of this transformation is a race to onshore chip manufacturing capacity that had, for thirty years, been quietly concentrated in Taiwan and South Korea. The United States CHIPS and Science Act, signed into law in 2022, allocated over $52 billion in subsidies to incentivize domestic semiconductor fabrication. Since then, companies including TSMC, Intel, Samsung, and Micron have announced or broken ground on new fabs in Arizona, Ohio, Texas, and Idaho. However, construction timelines have slipped, costs have ballooned, and a shortage of skilled workers has prompted some manufacturers to bring in engineers from overseas — a move that has drawn political scrutiny even as it addresses a genuine talent gap. Meanwhile, the explosion of AI model training and inference has fundamentally altered the demand profile for chips. Graphics processing units originally designed for gaming, particularly those made by NVIDIA, have become the primary compute substrate for large language models. NVIDIA's H100 and successor Blackwell-series GPUs now trade at significant premiums on secondary markets, with some hyperscalers reporting lead times of over a year for large cluster orders. This bottleneck has accelerated investment in custom silicon: Google's Tensor Processing Units, Amazon's Trainium and Inferentia chips, and Meta's MTIA accelerators are all attempts to reduce dependence on third-party GPU supply chains. On the geopolitical front, the United States has progressively tightened export controls on advanced chips and chip-making equipment destined for China, with the most recent rules targeting gate-all-around transistor technology and high-bandwidth memory. China, in response, has dramatically increased state investment in its domestic semiconductor ecosystem, with companies like SMIC and Huawei making incremental but closely watched progress at advanced nodes. Analysts remain divided on how quickly China can close the gap, with estimates ranging from five to fifteen years depending on assumptions about technology transfer, equipment access, and the pace of indigenous innovation. The equipment layer — often overlooked in public discourse — may prove to be the most consequential chokepoint. ASML, the Dutch company that holds a near-monopoly on extreme ultraviolet lithography machines, has become a focal point of both trade diplomacy and industrial strategy. Without EUV tools, fabricating chips below 7 nanometers at scale is effectively impossible, giving ASML and its supply chain extraordinary leverage in a world where nations are suddenly treating semiconductor access as a matter of national security.",
    "model": "llama-3.1-8b-instruct-fast"
  }'
  ```

  ```bash Chat Completions theme={null}
  curl https://api.zerogpu.ai/v1/chat/completions \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "model": "llama-3.1-8b-instruct-fast",
    "messages": [
      {
        "role": "user",
        "content": "The global semiconductor industry is undergoing one of its most significant structural shifts in decades, driven by a combination of geopolitical tensions, surging demand from artificial intelligence workloads, and a wave of government-backed industrial policy across the United States, Europe, and Asia. At the center of this transformation is a race to onshore chip manufacturing capacity that had, for thirty years, been quietly concentrated in Taiwan and South Korea. The United States CHIPS and Science Act, signed into law in 2022, allocated over $52 billion in subsidies to incentivize domestic semiconductor fabrication. Since then, companies including TSMC, Intel, Samsung, and Micron have announced or broken ground on new fabs in Arizona, Ohio, Texas, and Idaho. However, construction timelines have slipped, costs have ballooned, and a shortage of skilled workers has prompted some manufacturers to bring in engineers from overseas — a move that has drawn political scrutiny even as it addresses a genuine talent gap. Meanwhile, the explosion of AI model training and inference has fundamentally altered the demand profile for chips. Graphics processing units originally designed for gaming, particularly those made by NVIDIA, have become the primary compute substrate for large language models. NVIDIA's H100 and successor Blackwell-series GPUs now trade at significant premiums on secondary markets, with some hyperscalers reporting lead times of over a year for large cluster orders. This bottleneck has accelerated investment in custom silicon: Google's Tensor Processing Units, Amazon's Trainium and Inferentia chips, and Meta's MTIA accelerators are all attempts to reduce dependence on third-party GPU supply chains. On the geopolitical front, the United States has progressively tightened export controls on advanced chips and chip-making equipment destined for China, with the most recent rules targeting gate-all-around transistor technology and high-bandwidth memory. China, in response, has dramatically increased state investment in its domestic semiconductor ecosystem, with companies like SMIC and Huawei making incremental but closely watched progress at advanced nodes. Analysts remain divided on how quickly China can close the gap, with estimates ranging from five to fifteen years depending on assumptions about technology transfer, equipment access, and the pace of indigenous innovation. The equipment layer — often overlooked in public discourse — may prove to be the most consequential chokepoint. ASML, the Dutch company that holds a near-monopoly on extreme ultraviolet lithography machines, has become a focal point of both trade diplomacy and industrial strategy. Without EUV tools, fabricating chips below 7 nanometers at scale is effectively impossible, giving ASML and its supply chain extraordinary leverage in a world where nations are suddenly treating semiconductor access as a matter of national security."
      }
    ]
  }'
  ```
</CodeGroup>

The summary is returned as `output[].content[].text` (Responses API) or `choices[].message.content` (Chat Completions).

```text Response theme={null}
The global semiconductor industry is undergoing a significant transformation driven by geopolitical tensions, surging demand from artificial intelligence workloads, and government-backed industrial policies. A race to onshore chip manufacturing capacity is underway, with the US, Europe, and Asia investing heavily in domestic fabrication, while companies like TSMC, Intel, and Samsung are building new fabs in the US. However, construction timelines have slipped, costs have increased, and a shortage of skilled workers has prompted some manufacturers to bring in overseas engineers. The demand for chips has also changed, with AI model training and inference driving a shortage of graphics processing units, particularly those made by NVIDIA. In response, companies like Google, Amazon, and Meta are investing in custom silicon to reduce dependence on third-party supply chains. The US has tightened export controls on advanced chips and equipment destined for China, which has responded with increased state investment in its domestic semiconductor ecosystem. The equipment layer, particularly ASML's near-monopoly on extreme ultraviolet lithography machines, may prove to be the most consequential chokepoint in the industry.
```

## llama-guard-4-12b

> Meta's Llama Guard 4 12B is a multimodal safety classification model for moderating text, images, and mixed text-image inputs. It evaluates both incoming prompts and generated responses, returning a safe or unsafe classification along with the policy categories a violation falls under. It handles multilingual text and multiple images in one call, which makes it a fit for chat moderation, prompt and response filtering, policy enforcement, and agent guardrails. Built on a dense 12B architecture derived from Llama 4 Scout, it is designed as a dedicated safety layer in front of a production model rather than as a general-purpose generator.

**References:** [Model docs](https://huggingface.co/meta-llama/Llama-Guard-4-12B) • [Terms](https://huggingface.co/meta-llama/Llama-Guard-4-12B) • [Privacy](https://huggingface.co/meta-llama/Llama-Guard-4-12B)

<CodeGroup>
  ```bash Responses API theme={null}
  curl https://api.zerogpu.ai/v1/responses \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "model": "llama-guard-4-12b",
    "input": [
      {
        "type": "text",
        "text": "I am so angry at this person that I want to hurt them. They are worthless and should be scared of what I might do next."
      }
    ]
  }'
  ```

  ```bash Chat Completions theme={null}
  curl https://api.zerogpu.ai/v1/chat/completions \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "model": "llama-guard-4-12b",
    "messages": [
      {
        "role": "user",
        "content": "I am so angry at this person that I want to hurt them. They are worthless and should be scared of what I might do next."
      }
    ]
  }'
  ```

  ```bash Moderations API theme={null}
  curl https://api.zerogpu.ai/v1/moderations \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "model": "llama-guard-4-12b",
    "input": "I am so angry at this person that I want to hurt them. They are worthless and should be scared of what I might do next."
  }'
  ```
</CodeGroup>

On the Responses and Chat Completions endpoints the verdict comes back as the output text — `safe`, or `unsafe` followed by the violated policy categories. On `/v1/moderations` the response is OpenAI's moderations envelope, the same shape [`zlm-v1-moderation-edge`](/docs/moderation) returns.

## LFM2.5-1.2B-Instruct

> Liquid AI's LFM2.5-1.2B-Instruct is a hybrid architecture model purpose-built for on-device deployment, trained on 28 trillion tokens with multi-stage reinforcement learning. It delivers best-in-class instruction following and native tool-calling at the 1B scale — something very few models this size can do reliably. Its hybrid design gives it significantly faster inference with lower memory usage compared to transformer-only models of the same size. If you need a conversational model that can actually run on real devices without a GPU, this is it. Supports 8 languages including English, Chinese, Japanese, and Spanish.

**References:** [Model docs](https://docs.liquid.ai/deployment/on-device/android/ai-agent-usage-guide#text-models) • [Terms](https://www.liquid.ai/terms-conditions) • [Privacy](https://www.liquid.ai/lfm-license)

<CodeGroup>
  ```bash Responses API theme={null}
  curl https://api.zerogpu.ai/v1/responses \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "input": "I just completed a 5K run in 28 minutes. Give me a short motivational follow-up message.",
    "model": "LFM2.5-1.2B-Instruct",
    "instructions": "You are a friendly in-app assistant for a fitness app."
  }'
  ```

  ```bash Chat Completions theme={null}
  curl https://api.zerogpu.ai/v1/chat/completions \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "model": "LFM2.5-1.2B-Instruct",
    "messages": [
      {
        "role": "system",
        "content": "You are a friendly in-app assistant for a fitness app."
      },
      {
        "role": "user",
        "content": "I just completed a 5K run in 28 minutes. Give me a short motivational follow-up message."
      }
    ]
  }'
  ```
</CodeGroup>

```text Response theme={null}
That's an amazing accomplishment! Keep up the great work, each step brings you closer to your goals. You're stronger than you think!

Would you like tips to help you keep improving?
```

## LFM2.5-1.2B-Thinking

> Liquid AI's LFM2.5-1.2B-Thinking is a compact reasoning model that works through a problem step by step before it answers. Built by Liquid AI, it generates an explicit chain-of-thought trace so for multi-step math, planning, data extraction, and agentic tool-calling, you get verifiable intermediate reasoning, not just a final guess. At 1.2B parameters it fits under a gigabyte and runs fast on CPU at the edge, rivaling models several times its size on reasoning benchmarks. Reach for a frontier model when the task is knowledge-heavy or code-intensive, and keep this one as the planning brain that decides and orchestrates. When the task needs reasoning, this is the model.

**References:** [Model docs](https://docs.liquid.ai/deployment/on-device/android/ai-agent-usage-guide#text-models) • [Terms](https://www.liquid.ai/terms-conditions) • [Privacy](https://www.liquid.ai/lfm-license)

<CodeGroup>
  ```bash Responses API theme={null}
  curl https://api.zerogpu.ai/v1/responses \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "input": "Our whole team got locked out of the dashboard this morning after the payment failed. We have a product demo with a customer in 2 hours and cannot log in. Card on file should be valid.",
    "model": "LFM2.5-1.2B-Thinking",
    "instructions": "You are a support triage assistant. Read the ticket, briefly reason about the signals, then assign one team (Billing, Technical, Account, General) and one priority (Low, Medium, High). Decide once and commit. End with a single line: TEAM | PRIORITY."
  }'
  ```

  ```bash Chat Completions theme={null}
  curl https://api.zerogpu.ai/v1/chat/completions \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "model": "LFM2.5-1.2B-Thinking",
    "messages": [
      {
        "role": "system",
        "content": "You are a support triage assistant. Read the ticket, briefly reason about the signals, then assign one team (Billing, Technical, Account, General) and one priority (Low, Medium, High). Decide once and commit. End with a single line: TEAM | PRIORITY."
      },
      {
        "role": "user",
        "content": "Our whole team got locked out of the dashboard this morning after the payment failed. We have a product demo with a customer in 2 hours and cannot log in. Card on file should be valid."
      }
    ]
  }'
  ```
</CodeGroup>

The model returns its answer alongside an explicit reasoning trace.

```text Response theme={null}
The capital of Japan is **Tokyo**.

**Reasoning:**
1. Japan's political, cultural, and economic center is located in Tokyo.
2. Historically, Tokyo has served as Japan's...
```

## qwen3.6-35b-a3b

> Alibaba's Qwen3.6-35B-A3B is a fast, open-weight multimodal model for applications that need to understand text, images, and video, served on ZeroGPU as an FP8 build. It suits visual question answering, document and screenshot analysis, video understanding, content extraction, agent workflows, and tool calling, and it supports structured output. Its Mixture-of-Experts architecture keeps inference efficient while it works through large amounts of multimodal content. When the input is images or video rather than text alone, this is the model.

**References:** [Model docs](https://huggingface.co/Qwen/Qwen3.6-35B-A3B) • [Terms](https://huggingface.co/Qwen/Qwen3.6-35B-A3B) • [Privacy](https://huggingface.co/Qwen/Qwen3.6-35B-A3B)

<CodeGroup>
  ```bash Responses API theme={null}
  curl https://api.zerogpu.ai/v1/responses \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "input": [
      {
        "role": "user",
        "content": [
          {
            "type": "input_image",
            "detail": "auto",
            "image_url": "https://upload.wikimedia.org/wikipedia/commons/3/3a/Cat03.jpg"
          },
          {
            "text": "What animal is this, and what is it doing?",
            "type": "input_text"
          }
        ]
      }
    ],
    "model": "qwen3.6-35b-a3b",
    "temperature": 0.2,
    "max_output_tokens": 512
  }'
  ```

  ```bash Chat Completions theme={null}
  curl https://api.zerogpu.ai/v1/chat/completions \
    -H "content-type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -H "x-project-id: YOUR_PROJECT_ID" \
    -d '{
    "model": "qwen3.6-35b-a3b",
    "messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "video_url",
            "video_url": {
              "url": "https://qianwen-res.oss-accelerate.aliyuncs.com/Qwen3.5/demo/video/N1cdUjctpG8.mp4"
            }
          },
          {
            "type": "image_url",
            "image_url": {
              "url": "https://upload.wikimedia.org/wikipedia/commons/3/3a/Cat03.jpg",
              "detail": "auto"
            }
          },
          {
            "text": "Describe what happens in the video, then say whether the animal in the photo appears in it.",
            "type": "text"
          }
        ]
      }
    ],
    "max_tokens": 512,
    "temperature": 0.2
  }'
  ```
</CodeGroup>

The model returns a text answer describing the images and video it was given.


# Introduction
Source: https://docs.zerogpu.ai/index

Run high-volume AI tasks on specialized small and nano language models across an edge-powered network - faster and cheaper, through one OpenAI-compatible API.

ZeroGPU is the compute efficiency layer for AI inference. It runs repeatable, high-volume AI tasks on specialized small and nano language models across an edge-powered network - so the same workloads run faster and cheaper than on centralized GPUs.

Send a request to one OpenAI-compatible endpoint; ZeroGPU picks the right small or nano model and runs it on the right compute. Frontier models for reasoning. ZeroGPU for repeatable execution - classification, extraction, moderation, summarization, routing - at the scale AI demands.

## Make your first call

Find your API key and project ID in the [ZeroGPU dashboard](https://platform.zerogpu.ai/dashboard).

```bash wrap theme={null}
curl --location 'https://api.zerogpu.ai/v1/responses' \
  --header 'content-type: application/json' \
  --header 'x-api-key: YOUR_API_KEY' \
  --header 'x-project-id: YOUR_PROJECT_ID' \
  --data '{
  "input": "Technology has quietly reshaped the rhythm of everyday life, weaving itself into routines so seamlessly that it often goes unnoticed. From the moment a smartphone alarm wakes someone in the morning to the final glance at a glowing screen before sleep, digital systems guide communication, navigation, work, and entertainment. This transformation did not happen overnight. It emerged through decades of incremental innovation, each new tool building upon the last, until the modern world became deeply interconnected.One of the most significant changes has been the speed at which information travels.",
  "model": "zlm-v1-iab-classify-edge"
}'
```

Already using the OpenAI SDK? Point it at ZeroGPU:

```python wrap theme={null}
from openai import OpenAI

client = OpenAI(
    base_url="https://api.zerogpu.ai/v1",
    api_key="unused",  # ZeroGPU authenticates via the headers below
    default_headers={
        "x-api-key": "YOUR_API_KEY",
        "x-project-id": "YOUR_PROJECT_ID",
    },
)

resp = client.responses.create(
    model="zlm-v1-iab-classify-edge",
    input="Technology has quietly reshaped the rhythm of everyday life, weaving itself into routines so seamlessly that it often goes unnoticed. From the moment a smartphone alarm wakes someone in the morning to the final glance at a glowing screen before sleep, digital systems guide communication, navigation, work, and entertainment. This transformation did not happen overnight. It emerged through decades of incremental innovation, each new tool building upon the last, until the modern world became deeply interconnected.One of the most significant changes has been the speed at which information travels.",
)
```

🎉 Either call returns the same classification result:

```json theme={null}
{
  "audience": [
    { "name": "Technology & Computing", "score": 0.8194953318627214 },
    { "name": "Consumer Electronics", "score": 0.7730970665427267 },
    { "name": "IT and Internet Support", "score": 0.6476231908025892 }
  ],
  "content": {
    "iab_1_0": [
      { "name": "Technology & Computing", "score": 0.8130967783784443 },
      { "name": "Automotive", "score": 0.6588144281246334 }
    ],
    "iab_2_2": [
      { "name": "Technology & Computing", "score": 0.8130967783784443 },
      { "name": "Wearable Technology", "score": 0.7080836335415118 },
      { "name": "Auto Infotainment Technologies", "score": 0.6588144281246334 }
    ]
  }
}
```

## What you can run

<Columns>
  <Card title="Text classification" icon="tags" href="/docs/text-classification" />

  <Card title="Text generation" icon="pen-nib" href="/docs/text-generation" />

  <Card title="Open Weight" icon="cubes" href="/docs/open-weight" />
</Columns>

## What you get

<Columns>
  <Card title="Lower cost and latency" icon="gauge-high">
    Specialized small and nano models on distributed compute. \~10x faster and 50%+ cheaper on production tasks.
  </Card>

  <Card title="One OpenAI-compatible API" icon="plug">
    `POST /v1/responses`; integrate with minimal code changes.
  </Card>

  <Card title="Built for production" icon="chart-line">
    Token usage, latency, and volume per request, with project isolation for dev, staging, and production.
  </Card>
</Columns>

## How it works

<Steps>
  <Step title="Create a project">
    One organization, multiple projects. Each gets its own API key and dashboard.
  </Step>

  <Step title="Send requests">
    POST to `/v1/responses` with your key, project ID, and model. Response comes back as structured JSON.
  </Step>

  <Step title="Monitor everything">
    Token usage, request volume, latency, error rates: all visible in the dashboard. Debug individual requests in Logs.
  </Step>
</Steps>

Workload analysis → Model selection → Edge orchestration. ZeroGPU classifies each task, picks the fastest viable model, and routes it to the right compute. See [How ZeroGPU works](/docs/how-zerogpu-works) for the full picture.

## Go deeper

<Columns>
  <Card title="Quickstart" icon="play" href="/docs/quickstart" />

  <Card title="How ZeroGPU works" icon="diagram-project" href="/docs/how-zerogpu-works" />

  <Card title="Model Catalog" icon="table-list" href="/docs/model-catalog" />

  <Card title="API Reference" icon="terminal" href="/api-reference/responses" />
</Columns>

<Card title="Be an Edge Provider" icon="tower-broadcast" href="/docs/edge-providers">
  Contribute compute from your idle devices to the ZeroGPU edge network and help power distributed AI inference.
</Card>


# Claude Code Plugin
Source: https://docs.zerogpu.ai/integrations/claude-code-plugin

Route Claude Code subagents and slash commands through ZeroGPU's nano language models.

Claude Code is Anthropic's agentic coding tool that runs Claude directly in your terminal, with file editing, command execution, and a plugin system that extends sessions with custom slash commands and skills. Plugins package skills that Claude can auto-invoke when a user request matches, or that users can call explicitly by name. The plugin marketplace lets teams share these extensions across projects.

ZeroGPU is an ultra-fast, compute-efficient inference provider for apps and agents. We run purpose-built small and nano language models across an edge-powered network for the high-volume, purpose-specific tasks your app or agent runs constantly. Plug in our OpenAI-compatible API and you're live - zero GPU infrastructure, serverless, auto-scaling by default.

## Overview

This guide walks through installing the `zerogpu-router` plugin, which exposes every ZeroGPU CLI command as a Claude Code skill. You'll install the CLI, authenticate, load the plugin from its marketplace, and try both auto-invoked and manually-invoked skills. By the end you'll be able to offload cheap, well-defined NLP tasks (classification, entity and PII extraction, short chat) from Claude to ZeroGPU's edge-optimized models without leaving your terminal session.

## Video walkthrough

<iframe title="YouTube video player" />

## Quickstart

### Prerequisites

* Node.js 20 or newer.
* [Claude Code](https://docs.claude.com/en/docs/claude-code) installed: `npm install -g @anthropic-ai/claude-code`.
* A ZeroGPU [API key](https://platform.zerogpu.ai/dashboard) and Project ID.
* A model ID from the [model catalog](/docs/model-catalog) if you plan to call models by name.

### Get your ZeroGPU API key

1. Sign in to the [ZeroGPU dashboard](https://platform.zerogpu.ai/dashboard).
2. Open **API Keys** and click **Create key**.
3. Copy the key (starts with `zgpu-api-`) and grab your Project ID (UUID) from the project settings page.

### Install the ZeroGPU CLI

```bash theme={null}
npm install -g zerogpu-cli
zerogpu --version
```

Then authenticate so every skill call works without re-prompting:

```bash theme={null}
zerogpu login
```

You'll be prompted for your API key and Project ID. For CI or non-interactive setups, pass them as flags:

```bash theme={null}
zerogpu login \
  --api-key zgpu-api-XXXXXXXXXXXXXXXXXX \
  --project-id 4ed3e5bb-c2ed-4d4a-8a66-2b161a27fd1a
```

### Your first request

Start a Claude Code session by running `claude` in your terminal, then add the marketplace and install the plugin:

```text theme={null}
/plugin marketplace add zerogpu/zerogpu-router
/plugin install zerogpu-router@zerogpu
/reload-plugins
```

Confirm it's loaded with `/plugin`. You should see `zerogpu-router - enabled`. Now try a skill:

```text theme={null}
/zerogpu-router:redact-pii "Email John Smith at john@acme.com about invoice 12345."
```

You'll get back: `Email [PERSON] at [EMAIL] about invoice 12345.`

## Usage

Every inference skill auto-invokes when Claude detects a matching request, or you can call any skill explicitly with `/zerogpu-router:<name> <args>`. Two skills (`signin` and `status`) are manual-only. This section walks through every skill in the plugin: what it does, which ZeroGPU model it routes to, the full argument surface, and the shape of the response you'll get back.

#### How auto-invocation works

The plugin ships each skill with a description that Claude reads when deciding whether to invoke it for a given user message. You don't have to remember slash command names: just describe the task. Claude picks based on intent words like "redact", "scrub", "classify by", "extract", or "tag this as", and on the structure of the data you supply (a label list implies zero-shot; a JSON schema with category axes implies structured classification).

Three common auto-invoked patterns:

```text theme={null}
Redact PII from this support ticket before I paste it into our public tracker:

"Hi team, this is Sarah Chen (sarah.chen@northwind-labs.com, +1 415-555-0182).
Our prod database started throwing connection timeouts around 2:14 AM PT last
night. The on-call engineer Marcus Rivera restarted pgbouncer but the issue
came back. Billing should go to our CFO Priya Patel at
priya.patel@northwind-labs.com, billing address 1455 Market St, Suite 600,
San Francisco, CA 94103."
```

Claude routes to `redact-pii`. Names, emails, phone numbers, and street addresses come back replaced by uppercase label placeholders like `[PERSON]`, `[EMAIL]`, `[PHONE_NUMBER]`, and `[ADDRESS]`. The raw PII never enters Claude's context window, which is the point: you can paste the masked output into a public tracker or log without leaking customer data. Project-specific identifiers (internal hostnames, IPs, contract numbers, card last-fours) aren't in the model's label set; strip those yourself, or use `extract-entities` with custom labels.

```text theme={null}
Pull all the email addresses and phone numbers out of this:
"Reach Maria at maria@acme.io or 415-555-0188"
```

Claude routes to `extract-pii`, which returns structured JSON grouped by category instead of an in-line mask.

```text theme={null}
Classify this support ticket by sentiment and topic:
"Support replied quickly but the fix didn't work"
```

Claude routes to `classify-structured` and synthesizes a schema with `sentiment` and `topic` axes before calling the skill.

If you ever want a specific skill instead of letting Claude choose, invoke it explicitly with the `/zerogpu-router:<name>` syntax shown below.

#### `/zerogpu-router:signin`

Sign in to ZeroGPU and persist your credentials so every subsequent skill call works without re-prompting. Manual only; not auto-invoked by Claude.

**Wraps** `zerogpu login`.

**Synopsis**

```text theme={null}
/zerogpu-router:signin [--api-key <key>] [--project-id <id>]
```

| Flag | Required | Description |
| - | - | - |
| `--api-key <key>` | optional | API key. Must start with `zgpu-api-`. If omitted, you'll be prompted (masked). |
| `--project-id <id>` | optional | Project ID (UUID v4). If omitted, you'll be prompted. |

On success the API key and Project ID are written to your config file, and `ZEROGPU_API_KEY` is added to your shell profile so other tools can pick it up.

#### `/zerogpu-router:status`

Show your current ZeroGPU sign-in status and the masked API key. Manual only.

**Wraps** `zerogpu status`. Exit code is `0` when signed in, `1` when not. Use this in scripts that need a hard fail before running any inference skill.

#### `/zerogpu-router:chat`

Short, single-turn chat reply for things that don't need Claude-level reasoning or prior conversation context. Use this when you'd otherwise burn Claude tokens on a one-liner.

* **Model:** `LFM2.5-1.2B-Instruct`
* **Wraps:** `zerogpu chat`
* **When Claude auto-invokes:** quick factual answers, one-liners, basic rephrasings where you've signalled "use a small model."

**Synopsis**

```text theme={null}
/zerogpu-router:chat <text> [-i <instructions>]
```

| Name | Required | Description |
| - | - | - |
| `text` | yes | The user message / prompt (quoted). |
| `-i`, `--instructions <instructions>` | optional | System instructions to steer behavior. |

**Example**

```text theme={null}
/zerogpu-router:chat "Explain WebSockets in two sentences." -i "You are a concise technical writer."
```

The skill returns raw assistant text (or pretty-printed JSON if the model returned a structured object).

#### `/zerogpu-router:chat-thinking`

Same shape as `chat`, but the model returns its reasoning trace alongside the answer.

* **Model:** `LFM2.5-1.2B-Thinking`
* **Wraps:** `zerogpu chat_thinking`
* **When Claude auto-invokes:** short logic / math / word-problem questions where step-by-step reasoning is useful or explicitly requested.

**Example**

```text theme={null}
/zerogpu-router:chat-thinking "If a train leaves at 3 PM going 60 mph, when does it cover 150 miles?"
```

#### `/zerogpu-router:summarize`

Condense a longer passage into a short summary. Use this when you need the gist of a report, ticket thread, transcript, or document without spending Claude tokens on the read.

* **Model:** `llama-3.1-8b-instruct-fast`
* **Wraps:** `zerogpu summarize`
* **When Claude auto-invokes:** "summarize this", "give me the gist", "TL;DR this passage", "condense this into a few sentences."

**Synopsis**

```text theme={null}
/zerogpu-router:summarize <text>
```

**Example**

```text theme={null}
/zerogpu-router:summarize "The board met Thursday to review Q3 results. Revenue rose 18% \
year-over-year to $42M, driven mainly by enterprise renewals and a strong launch in the EU \
market. Operating margin slipped to 11% from 14% as headcount grew 30% ahead of the new \
data-center buildout. The CFO flagged rising cloud costs as the top risk for Q4 and proposed \
a hiring freeze on non-engineering roles until margins recover. The board approved the freeze \
and asked for a revised 2025 budget by mid-December."
```

**Illustrative output**

```text theme={null}
Q3 revenue grew 18% YoY to $42M on enterprise renewals and EU growth, but operating margin
fell to 11% due to a 30% headcount increase for the data-center buildout. Citing cloud costs
as the main Q4 risk, the board approved a hiring freeze on non-engineering roles and requested
a revised 2025 budget by mid-December.
```

#### `/zerogpu-router:classify-iab`

Classify text against the **IAB content / audience taxonomy** (standard ad-tech category labels).

* **Model:** `zlm-v1-iab-classify-edge`
* **Wraps:** `zerogpu classify_iab`
* **When Claude auto-invokes:** "what IAB category is this?", "tag this article for ad targeting", "give me the topic taxonomy."

**Example**

```text theme={null}
/zerogpu-router:classify-iab "The Lakers signed a new point guard ahead of the playoffs."
```

**Illustrative output**

```json theme={null}
{
  "categories": [
    { "id": "IAB17-44", "name": "Basketball", "confidence": 0.97 }
  ]
}
```

#### `/zerogpu-router:classify-iab-enriched`

Enriched IAB classification that returns audience categories **plus** topics, keywords, and inferred user intent.

* **Model:** `zlm-v2-iab-classify-edge-enriched`
* **Wraps:** `zerogpu classify_iab_enriched`
* **When Claude auto-invokes:** "give me topics, keywords, and intent", any request asking for richer ad/audience signals than plain IAB labels.

**Example**

```text theme={null}
/zerogpu-router:classify-iab-enriched "Compare the Tesla Model Y and the Hyundai Ioniq 5 for a family of four."
```

**Illustrative output**

```json theme={null}
{
  "categories": [{ "id": "IAB2-1", "name": "Auto Buyers", "confidence": 0.92 }],
  "topics": ["electric vehicles", "family cars"],
  "keywords": ["Tesla Model Y", "Hyundai Ioniq 5"],
  "intent": "comparison-shopping"
}
```

#### `/zerogpu-router:classify-zero-shot`

Zero-shot classification against an arbitrary list of candidate labels you supply at call time. The model picks the single best fit and returns scores for all candidates.

* **Model:** `deberta-v3-small`
* **Wraps:** `zerogpu classify_zero_shot`
* **When Claude auto-invokes:** "is this positive, negative, or neutral?", "tag this as bug, feature, or question."

**Synopsis**

```text theme={null}
/zerogpu-router:classify-zero-shot <text> (-l <label>...) | (--labels a,b,c)
```

| Name | Required | Description |
| - | - | - |
| `text` | yes | Text to classify (quoted). |
| `-l <label>` | one of `-l` / `--labels` | A single label. Repeatable. |
| `--labels <a,b,c>` | one of `-l` / `--labels` | Comma-separated label list. |

**Example**

```text theme={null}
/zerogpu-router:classify-zero-shot "I love how fast this laptop boots up." -l positive -l negative -l neutral
```

**Illustrative output**

```json theme={null}
{ "label": "positive", "scores": { "positive": 0.94, "neutral": 0.04, "negative": 0.02 } }
```

#### `/zerogpu-router:classify-structured`

Schema-driven, multi-axis classification. You define each category and its allowed labels, and the model returns one chosen label per category.

* **Model:** `gliner2.5-base-v1`
* **Wraps:** `zerogpu classify_structured`
* **When Claude auto-invokes:** "classify by sentiment and topic", any request that names multiple classification dimensions with explicit label sets.

**Synopsis**

```text theme={null}
/zerogpu-router:classify-structured <text> -s '<json schema>'
```

| Name | Required | Description |
| - | - | - |
| `text` | yes | Text to classify. |
| `-s`, `--schema <json>` | yes | JSON object mapping each category to its allowed labels. |

**Example**

```text theme={null}
/zerogpu-router:classify-structured "Support replied quickly but the fix didn't work." \
  -s '{"sentiment":["positive","negative","neutral"],"topic":["support","billing","product"]}'
```

**Illustrative output**

```json theme={null}
{ "sentiment": "negative", "topic": "support" }
```

#### `/zerogpu-router:extract-entities`

Custom-label named-entity recognition. You define the entity labels; the model finds matching spans with confidence scores.

* **Model:** `gliner2.5-base-v1`
* **Wraps:** `zerogpu extract_entities`
* **When Claude auto-invokes:** "extract all people, organizations, and locations from this", "find every product mention."

**Synopsis**

```text theme={null}
/zerogpu-router:extract-entities <text> (-l <label>... | --labels a,b,c) [-t <0..1>]
```

| Name | Required | Default | Description |
| - | - | - | - |
| `text` | yes | - | Source text. |
| `-l <label>` / `--labels <a,b,c>` | yes (one) | - | Entity labels to extract. |
| `-t`, `--threshold <number>` | optional | `0.3` | Minimum confidence in `[0, 1]`. |

**Example**

```text theme={null}
/zerogpu-router:extract-entities "Apple CEO Tim Cook met with Sundar Pichai in Cupertino on Monday." \
  --labels person,organization,location -t 0.4
```

**Illustrative output**

```json theme={null}
[
  { "label": "organization", "text": "Apple",         "score": 0.98 },
  { "label": "person",       "text": "Tim Cook",      "score": 0.97 },
  { "label": "person",       "text": "Sundar Pichai", "score": 0.96 },
  { "label": "location",     "text": "Cupertino",     "score": 0.91 }
]
```

#### `/zerogpu-router:extract-pii`

Extract personally identifiable information entities, grouped by category, **without modifying the source text**. Use when you need structured data about PII (for redaction policies, audits, or downstream tooling) rather than a masked version.

* **Model:** `gliner2.5-multi-v1`
* **Wraps:** `zerogpu extract_pii`
* **When Claude auto-invokes:** "find all PII", "what personal info is in this?", "list emails/phones/names."

**Synopsis**

```text theme={null}
/zerogpu-router:extract-pii <text> [-t <threshold>] [(-c | --categories) <list>]
```

| Name | Required | Default | Description |
| - | - | - | - |
| `text` | yes | - | Source text. |
| `-t`, `--threshold <number>` | optional | `0.5` | Minimum confidence. |
| `-c`, `--categories <list>` | optional | `identity,contact` | Comma-separated. Other values: `financial`, `medical`, `credentials`. |

**Example**

```text theme={null}
/zerogpu-router:extract-pii "Contact Jane Doe at jane@example.com or +1 (415) 555-1212." -t 0.6 -c identity,contact,financial
```

**Illustrative output**

```json theme={null}
[
  { "category": "identity", "label": "person", "text": "Jane Doe",          "score": 0.96 },
  { "category": "contact",  "label": "email",  "text": "jane@example.com",  "score": 0.99 },
  { "category": "contact",  "label": "phone",  "text": "+1 (415) 555-1212", "score": 0.95 }
]
```

If you want to **mask** PII inline rather than extract it, use `/zerogpu-router:redact-pii`.

#### `/zerogpu-router:redact-pii`

Detect PII and replace each span in-line with a `[LABEL]` placeholder. Use this before sharing or logging sensitive text, or before forwarding user input to another LLM that you don't want to expose raw PII to.

* **Model:** `gliner2.5-multi-v1` (with `mask: "label"`)
* **Wraps:** `zerogpu redact_pii`
* **When Claude auto-invokes:** "redact", "scrub", "mask", "anonymize", or "sanitize this for sharing."

**Example**

```text theme={null}
/zerogpu-router:redact-pii "Email John Smith at john@acme.com about invoice 12345."
```

**Output**

```text theme={null}
Email [PERSON] at [EMAIL] about invoice 12345.
```

Note that `12345` is not masked: only spans the model recognizes as PII are replaced. Domain-specific identifiers (account numbers, internal ticket IDs) should be handled by your own redaction layer or by `extract-entities` with custom labels.

#### `/zerogpu-router:extract-json`

Pull specific named fields out of free text into a structured JSON object, defined by a per-field schema. Each field is declared as `name::type::description`.

* **Model:** `gliner2.5-base-v1`
* **Wraps:** `zerogpu extract_json`
* **When Claude auto-invokes:** "extract the contact info as JSON", "parse this invoice", "pull these fields out."

**Synopsis**

```text theme={null}
/zerogpu-router:extract-json <text> -s '<json schema>'
```

**Example**

```text theme={null}
/zerogpu-router:extract-json "Reach Maria Lopez at maria.lopez@acme.io or 415-555-0188." \
  -s '{"contact":["name::str::Full name","email::str::Email address","phone::str::Phone number"]}'
```

**Illustrative output**

```json theme={null}
{
  "contact": {
    "name": "Maria Lopez",
    "email": "maria.lopez@acme.io",
    "phone": "415-555-0188"
  }
}
```

#### Skills reference table

Every skill at a glance.

| Skill | Purpose | Example |
| - | - | - |
| `/zerogpu-router:signin` | Sign in and persist API key + Project ID (manual only) | `/zerogpu-router:signin` |
| `/zerogpu-router:status` | Show current sign-in status (manual only) | `/zerogpu-router:status` |
| `/zerogpu-router:chat <text>` | Short chat reply via `LFM2.5-1.2B-Instruct` | `/zerogpu-router:chat "Explain WebSockets in two sentences."` |
| `/zerogpu-router:chat-thinking <text>` | Chat with the Thinking variant (returns reasoning) | `/zerogpu-router:chat-thinking "If a train leaves at 3 PM going 60 mph..."` |
| `/zerogpu-router:summarize <text>` | Condense a longer passage into a short summary | `/zerogpu-router:summarize "The board met Thursday to review Q3 results..."` |
| `/zerogpu-router:classify-iab <text>` | IAB taxonomy classification | `/zerogpu-router:classify-iab "The Lakers signed a new point guard."` |
| `/zerogpu-router:classify-iab-enriched <text>` | IAB + topics/keywords/intent | `/zerogpu-router:classify-iab-enriched "Compare the Tesla Model Y..."` |
| `/zerogpu-router:classify-zero-shot <text> -l ...` | Zero-shot against custom labels | `/zerogpu-router:classify-zero-shot "fast laptop" -l positive -l negative` |
| `/zerogpu-router:classify-structured <text> -s '...'` | Schema-based multi-axis classification | `/zerogpu-router:classify-structured "ticket text" -s '{"sentiment":[...]}'` |
| `/zerogpu-router:extract-entities <text> -l ...` | Custom-label NER | `/zerogpu-router:extract-entities "Tim Cook met Sundar Pichai..." -l person -l location` |
| `/zerogpu-router:extract-pii <text>` | Extract PII entities (returns JSON) | `/zerogpu-router:extract-pii "Contact Jane at jane@example.com"` |
| `/zerogpu-router:redact-pii <text>` | Mask PII in-line with `[LABEL]` placeholders | `/zerogpu-router:redact-pii "Email John at john@acme.com"` |
| `/zerogpu-router:extract-json <text> -s '...'` | Schema-driven JSON extraction | `/zerogpu-router:extract-json "..." -s '{"contact":["name::str::Full name"]}'` |

For full flag reference on any command, run `zerogpu <command> --help` outside the Claude Code session.

#### Patterns and recipes

**Sanitize before Claude sees raw input.** Pipe untrusted text through `redact-pii` first when you don't want personal data captured in Claude's transcript or sent to downstream LLMs. Combine with `extract-pii` if you also need an audit log of what was masked.

**Cheap router in front of Claude.** Use `classify-zero-shot` or `classify-structured` to triage an incoming message (bug / feature / question, urgent / normal, in-scope / out-of-scope) and only escalate the hard cases to Claude itself. The classifier call costs orders of magnitude less than a Claude turn.

**Structured extraction over free-form parsing.** When you have semi-structured text (signatures, invoices, contact blocks), prefer `extract-json` over asking Claude to "parse this into JSON." It's deterministic on the schema, faster, and cheaper. Keep field descriptions short and specific - the description is what the model uses to find the span.

**Confidence thresholds.** For NER and PII extraction, the default thresholds (`0.3` and `0.5` respectively) are tuned for recall. Raise `-t` to `0.6` or higher when you need precision (e.g. compliance-grade redaction lists); lower it when you'd rather over-extract and filter downstream.

## Troubleshooting

**`zerogpu: command not found`** - the CLI isn't installed or isn't on your `PATH`. Run `npm install -g zerogpu-cli` and restart your shell. If you use a Node version manager (nvm, fnm, volta), make sure the shell that launched Claude Code has the same Node version active.

**Skill returns "You're not signed in yet."** - no credentials on disk. Run `/zerogpu-router:signin` inside Claude Code, or `zerogpu login` in your terminal. Check `/zerogpu-router:status` to confirm.

**`/zerogpu-router:*` skills don't appear in `/help`** - the plugin isn't enabled. Run `/plugin` to view installed plugins, enable `zerogpu-router`, then `/reload-plugins`. If it's not listed at all, re-run `/plugin marketplace add zerogpu/zerogpu-router` followed by `/plugin install zerogpu-router@zerogpu`.

**`Request failed with status 401`** - your API key is missing, revoked, or mistyped. Rotate the key in the [dashboard](https://platform.zerogpu.ai/dashboard) and re-run `/zerogpu-router:signin`. Keys must start with `zgpu-api-`.

**`Request failed with status 403`** - the key is valid but doesn't have access to the project, or the project doesn't have access to the requested model. Confirm your Project ID matches the project that owns the key.

**`Request failed with status 429`** - you're being rate-limited. Back off and retry with exponential delay, or switch heavy workloads to the [Batch API](/docs/batch), which has separate quotas tuned for bulk jobs.

**Wrong skill auto-invoked for a request.** Claude picks based on phrasing. If you want a specific skill, call it explicitly with `/zerogpu-router:<name>`. If a skill keeps getting picked when you don't want it, rephrase to remove the trigger words ("redact", "classify", "extract") or invoke the right one by name.

**Schema parsing errors on `classify-structured` / `extract-json`.** The `-s` flag expects a single-quoted JSON string. On Windows PowerShell, escape inner double quotes or use a here-string. Run the CLI directly (`zerogpu classify_structured ... -s '...'`) to isolate quoting issues from Claude Code.

**Empty or low-confidence results.** Lower `-t` to surface more candidates, or check that the label set you supplied matches the language of the source text (the underlying models are English-tuned for most label sets). For very short inputs (one or two words), expect lower confidence across the board.

**`/reload-plugins` doesn't pick up a new CLI version.** Plugin reload only touches Claude Code state; it doesn't reinstall the CLI. Run `npm install -g zerogpu-cli@latest` in your terminal, then `zerogpu --version` to confirm.

## Conclusion

The `zerogpu-router` plugin turns ZeroGPU's nano language models into first-class Claude Code skills, so Claude can hand off classification, extraction, and short chat tasks to a cheaper, faster model without you leaving the session. It's a fast way to keep raw PII out of Claude's context, cut token spend on well-defined NLP work, and prototype routing patterns you can later promote to production.

<CardGroup>
  <Card title="Model Catalog" icon="layer-group" href="/docs/model-catalog">
    Browse every model the plugin can route to.
  </Card>

  <Card title="API Reference" icon="code" href="/api-reference/responses">
    Explore the full OpenAI-compatible API surface.
  </Card>

  <Card title="Cookbook" icon="book" href="/cookbook/index">
    Worked examples for classification, extraction, and batch jobs.
  </Card>

  <Card title="Join Discord" icon="discord" href="https://discord.gg/58XZuxKyq6">
    Ask questions and share what you're building.
  </Card>
</CardGroup>


# Claude Skill
Source: https://docs.zerogpu.ai/integrations/claude-skill

Upload the ZeroGPU Skill to Claude (Desktop or Web) so Claude follows ZeroGPU best practices when you ask it to classify, extract, redact, or summarize content.

The Claude Skill is a single `SKILL.md` file you upload to Claude (Desktop or Web). Skills are reusable instruction packs that Claude loads on demand: once the ZeroGPU Skill is installed, Claude recognizes when a request is a repeatable, high-volume inference task and follows ZeroGPU's documented patterns instead of improvising. It teaches Claude the right endpoint, the real model catalog, the required authentication, and the rule never to invent results: every answer comes from an actual API call.

ZeroGPU is an ultra-fast, compute-efficient inference provider for apps and agents. We run purpose-built small and nano language models across an edge-powered network for the high-volume, purpose-specific tasks your app or agent runs constantly. Plug in our OpenAI-compatible API and you're live - zero GPU infrastructure, serverless, auto-scaling by default.

## Overview

This guide shows how to add the ZeroGPU Skill to Claude (Desktop or Web) and what changes once it's active. The Skill doesn't replace Claude; it steers Claude toward ZeroGPU for the well-defined tasks small models do best - classification, PII detection and redaction, entity and structured extraction, and summarization. With the Skill installed, Claude routes those tasks to the correct ZeroGPU model through the OpenAI-compatible API, requires your API key before running anything, and returns only output produced by a live API call. By the end you'll know how to install it, the prompts that trigger it, and exactly how Claude behaves when it does.

## Video walkthrough

<iframe title="YouTube video player" />

## Quickstart

### Prerequisites

* The [Claude Desktop](https://claude.ai/download) app, or access to [Claude Web](https://claude.ai).
* A ZeroGPU [API key](https://platform.zerogpu.ai/dashboard).
* A model ID from the [model catalog](/docs/model-catalog) if you want to call a specific model by name.

### Get your ZeroGPU API key

1. Sign in to the [ZeroGPU dashboard](https://platform.zerogpu.ai/dashboard).
2. Open **API Keys** and click **Create key**.
3. Copy the key (starts with `zgpu-api-`).

Keep it handy. Claude will ask for it before it runs any ZeroGPU inference.

### Install the Skill

First, download the Skill from [https://zerogpu.ai/SKILL.md](https://zerogpu.ai/SKILL.md). Then follow whichever path matches how you use Claude.

#### Claude Desktop

<Steps>
  <Step title="Open Claude Desktop">
    Launch the Claude Desktop app.
  </Step>

  <Step title="Go to Settings -> Skills">
    Open **Settings -> Skills**.
  </Step>

  <Step title="Upload the file">
    Add a new Skill and upload the `SKILL.md` file you downloaded.
  </Step>
</Steps>

#### Claude Web (file upload)

<Steps>
  <Step title="Open Claude Web">
    Go to [claude.ai](https://claude.ai) and start (or open) a conversation.
  </Step>

  <Step title="Attach SKILL.md">
    Click the attachment button and upload the `SKILL.md` file you downloaded.
  </Step>

  <Step title="Ask Claude to follow it">
    Tell Claude to follow the attached Skill, then make your request. Claude reads `SKILL.md` from the conversation and applies ZeroGPU's patterns.
  </Step>
</Steps>

### Your first request

With the Skill installed, ask Claude to do a task it recognizes:

```text theme={null}
Redact the PII in this text using ZeroGPU:
"Email John Smith at john@acme.com about invoice 12345."
```

Claude routes the request to ZeroGPU's PII model (`gliner2.5-multi-v1`) through the OpenAI-compatible API. If your API key isn't available yet, Claude asks for it first. Once authenticated, it makes the call and returns the model's actual output:

```text theme={null}
Email [PERSON] at [EMAIL] about invoice 12345.
```

Note that `12345` is not masked: only spans the model recognizes as PII are replaced.

## Usage

The Skill activates whenever a prompt looks like a repeatable inference task. You don't call a command; you describe what you want, and Claude picks the right ZeroGPU model. Two behaviors hold across every example below:

* **Claude requires your API key before executing.** No inference runs until your ZeroGPU API key is available. If it's missing, Claude pauses and asks for it rather than proceeding.
* **Claude never fabricates results.** Every result comes from a live API call to the real model. With the Skill active, Claude will not guess a classification, invent extracted fields, or mock a response before the call runs. If it can't make the call, it tells you why instead of making something up.

Each task below shows a triggering prompt, the ZeroGPU model Claude routes to, and the shape of the response.

### Summarize a long passage

Condense a report, transcript, or thread without spending Claude tokens on the full read.

* **Model:** `llama-3.1-8b-instruct-fast`
* **Triggers on:** "summarize this", "give me the gist", "TL;DR this passage."

```text theme={null}
Summarize this with ZeroGPU:
"The board met Thursday to review Q3 results. Revenue rose 18% year-over-year to
$42M, driven mainly by enterprise renewals and a strong launch in the EU market.
Operating margin slipped to 11% from 14% as headcount grew 30% ahead of the new
data-center buildout. The CFO flagged rising cloud costs as the top risk for Q4
and proposed a hiring freeze on non-engineering roles until margins recover."
```

**Example output (returned after API call)**

```text theme={null}
Q3 revenue grew 18% YoY to $42M on enterprise renewals and EU growth, but operating
margin fell to 11% as headcount rose 30% for the data-center buildout. Citing cloud
costs as the main Q4 risk, the CFO proposed a hiring freeze on non-engineering roles.
```

### Classify against your own labels

Zero-shot classification against a candidate label list you supply in the prompt.

* **Model:** `deberta-v3-small`
* **Triggers on:** "is this positive, negative, or neutral?", "tag this as bug, feature, or question."

```text theme={null}
Classify this with ZeroGPU as positive, negative, or neutral:
"I love how fast this laptop boots up."
```

**Example output (returned after API call)**

```json theme={null}
{ "label": "positive", "scores": { "positive": 0.94, "neutral": 0.04, "negative": 0.02 } }
```

For multi-axis classification (for example sentiment **and** topic at once), Claude routes to `gliner2.5-base-v1` and returns one chosen label per axis:

```text theme={null}
Classify this support ticket by sentiment and topic with ZeroGPU:
"Support replied quickly but the fix didn't work."
```

```json theme={null}
{ "sentiment": "negative", "topic": "support" }
```

### Classify ad-tech / contextual categories

Standard IAB content and audience taxonomy labels.

* **Model:** `zlm-v1-iab-classify-edge` (use the `-enriched` variant for topics, keywords, and intent)
* **Triggers on:** "what IAB category is this?", "tag this article for ad targeting."

```text theme={null}
What IAB category is this, using ZeroGPU?
"The Lakers signed a new point guard ahead of the playoffs."
```

**Example output (returned after API call)**

```json theme={null}
{
  "categories": [
    { "id": "IAB17-44", "name": "Basketball", "confidence": 0.97 }
  ]
}
```

### Classify a domain by name

Map a bare domain to IAB categories, topics, and intent without fetching the page - the input is just the hostname, which keeps the payload tiny for bidstream and contextual targeting.

* **Model:** `zlm-v1-iab-domain-classifier`
* **Triggers on:** "what categories is this domain?", "classify indeed.com for targeting."

```text theme={null}
Classify this domain with ZeroGPU:
"indeed.com"
```

**Example output (returned after API call)**

```json theme={null}
{
  "domain": "indeed.com",
  "categories": [
    { "id": "IAB4", "name": "Careers", "confidence": 0.96 }
  ]
}
```

### Moderate content for safety

Screen text for unsafe or policy-violating content and get OpenAI-style moderation signals back - useful as a lightweight safety layer in front of your app or another model. See the [moderation benchmark](https://zerogpu.ai/benchmarks/moderation-edge) for accuracy and latency results.

* **Model:** `zlm-v1-moderation-edge`
* **Triggers on:** "moderate this", "is this safe?", "flag any harmful content."

```text theme={null}
Moderate this with ZeroGPU:
"I am so angry at this person that I want to hurt them. They are worthless and
should be scared of what I might do next."
```

**Example output (returned after API call)**

```json theme={null}
{
  "model": "zlm-v1-moderation-edge",
  "results": [
    {
      "flagged": true,
      "unsafe_score": 1,
      "categories": {
        "hate": false,
        "hate/threatening": false,
        "harassment": true,
        "harassment/threatening": true,
        "self-harm": false,
        "self-harm/intent": false,
        "self-harm/instructions": false,
        "sexual": false,
        "sexual/minors": false,
        "violence": true,
        "violence/graphic": false,
        "illicit": true,
        "illicit/violent": true
      },
      "category_scores": {
        "hate": 0.21574,
        "hate/threatening": 0.168273,
        "harassment": 0.82545,
        "harassment/threatening": 0.957703,
        "self-harm": 0.015232,
        "self-harm/intent": 0.017179,
        "self-harm/instructions": 0.01807,
        "sexual": 0.035671,
        "sexual/minors": 0.094207,
        "violence": 0.665897,
        "violence/graphic": 0.198346,
        "illicit": 0.732167,
        "illicit/violent": 0.780693
      }
    }
  ]
}
```

### Detect PII

Find personally identifiable information and return it as structured data, without altering the source text.

* **Model:** `gliner2.5-multi-v1`
* **Triggers on:** "find all PII", "what personal info is in this?", "detect PII."

```text theme={null}
Detect PII in this text with ZeroGPU:
"Contact Jane Doe at jane@example.com or +1 (415) 555-1212."
```

**Example output (returned after API call)**

```json theme={null}
[
  { "category": "identity", "label": "person", "text": "Jane Doe",          "score": 0.96 },
  { "category": "contact",  "label": "email",  "text": "jane@example.com",  "score": 0.99 },
  { "category": "contact",  "label": "phone",  "text": "+1 (415) 555-1212", "score": 0.95 }
]
```

To **mask** PII inline instead of listing it, ask Claude to redact: the same model returns `[LABEL]` placeholders, as shown in the Quickstart.

### Extract named entities

Custom-label named-entity recognition: you name the entity types, the model finds the spans.

* **Model:** `gliner2.5-base-v1`
* **Triggers on:** "extract all people, organizations, and locations", "find every product mention."

```text theme={null}
Extract the people, organizations, and locations from this with ZeroGPU:
"Apple CEO Tim Cook met with Sundar Pichai in Cupertino on Monday."
```

**Example output (returned after API call)**

```json theme={null}
[
  { "label": "organization", "text": "Apple",         "score": 0.98 },
  { "label": "person",       "text": "Tim Cook",      "score": 0.97 },
  { "label": "person",       "text": "Sundar Pichai", "score": 0.96 },
  { "label": "location",     "text": "Cupertino",     "score": 0.91 }
]
```

### Extract fields into JSON

Pull specific named fields out of free text into a structured object, defined by a schema Claude builds from your request.

* **Model:** `gliner2.5-base-v1`
* **Triggers on:** "extract the contact info as JSON", "parse this into fields."

```text theme={null}
Extract the name, email, and phone as JSON with ZeroGPU:
"Reach Maria Lopez at maria.lopez@acme.io or 415-555-0188."
```

**Example output (returned after API call)**

```json theme={null}
{
  "contact": {
    "name": "Maria Lopez",
    "email": "maria.lopez@acme.io",
    "phone": "415-555-0188"
  }
}
```

### Translate or reply conversationally

A compact instruct model for short conversational replies, translation, and intent detection - pass an optional system instruction to set its role.

* **Model:** `LFM2.5-1.2B-Instruct`
* **Triggers on:** "translate this", "write a short reply", "draft an in-app message."

```text theme={null}
Reply to this with ZeroGPU, as a friendly fitness-app assistant:
"I just completed a 5K run in 28 minutes. Give me a short motivational follow-up
message."
```

**Example output (returned after API call)**

```text theme={null}
Nice work - a 28-minute 5K is a strong pace! Keep that momentum going and see if
you can shave off a few seconds on your next run. You've got this.
```

### Reason through a decision

A small reasoning model that thinks step by step before committing - useful for triage, routing, and multi-step decisions where you want to see the why, not just the answer.

* **Model:** `LFM2.5-1.2B-Thinking`
* **Triggers on:** "triage this ticket", "reason through this then decide", "route this request."

```text theme={null}
Triage this support ticket with ZeroGPU - assign one team (Billing, Technical,
Account, General) and one priority (Low, Medium, High):
"Our whole team got locked out of the dashboard this morning after the payment
failed. We have a product demo with a customer in 2 hours and cannot log in.
Card on file should be valid."
```

**Example output (returned after API call)**

```text theme={null}
The lockout follows a failed payment, so the root cause is billing rather than a
technical fault. A customer demo in 2 hours makes this time-critical.

Billing | High
```

### Patterns and recipes

**Sanitize before Claude keeps raw input.** Ask Claude to redact PII first when you don't want personal data captured in the conversation transcript or forwarded downstream. The PII spans never need to stay in plain text.

**Cheap router in front of Claude.** Use a zero-shot or structured classification to triage an incoming message (bug / feature / question, urgent / normal) and only escalate the hard cases to Claude's own reasoning. The classifier call costs a fraction of a full Claude turn.

**Structured extraction over free-form parsing.** For semi-structured text (signatures, invoices, contact blocks), prefer JSON extraction over asking Claude to "parse this into JSON." It's deterministic on the schema, faster, and cheaper.

### Task reference

| Task | Model | Example prompt |
| - | - | - |
| Summarize | `llama-3.1-8b-instruct-fast` | "Summarize this with ZeroGPU: ..." |
| Zero-shot classify | `deberta-v3-small` | "Classify this as positive, negative, or neutral with ZeroGPU: ..." |
| Multi-axis classify | `gliner2.5-base-v1` | "Classify by sentiment and topic with ZeroGPU: ..." |
| IAB classify | `zlm-v1-iab-classify-edge` | "What IAB category is this, using ZeroGPU? ..." |
| Domain classify | `zlm-v1-iab-domain-classifier` | "Classify this domain with ZeroGPU: ..." |
| Moderate content | `zlm-v1-moderation-edge` | "Moderate this with ZeroGPU: ..." |
| Detect PII | `gliner2.5-multi-v1` | "Detect PII in this text with ZeroGPU: ..." |
| Redact PII | `gliner2.5-multi-v1` | "Redact the PII in this with ZeroGPU: ..." |
| Extract entities | `gliner2.5-base-v1` | "Extract the people and organizations with ZeroGPU: ..." |
| Extract JSON | `gliner2.5-base-v1` | "Extract the name, email, and phone as JSON with ZeroGPU: ..." |
| Translate / reply | `LFM2.5-1.2B-Instruct` | "Reply to this with ZeroGPU: ..." |
| Reason / triage | `LFM2.5-1.2B-Thinking` | "Triage this ticket with ZeroGPU: ..." |

## Troubleshooting

**Claude doesn't route the task to ZeroGPU** - the Skill activates on intent, so a request that reads like general chat may get answered directly. Add "with ZeroGPU" to the prompt, or confirm the Skill is installed and enabled (Claude Desktop: **Settings -> Skills**; Claude Web: re-attach `SKILL.md` and tell Claude to follow it).

**Claude keeps asking for your API key** - no inference runs until the key is available. In Claude Web there's no settings store, so the key lives only in the current conversation; paste it again as its own message when Claude prompts. In Claude Desktop, re-add the Skill if the key didn't persist across sessions.

**"Invalid API key" or the call is rejected** - the key must start with `zgpu-api-`. You've likely pasted a truncated key or a different credential; copy it again from the [dashboard](https://platform.zerogpu.ai/dashboard).

**Network or "couldn't reach the API" errors** - the machine running Claude must be able to reach `api.zerogpu.ai`. Corporate proxies, VPNs, and firewalls are the usual blockers; allowlist the host or switch networks.

**Claude references a model, parameter, or price that doesn't exist** - this shouldn't happen with the Skill active, since it pins Claude to the real [model catalog](/docs/model-catalog). If it does, the Skill probably isn't loaded - re-download `SKILL.md` from [zerogpu.ai/SKILL.md](https://zerogpu.ai/SKILL.md), re-upload it, and confirm Claude says it's following it.

**Claude picks the wrong model for a task** - routing is driven by intent. Phrase the request with the task verb ("summarize", "redact PII", "classify by sentiment and topic", "moderate this") or name the model from the [model catalog](/docs/model-catalog) directly.

**Empty or low-confidence results** - very short inputs (a word or two) score low across the board. Give the model more context, and check that the source-text language matches your labels - most label sets are English-tuned.

**Rate limited, or a 429 / 5xx error** - back off and retry after a short delay. For bulk or offline workloads, move the job to the [Batch API](/docs/batch/index), which has separate quotas tuned for high-volume processing.

## Conclusion

The ZeroGPU Skill turns Claude into a smart router for the repeatable work small models do best - classification, IAB and domain tagging, PII detection and redaction, entity and JSON extraction, summarization, moderation, and lightweight reasoning - so Claude's own reasoning is spent only where it's needed and every result comes from a real API call. Install it once, keep your API key handy, and describe the task in plain language. For anything beyond interactive use, read your key from environment variables or a secrets manager rather than pasting it into a chat - see [Production patterns](/docs/production-patterns).

<CardGroup>
  <Card title="Model Catalog" icon="layer-group" href="/docs/model-catalog">
    Browse every model the Skill routes to and pick the best fit.
  </Card>

  <Card title="API Reference" icon="code" href="/api-reference/responses">
    Explore the full OpenAI-compatible API surface.
  </Card>

  <Card title="Cookbook" icon="book" href="/cookbook/index">
    Worked examples for classification, extraction, and batch jobs.
  </Card>

  <Card title="Join Discord" icon="discord" href="https://discord.gg/Ad5KZvAyky">
    Ask questions and share what you're building.
  </Card>
</CardGroup>


# Hermes Agent
Source: https://docs.zerogpu.ai/integrations/hermes

Run Hermes Agent on ZeroGPU: set it as the model provider and connect the ZeroGPU MCP server for fourteen ready-made tools.

Hermes Agent is an open-source agent from Nous Research that runs in your terminal, a TUI, a desktop app, or a self-hosted web UI. It ships with file, terminal, web, and memory toolsets, a skills system built on the `agentskills.io` standard, and a built-in MCP client that discovers and calls tools from any external MCP server at startup. Because Hermes lets you point it at any OpenAI-compatible endpoint, you can choose which model does the agent's reasoning and which models do everything else.

ZeroGPU is an ultra-fast, compute-efficient inference provider for apps and agents. We run purpose-built small and nano language models across an edge-powered network for the high-volume, purpose-specific tasks your app or agent runs constantly. Plug in our OpenAI-compatible API and you're live - zero GPU infrastructure, serverless, auto-scaling by default.

## Overview

This guide wires ZeroGPU into Hermes Agent in two places. First as the **model provider**, so the agent's own reasoning runs on ZeroGPU instead of a frontier model. Second as an **MCP server**, so the agent gains fourteen task-specific tools for classification, extraction, PII handling, moderation, embeddings, and summarization. By the end, your Hermes agent will reason on ZeroGPU and hand every repeatable NLP step to a purpose-built small model, with a per-call cost comparison returned in every tool response.

## Cookbook

For a runnable, real-world example, see [Raise Your Seed Round with a Hermes Agent](/cookbook/hermes-investor-outreach). It walks through an agent that researches investors, ranks them against your one-liner, writes a personalized email for each, and saves them as Gmail drafts, with every step except the drafting running on a ZeroGPU nano model.

<Note>
  For more worked examples, browse the [cookbook index](/cookbook/index) for
  end-to-end walkthroughs of classification, extraction, and PII workflows on
  ZeroGPU's small models.
</Note>

## Video walkthrough

Video walkthrough coming soon.

## Quickstart

### Prerequisites

* Hermes Agent installed and running. See the [Hermes Agent docs](https://hermes-agent.nousresearch.com/docs/getting-started/quickstart).
* A ZeroGPU API key from the [dashboard](https://platform.zerogpu.ai/dashboard).
* A model with at least 64,000 tokens of context. Hermes rejects anything smaller at startup. See the [model catalog](/docs/model-catalog) for the options.

### Get your ZeroGPU API key

Log in to the [ZeroGPU dashboard](https://platform.zerogpu.ai/dashboard), open the API keys section, and copy your key. It starts with `zgpu-api-`.

### Set ZeroGPU as the model provider

Hermes reads secrets from `.env` and settings from `config.yaml` in its home directory. Run `hermes config path` to find yours. Set everything from the CLI:

```bash theme={null}
hermes config set ZEROGPU_API_KEY 'zgpu-api-...'

hermes config set providers.ZeroGPU.name 'ZeroGPU'
hermes config set providers.ZeroGPU.api 'https://api.zerogpu.ai/v1'
hermes config set providers.ZeroGPU.key_env 'ZEROGPU_API_KEY'
hermes config set providers.ZeroGPU.transport 'chat_completions'
hermes config set providers.ZeroGPU.default_model 'deepseek-v4.1-flash'
hermes config set providers.ZeroGPU.context_length 1048576

hermes config set model.default 'deepseek-v4.1-flash'
hermes config set model.provider 'custom:ZeroGPU'
```

The equivalent `config.yaml` block, if you prefer editing the file:

```yaml theme={null}
providers:
  ZeroGPU:
    name: ZeroGPU
    api: https://api.zerogpu.ai/v1
    key_env: ZEROGPU_API_KEY
    transport: chat_completions
    default_model: deepseek-v4.1-flash
    context_length: 1048576

model:
  default: deepseek-v4.1-flash
  provider: "custom:ZeroGPU"
```

The `custom:` prefix is required. It tells Hermes to look under `providers:` rather than in its built-in provider list.

### Connect the ZeroGPU MCP server

```bash theme={null}
hermes mcp add zerogpu --url "https://mcp.zerogpu.ai/mcp?apiKey=YOUR_ZEROGPU_KEY" --connect-timeout 60
```

The key can travel in the URL as `?apiKey=` or as an `x-api-key` header. The header keeps it out of logs and out of `hermes mcp list` output:

```bash theme={null}
hermes mcp add zerogpu --url "https://mcp.zerogpu.ai/mcp" --auth header
```

### Your first request

```bash theme={null}
hermes mcp test zerogpu
hermes -z "Reply with exactly: ZEROGPU BRAIN OK"
```

The first prints fourteen tools. The second confirms the agent's own model is answering from ZeroGPU. Then ask for real work:

```bash theme={null}
hermes -z "Use the ZeroGPU tools to pull the people, organizations, and funding rounds out of this: 'Index Ventures led a 40M dollar Series A last Tuesday, with partner Jan Hammer taking a board seat.' Show the entities and the savings block."
```

```json theme={null}
{
  "entities": {
    "person": ["Jan Hammer"],
    "organization": ["Index Ventures"],
    "funding round": ["40M dollar Series A"]
  },
  "model": "gliner2.5-base-v1",
  "usage": { "input_tokens": 33, "output_tokens": 52 },
  "savings": {
    "zerogpu_cost_usd": 0.00000326,
    "baseline_cost_usd": 0.000879,
    "savings_usd": 0.00087574,
    "price_table_version": "2026-08-21"
  }
}
```

## Usage

### Choosing the agent's model

Any ZeroGPU text-generation model with at least 64K context works as the agent's brain. Set it with `providers.ZeroGPU.default_model` and `model.default`.

| Model | Context | Input / 1M | Output / 1M | Use it when |
| - | - | - | - | - |
| `deepseek-v4.1-flash` | 1,048,576 | \$0.30 | \$1.20 | Default. Agentic work with tool calling. |
| `gpt-oss-120b` | 131,072 | \$0.15 | \$0.60 | Reasoning and function calling at mid cost. |
| `qwen3-30b-a3b-fp8` | 32,768 | \$0.10 | \$0.45 | Below the 64K minimum. Not usable as the agent model. |

Switch mid-session with `/model custom:ZeroGPU:gpt-oss-120b`.

### The fourteen MCP tools

Hermes registers MCP tools as `mcp__<server>__<tool>`. Because this server's tools are already named `zerogpu_*`, the full name repeats the prefix, for example `mcp__zerogpu__zerogpu_summarize`. Run `hermes mcp list` to see the exact names on your install.

| Tool | Model | Required arguments |
| - | - | - |
| `zerogpu_chat` | `LFM2.5-1.2B-Instruct`, or `LFM2.5-1.2B-Thinking` with `thinking` | `messages` |
| `zerogpu_summarize` | `llama-3.1-8b-instruct-fast` | `text` |
| `zerogpu_classify_iab` | `zlm-v1-iab-classify-edge` and variants | `text` |
| `zerogpu_classify_domain` | `zlm-v1-iab-domain-classifier` | `domain` |
| `zerogpu_classify_zero_shot` | `deberta-v3-small` | `text`, `labels` |
| `zerogpu_classify_structured` | `gliner2.5-base-v1` | `text`, `schema` |
| `zerogpu_extract_entities` | `gliner2.5-base-v1` | `text`, `labels` |
| `zerogpu_extract_json` | `gliner2.5-base-v1` | `text`, `schema` |
| `zerogpu_extract_pii` | `gliner2.5-multi-v1` | `text` |
| `zerogpu_redact_pii` | `gliner2.5-multi-v1` | `text` |
| `zerogpu_moderate` | `zlm-v1-moderation-edge` | `input` |
| `zerogpu_embed` | `all-minilm-l6-v2` or `bge-small-en-v1.5` | `input` |
| `zerogpu_generate_followups` | `llama-3.1-8b-instruct-fast` | `text` |
| `zerogpu_health` | none | none |

Every model-backed tool returns `usage` and `savings` alongside its result. `savings.baseline_cost_usd` prices the identical tokens at a published frontier rate of $3.00 per 1M input and $15.00 per 1M output, so `savings_usd` is a like-for-like comparison rather than an estimate. For the full argument reference, see the [MCP server integration guide](/integrations/mcp).

### Controlling which tools load

Hermes can filter an MCP server's tools. `include` wins over `exclude`, so an allowlist is a hard boundary:

```yaml theme={null}
mcp_servers:
  zerogpu:
    url: "https://mcp.zerogpu.ai/mcp?apiKey=..."
    tools:
      include: [zerogpu_summarize, zerogpu_redact_pii, zerogpu_classify_zero_shot]
      prompts: false
      resources: false
```

Or interactively:

```bash theme={null}
hermes mcp configure zerogpu
```

### Installing a skill

Hermes skills are Markdown procedures the agent loads on demand. Install one from any public URL:

```bash theme={null}
hermes skills install https://raw.githubusercontent.com/zerogpu/docs/main/skills/investor-outreach/SKILL.md
hermes skills list
```

Skills are also auto-discovered from the skills directory, so dropping a folder containing a `SKILL.md` into `<hermes-home>/skills/` works too. Trigger one in chat with `/<skill-name>`.

### Patterns and recipes

**Redact before you reason.** Run `zerogpu_redact_pii` over scraped or user-supplied text before it reaches the agent's context. Names, emails, and phone numbers never enter the reasoning model's prompt or the session transcript.

**Rank without an LLM.** `zerogpu_embed` scores hundreds of candidates against a reference string using cosine similarity and no generative call at all. This is usually the cheapest step in an agent workflow and the one that replaces the most frontier tokens.

**Gate before you act.** Put `zerogpu_moderate` between generation and any outward-facing action, such as sending a message or posting a comment. A flagged result stops the action rather than logging a warning after the fact.

**Structure before you branch.** `zerogpu_extract_json` and `zerogpu_classify_structured` turn free text into fixed fields, so your agent branches on data instead of re-parsing prose on every turn.

### At a glance

| Task | Entry point | Where it lives |
| - | - | - |
| Agent reasoning | `model.provider: custom:ZeroGPU` | `config.yaml` |
| Connect the tools | `hermes mcp add zerogpu --url ...` | `mcp_servers` |
| Verify the tools | `hermes mcp test zerogpu` | CLI |
| Limit the tools | `hermes mcp configure zerogpu` | `mcp_servers.<name>.tools` |
| Install a procedure | `hermes skills install <url>` | skills directory |
| Check spend | `hermes insights`, or `savings` on any tool result | CLI and tool output |

## Troubleshooting

**`hermes mcp test zerogpu` returns no tools.** The key is wrong or missing. The endpoint returns 401 with a plain-text body naming both accepted auth methods. Re-add the server with a valid `zgpu-api-` key.

**The agent's model fails to authenticate while the MCP tools work.** These are two different endpoints with two different credentials. `api.zerogpu.ai` reads `providers.ZeroGPU.key_env` from `.env`; `mcp.zerogpu.ai` reads the key in the MCP URL or header. Fix whichever one is failing.

**Hermes rejects the model at startup.** The context window is under 64,000 tokens. Use `deepseek-v4.1-flash` or `gpt-oss-120b`, and set `context_length` explicitly if auto-detection fails.

**A tool call returns an error inside a normal-looking result.** ZeroGPU MCP failures come back as `isError: true` in the payload, not as a transport error. Check the tool output, not just the HTTP status.

**`zerogpu_embed` rejects the model name.** The `model` argument is a strict, case-sensitive enum: `bge-small-en-v1.5` or `all-minilm-l6-v2`. `BGE-Small-EN-v1.5` fails schema validation before the call is made.

**A new server or skill does not appear in an open chat.** Hermes loads MCP servers and skills at session start. Run `/reload-mcp` in the session, or start a new chat.

**A skill never shows up in `hermes skills list`.** The YAML frontmatter failed to parse, usually a stray tab or an unquoted colon. There is no lint command; the listing is the check.

**429 responses under load.** You are being rate limited. Honor `Retry-After`, and move bulk work to the [Batch API](/docs/batch/index), which accepts 50,000 requests per 24-hour window and sidesteps per-request limits.

**Costs look higher than expected.** Check which model actually served each call. If a fallback provider is configured, it may be answering some requests, and those carry no ZeroGPU savings. `hermes fallback list` shows the chain.

## Conclusion

Hermes gives you an agent whose reasoning model and tool models are chosen independently. Pointing both at ZeroGPU means the frontier-priced work shrinks to the handful of steps that genuinely need judgment, while classification, extraction, ranking, redaction, and moderation run on models built for exactly those jobs. Every tool call returns what it cost and what the same tokens would have cost at frontier pricing, so the tradeoff stays visible rather than assumed.

<CardGroup>
  <Card title="Model catalog" icon="layers" href="/docs/model-catalog">
    Every model, its task, context window, and price.
  </Card>

  <Card title="MCP server" icon="plug" href="/integrations/mcp">
    Full argument reference for all fourteen tools.
  </Card>

  <Card title="Cookbook" icon="book-open" href="/cookbook/hermes-investor-outreach">
    Build an investor outreach agent end to end.
  </Card>

  <Card title="Discord" icon="discord" href="https://discord.gg/zerogpu">
    Ask questions and share what you built.
  </Card>
</CardGroup>


# Overview
Source: https://docs.zerogpu.ai/integrations/index

Plug ZeroGPU's nano language models into the tools and runtimes you already use.

ZeroGPU is an inference provider. We run specialized small and nano language models across an edge-powered network, purpose-built for the high-volume tasks specific verticals depend on, from ad-tech classification to content moderation to document extraction. The result: more efficient compute, faster latency, and roughly 50% lower cost than frontier-model workflows.

Because ZeroGPU is OpenAI-compatible, it drops into the tools you already use. The integrations below let you route eligible workloads to ZeroGPU from your coding agents, frameworks, gateways, automation tools, and ad-tech pipelines, without rebuilding your stack. Each page ships when its integration is live.

Before you begin, read the [quickstart](/docs/quickstart) to provision an [API key](https://platform.zerogpu.ai/dashboard) and pick a model from the [model catalog](/docs/model-catalog). Every integration below points at the same OpenAI-compatible endpoint at `https://api.zerogpu.ai/v1`. Prefer to call the API directly? Use one of our [official SDKs](#official-sdks) for your language.

## Editor & Agent Integrations

<CardGroup>
  <Card title="Claude Code Plugin" icon="terminal" href="/integrations/claude-code-plugin">
    Route Claude Code's repetitive steps through ZeroGPU from the terminal.
    Powered by the ZeroGPU Router.
  </Card>

  <Card title="Claude Skill" icon="sparkles" href="/integrations/claude-skill">
    Upload one SKILL.md so Claude follows ZeroGPU best practices whenever you
    ask about inference, models, the CLI, or the API.
  </Card>

  <Card title="Hermes Agent" icon="robot" href="/integrations/hermes">
    Run Hermes Agent's own reasoning on ZeroGPU and give it fourteen
    task-specific tools over MCP.
  </Card>

  <Card title="LangChain" icon="crow" href="/integrations/langchain">
    Offload classification, extraction, PII redaction, and chat from LangChain
    agents to ZeroGPU's nano models with eleven ready-made tools.
  </Card>

  <Card title="MCP Server" icon="plug" href="/integrations/mcp">
    Connect any MCP client to ZeroGPU's nano models over the Model Context
    Protocol.
  </Card>

  <Card title="OpenClaw" icon="lobster" href="/integrations/openclaw">
    Route your OpenClaw agent's routine tasks - classification, extraction, PII
    redaction, chat - to ZeroGPU's nano models. Powered by the ZeroGPU Router.
  </Card>
</CardGroup>

## Official SDKs

Call the OpenAI-compatible endpoint from your language of choice. Each guide shows how to point an existing OpenAI client at ZeroGPU.

<CardGroup>
  <Card title="Go" icon="golang" href="/integrations/sdk/go">
    Wire ZeroGPU into Go services using an OpenAI-compatible client.
  </Card>

  <Card title="JavaScript" icon="js" href="/integrations/sdk/javascript">
    Drop ZeroGPU into Node.js and the browser with the OpenAI JavaScript SDK.
  </Card>

  <Card title="Python" icon="python" href="/integrations/sdk/python">
    Use the OpenAI Python SDK with ZeroGPU's base URL for chat, classification,
    and extraction.
  </Card>

  <Card title="Ruby" icon="gem" href="/integrations/sdk/ruby">
    Use ZeroGPU from Ruby with the OpenAI-compatible client library.
  </Card>

  <Card title="Rust" icon="rust" href="/integrations/sdk/rust">
    Call ZeroGPU's endpoint from Rust with an OpenAI-compatible client.
  </Card>
</CardGroup>


# LangChain
Source: https://docs.zerogpu.ai/integrations/langchain

Offload classification, extraction, PII redaction, and chat from LangChain agents to ZeroGPU's nano language models with eleven ready-made tools.

LangChain is an open-source framework for building applications powered by large language models. It provides composable building blocks for prompts, chains, agents, retrieval, and tool use, and its [tool abstraction](https://python.langchain.com/docs/concepts/tools/) lets any agent call out to external capabilities with validated, typed inputs. Teams use LangChain to wire LLMs into production pipelines without rewriting glue code for every model vendor.

## Overview

This guide walks through [`langchain-zerogpu`](https://github.com/zerogpu/langchain-zerogpu), the official package that exposes ZeroGPU's task models as first-class LangChain `BaseTool` subclasses. You'll install the package from [PyPI](https://pypi.org/project/langchain-zerogpu/), authenticate with your API key and project ID, invoke your first tool, and then work through all eleven tools - chat, summarization, classification, entity and JSON extraction, and PII handling - plus the `ZeroGPUToolkit` that binds the whole set to an agent in one line. By the end, any LangChain agent (including `create_agent` and LangGraph graphs) can hand its repeatable NLP work to ZeroGPU instead of spending frontier-model tokens.

## Cookbook

Prefer to learn by running real code? A worked, end-to-end example is available as a notebook: **[Screen Resumes with LangChain and ZeroGPU](https://colab.research.google.com/drive/10OYr9s4kfp63twlN_FjXfuhtZpwjba6A?usp=sharing)**. It chains three of the tools below to pull structured fields out of a PDF resume, strip the PII before anything is stored or shared, and route the candidate to the right team, all on ZeroGPU's small models. Open it in Google Colab and run it top to bottom.

<Note>
  More LangChain cookbooks are on the way. Check the [cookbook index](/cookbook/index) for the latest worked examples.
</Note>

## Video walkthrough

<iframe title="YouTube video player" />

## Quickstart

### Prerequisites

* Python 3.10 or newer.
* A ZeroGPU [API key](https://platform.zerogpu.ai/dashboard) (starts with `zgpu-api-`) and a Project ID.
* A look at the [model catalog](/docs/model-catalog) if you want to see what each tool routes to.

### Get your ZeroGPU API key

1. Sign in to the [ZeroGPU dashboard](https://platform.zerogpu.ai/dashboard).
2. Open **API Keys** and click **Create key**.
3. Copy the key (starts with `zgpu-api-`) and grab your Project ID (UUID) from the project settings page.
4. Export both so every tool can pick them up automatically:

```bash theme={null}
export ZEROGPU_API_KEY="zgpu-api-..."
export ZEROGPU_PROJECT_ID="your-project-id"
```

### Install langchain-zerogpu

```bash theme={null}
pip install langchain-zerogpu
```

The package depends on `langchain-core` and the official [`zerogpu-api`](https://pypi.org/project/zerogpu-api/) Python SDK; every call goes through the SDK to ZeroGPU's Responses API at `https://api.zerogpu.ai/v1`. Source lives at [github.com/zerogpu/langchain-zerogpu](https://github.com/zerogpu/langchain-zerogpu).

### Your first request

With `ZEROGPU_API_KEY` and `ZEROGPU_PROJECT_ID` exported, construct a tool with no arguments and invoke it:

```python theme={null}
from langchain_zerogpu import ZeroGPUClassifyZeroShotTool

tool = ZeroGPUClassifyZeroShotTool()  # reads creds from the environment

print(tool.invoke({
    "text": "The new GPU smashes every benchmark we threw at it.",
    "labels": ["tech", "politics", "sports"],
}))
```

```json theme={null}
{ "label": "tech", "scores": { "tech": 0.95, "politics": 0.03, "sports": 0.02 } }
```

## Usage

The package ships eleven tools, each a LangChain `BaseTool` with a typed `args_schema`, plus a `ZeroGPUToolkit` that bundles them behind one shared client. Every tool supports synchronous `invoke` and asynchronous `ainvoke`, and all calls route through ZeroGPU's `POST /v1/responses` endpoint. Chat, summarize, and redact tools return plain strings; classification and extraction tools return parsed JSON (a `dict` or `list`), falling back to the raw string if the model output isn't valid JSON.

### Construction and credentials

Every tool (and the toolkit) accepts the same constructor arguments. Pass nothing to resolve credentials from the environment:

| Argument | Required | Default | Description |
| - | - | - | - |
| `api_key` | no | `ZEROGPU_API_KEY` env var | ZeroGPU API key. Must start with `zgpu-api-`. Stored as a `pydantic.SecretStr`, never logged. |
| `project_id` | no | `ZEROGPU_PROJECT_ID` env var | ZeroGPU Project ID (UUID). |
| `base_url` | no | production API | Optional base URL override for the ZeroGPU API. |
| `client` | no | built from the above | A shared `ZeroGPUClient`; this is how the toolkit wires one client (and one connection pool) across all tools. |

Failures surface as typed exceptions instead of raw stack traces: `ZeroGPUAuthError` for missing or malformed credentials and `401`/`403` responses, and `ZeroGPUError` for rate limits (`429`), server errors (`5xx`), and network failures.

### `ZeroGPUChatTool`

Short, single-turn chat reply for prompts that don't need frontier-model reasoning or conversation history.

* **Tool name:** `zerogpu_chat`
* **Model:** `LFM2.5-1.2B-Instruct`
* **Returns:** `str` (the assistant reply)

| Argument | Required | Default | Description |
| - | - | - | - |
| `text` | yes | - | The user message to respond to. |
| `system` | no | `None` | Optional system prompt that steers the reply. |

```python theme={null}
from langchain_zerogpu import ZeroGPUChatTool

tool = ZeroGPUChatTool()
print(tool.invoke({
    "text": "Explain WebSockets in two sentences.",
    "system": "You are a concise technical writer.",
}))
```

```text theme={null}
WebSockets provide a persistent, full-duplex connection between a client and a
server over a single TCP socket. Unlike HTTP request/response cycles, both sides
can push messages at any time, making them ideal for chat, live dashboards, and
multiplayer state sync.
```

### `ZeroGPUChatThinkingTool`

Same input shape as `ZeroGPUChatTool`, but the model returns a visible step-by-step reasoning trace followed by its answer. Use it for short logic, math, or word problems where you want the small model's intermediate reasoning.

* **Tool name:** `zerogpu_chat_thinking`
* **Model:** `LFM2.5-1.2B-Thinking`
* **Returns:** `str` (reasoning trace plus answer)

| Argument | Required | Default | Description |
| - | - | - | - |
| `text` | yes | - | The user message to respond to. |
| `system` | no | `None` | Optional system prompt that steers the reply. |

```python theme={null}
from langchain_zerogpu import ZeroGPUChatThinkingTool

tool = ZeroGPUChatThinkingTool()
print(tool.invoke({
    "text": "If a train leaves at 3 PM going 60 mph, when does it cover 150 miles?"
}))
```

```text theme={null}
<thinking>
The train travels at 60 mph and needs to cover 150 miles.
Time = distance / speed = 150 / 60 = 2.5 hours.
Starting at 3:00 PM, adding 2 hours and 30 minutes lands at 5:30 PM.
</thinking>

The train covers 150 miles at 5:30 PM.
```

### `ZeroGPUSummarizeTool`

Condense a passage into a short summary. Best for passages up to a few paragraphs - reports, ticket threads, transcripts.

* **Tool name:** `zerogpu_summarize`
* **Model:** `llama-3.1-8b-instruct-fast`
* **Returns:** `str` (the summary)

| Argument | Required | Default | Description |
| - | - | - | - |
| `text` | yes | - | The input text to summarize. |

```python theme={null}
from langchain_zerogpu import ZeroGPUSummarizeTool

tool = ZeroGPUSummarizeTool()
print(tool.invoke({
    "text": (
        "The board met Thursday to review Q3 results. Revenue rose 18% "
        "year-over-year to $42M, driven mainly by enterprise renewals and a "
        "strong launch in the EU market. Operating margin slipped to 11% from "
        "14% as headcount grew 30% ahead of the new data-center buildout."
    )
}))
```

```text theme={null}
Q3 revenue grew 18% YoY to $42M on enterprise renewals and EU growth, but
operating margin fell to 11% due to a 30% headcount increase for the
data-center buildout.
```

### `ZeroGPUClassifyIABTool`

Classify text into the IAB content taxonomy (the standard ad / content category taxonomy).

* **Tool name:** `zerogpu_classify_iab`
* **Model:** `zlm-v1-iab-classify-edge`
* **Returns:** parsed JSON (`dict`) with the IAB categories

| Argument | Required | Default | Description |
| - | - | - | - |
| `text` | yes | - | The input text to classify. |

```python theme={null}
from langchain_zerogpu import ZeroGPUClassifyIABTool

tool = ZeroGPUClassifyIABTool()
print(tool.invoke({"text": "The Lakers signed a new point guard ahead of the playoffs."}))
```

```json theme={null}
{
  "categories": [
    { "id": "IAB17-44", "name": "Basketball", "confidence": 0.97 }
  ]
}
```

### `ZeroGPUClassifyIABEnrichedTool`

Enriched IAB classification: categories plus topics, keywords, and inferred user intent. Use when you need richer ad / audience signals than plain IAB labels.

* **Tool name:** `zerogpu_classify_iab_enriched`
* **Model:** `zlm-v2-iab-classify-edge-enriched`
* **Returns:** parsed JSON (`dict`) with categories, topics, keywords, and intent

| Argument | Required | Default | Description |
| - | - | - | - |
| `text` | yes | - | The input text to classify. |

```python theme={null}
from langchain_zerogpu import ZeroGPUClassifyIABEnrichedTool

tool = ZeroGPUClassifyIABEnrichedTool()
print(tool.invoke({
    "text": "Compare the Tesla Model Y and the Hyundai Ioniq 5 for a family of four."
}))
```

```json theme={null}
{
  "categories": [{ "id": "IAB2-1", "name": "Auto Buyers", "confidence": 0.92 }],
  "topics": ["electric vehicles", "family cars"],
  "keywords": ["Tesla Model Y", "Hyundai Ioniq 5"],
  "intent": "comparison-shopping"
}
```

### `ZeroGPUClassifyZeroShotTool`

Zero-shot classification against a flat list of candidate labels you supply at call time. Returns a score per label so you (or your agent) can pick the best match.

* **Tool name:** `zerogpu_classify_zero_shot`
* **Model:** `deberta-v3-small`
* **Returns:** parsed JSON (`dict`) with the winning label and per-label scores

| Argument | Required | Default | Description |
| - | - | - | - |
| `text` | yes | - | The text to classify. |
| `labels` | yes | - | Candidate labels to score against, e.g. `["tech", "politics", "sports"]`. At least one. |

```python theme={null}
from langchain_zerogpu import ZeroGPUClassifyZeroShotTool

tool = ZeroGPUClassifyZeroShotTool()
print(tool.invoke({
    "text": "I love how fast this laptop boots up.",
    "labels": ["positive", "negative", "neutral"],
}))
```

```json theme={null}
{ "label": "positive", "scores": { "positive": 0.94, "neutral": 0.04, "negative": 0.02 } }
```

### `ZeroGPUClassifyStructuredTool`

Multi-axis classification driven by a labelled schema. You define each axis and its allowed labels; the model returns one chosen label per axis in a single call.

* **Tool name:** `zerogpu_classify_structured`
* **Model:** `gliner2.5-base-v1`
* **Returns:** parsed JSON (`dict`) mapping each axis to its chosen label

| Argument | Required | Default | Description |
| - | - | - | - |
| `text` | yes | - | The text to classify. |
| `schema` | yes | - | Axes mapped to candidate labels, e.g. `{"sentiment": ["positive", "negative"], "topic": ["billing", "support"]}`. |
| `threshold` | no | `None` (service default) | Confidence threshold in `[0, 1]` for filtering labels. |

```python theme={null}
from langchain_zerogpu import ZeroGPUClassifyStructuredTool

tool = ZeroGPUClassifyStructuredTool()
print(tool.invoke({
    "text": "Support replied quickly but the fix didn't work.",
    "schema": {
        "sentiment": ["positive", "negative", "neutral"],
        "topic": ["support", "billing", "product"],
    },
}))
```

```json theme={null}
{ "sentiment": "negative", "topic": "support" }
```

### `ZeroGPUExtractEntitiesTool`

Custom-label named-entity recognition. You define the entity types; the model finds matching spans with confidence scores.

* **Tool name:** `zerogpu_extract_entities`
* **Model:** `gliner2.5-base-v1`
* **Returns:** parsed JSON (`list`) of matched spans grouped by label

| Argument | Required | Default | Description |
| - | - | - | - |
| `text` | yes | - | The text to extract entities from. |
| `labels` | yes | - | Entity types to extract, e.g. `["person", "company", "date"]`. At least one. |
| `threshold` | no | `None` (service default) | Confidence threshold in `[0, 1]` for filtering spans. |

```python theme={null}
from langchain_zerogpu import ZeroGPUExtractEntitiesTool

tool = ZeroGPUExtractEntitiesTool()
print(tool.invoke({
    "text": "Apple CEO Tim Cook met with Sundar Pichai in Cupertino on Monday.",
    "labels": ["person", "organization", "location"],
    "threshold": 0.4,
}))
```

```json theme={null}
[
  { "label": "organization", "text": "Apple",         "score": 0.98 },
  { "label": "person",       "text": "Tim Cook",      "score": 0.97 },
  { "label": "person",       "text": "Sundar Pichai", "score": 0.96 },
  { "label": "location",     "text": "Cupertino",     "score": 0.91 }
]
```

### `ZeroGPUExtractPIITool`

Detect and extract personally identifiable information, grouped by category, without modifying the source text. Use when you need structured data about PII (for redaction policies, audits, or downstream tooling) rather than a masked version.

* **Tool name:** `zerogpu_extract_pii`
* **Model:** `gliner2.5-multi-v1`
* **Returns:** parsed JSON (`list`) of detected PII entities

| Argument | Required | Default | Description |
| - | - | - | - |
| `text` | yes | - | The text to scan for PII. |
| `categories` | no | `None` (all detected PII) | Categories to restrict the scan to, e.g. `["identity", "contact"]`. Other values: `financial`, `medical`, `credentials`. |
| `threshold` | no | `None` (service default) | Confidence threshold in `[0, 1]` for filtering matches. |

```python theme={null}
from langchain_zerogpu import ZeroGPUExtractPIITool

tool = ZeroGPUExtractPIITool()
print(tool.invoke({
    "text": "Contact Jane Doe at jane@example.com or +1 (415) 555-1212.",
    "categories": ["identity", "contact"],
}))
```

```json theme={null}
[
  { "category": "identity", "label": "person", "text": "Jane Doe",          "score": 0.96 },
  { "category": "contact",  "label": "email",  "text": "jane@example.com",  "score": 0.99 },
  { "category": "contact",  "label": "phone",  "text": "+1 (415) 555-1212", "score": 0.95 }
]
```

### `ZeroGPURedactPIITool`

Detect PII and replace each span inline with a `[LABEL]` placeholder (the tool calls the PII model with `mask: "label"`). Use it before logging, sharing, or forwarding text to another LLM you don't want to expose raw PII to.

* **Tool name:** `zerogpu_redact_pii`
* **Model:** `gliner2.5-multi-v1` (with `mask: "label"`)
* **Returns:** `str` (the redacted text)

| Argument | Required | Default | Description |
| - | - | - | - |
| `text` | yes | - | The text to redact PII from. |
| `categories` | no | `None` (all detected PII) | Categories to restrict redaction to. |
| `threshold` | no | `None` (service default) | Confidence threshold in `[0, 1]` for filtering matches. |

```python theme={null}
from langchain_zerogpu import ZeroGPURedactPIITool

tool = ZeroGPURedactPIITool()
print(tool.invoke({"text": "Email John Smith at john@acme.com about invoice 12345."}))
```

```text theme={null}
Email [PERSON] at [EMAIL] about invoice 12345.
```

Note that `12345` is not masked: only spans the model recognizes as PII are replaced. Handle domain-specific identifiers (account numbers, internal ticket IDs) with your own redaction layer or with `ZeroGPUExtractEntitiesTool` and custom labels.

### `ZeroGPUExtractJSONTool`

Schema-driven JSON extraction: pull named fields out of free text into a structured object. Each field is declared as `name::type::description`, grouped under a group name.

* **Tool name:** `zerogpu_extract_json`
* **Model:** `gliner2.5-base-v1`
* **Returns:** parsed JSON (`dict`) with the extracted fields

| Argument | Required | Default | Description |
| - | - | - | - |
| `text` | yes | - | The text to extract fields from. |
| `schema` | yes | - | Grouped schema mapping a group name to `"field::type::description"` specs, e.g. `{"contact": ["name::str::Full name", "email::str::Email address"]}`. |
| `threshold` | no | `None` (service default) | Confidence threshold in `[0, 1]` for filtering fields. |

```python theme={null}
from langchain_zerogpu import ZeroGPUExtractJSONTool

tool = ZeroGPUExtractJSONTool()
print(tool.invoke({
    "text": "Reach Maria Lopez at maria.lopez@acme.io or 415-555-0188.",
    "schema": {
        "contact": [
            "name::str::Full name",
            "email::str::Email address",
            "phone::str::Phone number",
        ]
    },
}))
```

```json theme={null}
{
  "contact": {
    "name": "Maria Lopez",
    "email": "maria.lopez@acme.io",
    "phone": "415-555-0188"
  }
}
```

### `ZeroGPUToolkit`

The toolkit bundles all eleven tools behind a single shared SDK client (one connection pool, one credential resolution). Construct it once and call `get_tools()` to register the whole set with an agent in one line. It takes the same `api_key` / `project_id` / `base_url` arguments as the individual tools.

```python theme={null}
from langchain.agents import create_agent
from langchain_zerogpu import ZeroGPUToolkit

toolkit = ZeroGPUToolkit()  # reads ZEROGPU_API_KEY / ZEROGPU_PROJECT_ID
tools = toolkit.get_tools()  # all eleven tools, one shared client

agent = create_agent("anthropic:claude-sonnet-4-6", tools=tools)

result = agent.invoke({
    "messages": [
        {"role": "user", "content": (
            "Triage this support ticket: summarize it in one line and tell me "
            "whether it's a bug, feature request, or question. "
            "'The CSV export on the billing page returns a 500 error whenever I "
            "select more than 90 days of data. Smaller ranges work fine.'"
        )}
    ]
})
print(result["messages"][-1].content)
```

```text theme={null}
Summary: Exporting more than 90 days of billing data as CSV returns a 500 error,
while smaller date ranges work fine.
Category: bug.
```

Behind that one call, the agent routed the summary to `ZeroGPUSummarizeTool` and the label to `ZeroGPUClassifyZeroShotTool`, each a cheap ZeroGPU call instead of more frontier-model tokens.

Each tool ships an LLM-facing description, so the agent picks the right one from intent ("redact", "summarize", "classify by sentiment and topic") without you naming tools explicitly. You can also bind a single tool directly to a chat model:

```python theme={null}
from langchain.chat_models import init_chat_model
from langchain_zerogpu import ZeroGPUExtractPIITool

llm = init_chat_model("anthropic:claude-sonnet-4-6")
llm_with_tools = llm.bind_tools([ZeroGPUExtractPIITool()])
```

### Async usage

Every tool implements the async path, so the same inputs work with `ainvoke` inside LangGraph nodes or any asyncio application:

```python theme={null}
from langchain_zerogpu import ZeroGPUSummarizeTool

summary = await ZeroGPUSummarizeTool().ainvoke({
    "text": (
        "The board met Thursday to review Q3 results. Revenue rose 18% "
        "year-over-year to $42M, driven by enterprise renewals and a strong EU "
        "launch, while operating margin slipped to 11% from 14% as headcount "
        "grew ahead of the new data-center buildout."
    )
})
print(summary)
```

```text theme={null}
Q3 revenue grew 18% year-over-year to $42M on enterprise renewals and EU
expansion, though operating margin fell to 11% as headcount outpaced the new
data-center buildout.
```

### Patterns and recipes

**Sanitize before the frontier model sees raw input.** Run untrusted text through `ZeroGPURedactPIITool` before it enters your agent's context or transcript. Combine with `ZeroGPUExtractPIITool` when you also need an audit log of what was masked.

```python theme={null}
redacted = ZeroGPURedactPIITool().invoke({"text": user_input})
audit = ZeroGPUExtractPIITool().invoke({"text": user_input})
agent.invoke({"messages": [{"role": "user", "content": redacted}]})
```

**Cheap classifier in front of an expensive model.** Use `ZeroGPUClassifyZeroShotTool` or `ZeroGPUClassifyStructuredTool` to triage incoming messages (bug / feature / question, urgent / normal) and only escalate hard cases to the frontier model. The classifier call costs orders of magnitude less than a frontier-model turn.

**Structured extraction over free-form parsing.** For semi-structured text (signatures, invoices, contact blocks), prefer `ZeroGPUExtractJSONTool` over asking a chat model to "parse this into JSON". It's deterministic on the schema, faster, and cheaper. Keep field descriptions short and specific - the description is what the model uses to find each span.

**Confidence thresholds.** For NER and PII extraction, omitting `threshold` uses service defaults tuned for recall (roughly `0.3` for NER, `0.5` for PII). Raise it to `0.6` or higher when you need precision (compliance-grade redaction lists); lower it when you'd rather over-extract and filter downstream.

### Tools reference table

Every tool at a glance.

| Tool class | Tool name | ZeroGPU model | Purpose | Returns |
| - | - | - | - | - |
| `ZeroGPUChatTool` | `zerogpu_chat` | `LFM2.5-1.2B-Instruct` | Short single-turn chat reply | `str` |
| `ZeroGPUChatThinkingTool` | `zerogpu_chat_thinking` | `LFM2.5-1.2B-Thinking` | Chat with a visible reasoning trace | `str` |
| `ZeroGPUSummarizeTool` | `zerogpu_summarize` | `llama-3.1-8b-instruct-fast` | Condense a passage | `str` |
| `ZeroGPUClassifyIABTool` | `zerogpu_classify_iab` | `zlm-v1-iab-classify-edge` | IAB taxonomy classification | JSON |
| `ZeroGPUClassifyIABEnrichedTool` | `zerogpu_classify_iab_enriched` | `zlm-v2-iab-classify-edge-enriched` | IAB + topics / keywords / intent | JSON |
| `ZeroGPUClassifyZeroShotTool` | `zerogpu_classify_zero_shot` | `deberta-v3-small` | Zero-shot vs. custom labels | JSON |
| `ZeroGPUClassifyStructuredTool` | `zerogpu_classify_structured` | `gliner2.5-base-v1` | Multi-axis schema classification | JSON |
| `ZeroGPUExtractEntitiesTool` | `zerogpu_extract_entities` | `gliner2.5-base-v1` | Custom-label NER | JSON |
| `ZeroGPUExtractPIITool` | `zerogpu_extract_pii` | `gliner2.5-multi-v1` | Extract PII entities | JSON |
| `ZeroGPURedactPIITool` | `zerogpu_redact_pii` | `gliner2.5-multi-v1` (mask: label) | Mask PII inline with `[LABEL]` | `str` |
| `ZeroGPUExtractJSONTool` | `zerogpu_extract_json` | `gliner2.5-base-v1` | Schema-driven JSON extraction | JSON |
| `ZeroGPUToolkit` | - | - | Bundles all eleven tools behind one shared client | `list[BaseTool]` |

## Troubleshooting

**`ZeroGPUAuthError: No ZeroGPU API key provided`** - no key was passed and `ZEROGPU_API_KEY` isn't set in the environment the Python process sees. Export it (`export ZEROGPU_API_KEY="zgpu-api-..."`) or pass `api_key=...` to the tool or toolkit. Remember that notebooks and IDE run configurations often don't inherit your shell profile.

**`ZeroGPUAuthError: Invalid ZeroGPU API key`** - the key must start with `zgpu-api-`. You've likely pasted a truncated key or a different credential; copy it again from the [dashboard](https://platform.zerogpu.ai/dashboard).

**`ZeroGPUAuthError: No ZeroGPU project id provided`** - the package needs both a key and a project ID. Set `ZEROGPU_PROJECT_ID` or pass `project_id=...`.

**`ZeroGPU authentication failed (401)`** - the key was rejected. It's been revoked or mistyped; rotate it in the dashboard and update `ZEROGPU_API_KEY`.

**`ZeroGPU access denied (403)`** - the key is valid but the project doesn't have access to the requested model, or the Project ID doesn't match the project that owns the key. Check `ZEROGPU_PROJECT_ID` and your model entitlements.

**`ZeroGPU rate limit exceeded (429)`** - back off and retry with exponential delay, or move bulk workloads to the [Batch API](/docs/batch/index), which has separate quotas tuned for offline jobs.

**Pydantic validation error when invoking a tool** - tool inputs are validated against each tool's `args_schema`. Pass a `dict` with the documented argument names (`tool.invoke({"text": ..., "labels": [...]})`), not a bare string, and make sure list arguments like `labels` contain at least one entry.

**Classification or extraction returns a string instead of a dict** - structured tools parse the model output as JSON and fall back to the raw string when parsing fails. This usually means the input was too short or ambiguous for the model to produce a structured result; retry with more context or log the raw string to inspect it.

**Empty or low-confidence extraction results** - lower `threshold` to surface more candidates, or check that your labels match the language of the source text (the underlying models are English-tuned for most label sets). Very short inputs (one or two words) score low across the board.

**The agent never picks a ZeroGPU tool** - tool selection is driven by each tool's description and your prompt. Phrase requests with intent words that match the task ("redact", "summarize", "classify as bug / feature / question"), or invoke the tool directly instead of going through the agent.

**`UserWarning: Field name "schema" shadows an attribute`** - harmless and already suppressed inside the package; the schema-driven tools deliberately expose a field named `schema` because that's the natural LLM-facing argument name. If you see it, you're likely re-declaring the models yourself; the filter only covers the package's own schemas.

## Conclusion

`langchain-zerogpu` gives every LangChain agent a set of fast, low-cost specialists for the NLP work it runs constantly - classification, extraction, PII handling, summarization, and short chat - so frontier-model tokens are spent only where frontier-model reasoning is needed. Install the package, export two environment variables, and `ZeroGPUToolkit().get_tools()` puts all eleven on the table.

<CardGroup>
  <Card title="Model Catalog" icon="layer-group" href="/docs/model-catalog">
    Browse every model the tools route to and pick the best fit.
  </Card>

  <Card title="API Reference" icon="code" href="/api-reference/responses">
    Explore the full OpenAI-compatible API surface.
  </Card>

  <Card title="Cookbook" icon="book" href="/cookbook/index">
    Worked examples for classification, extraction, and batch jobs.
  </Card>

  <Card title="Join Discord" icon="discord" href="https://discord.gg/Ad5KZvAyky">
    Ask questions and share what you're building.
  </Card>
</CardGroup>


# MCP Server
Source: https://docs.zerogpu.ai/integrations/mcp

Connect any MCP client to ZeroGPU's nano language models over the Model Context Protocol.

The [Model Context Protocol](https://modelcontextprotocol.io) (MCP) is an open standard that lets AI assistants and agents discover and call external tools through a uniform interface. An MCP client (Claude Desktop, Claude Code, Cursor, your own agent runtime, and others) connects to an MCP server, lists the tools it exposes, and invokes them with structured arguments. Because the protocol is transport- and vendor-neutral, one server works across every compliant client without bespoke glue code.

ZeroGPU is an ultra-fast, compute-efficient inference provider for apps and agents. We run purpose-built small and nano language models across an edge-powered network for the high-volume, purpose-specific tasks your app or agent runs constantly. Plug in our OpenAI-compatible API and you're live - zero GPU infrastructure, serverless, auto-scaling by default.

## Overview

This guide connects an MCP client to the hosted ZeroGPU MCP server at `https://mcp.zerogpu.ai/mcp`. The server speaks the streamable HTTP transport and exposes ZeroGPU's edge models (summarization, classification, entity and PII extraction, safety moderation, embeddings, short chat, and more) as MCP tools, so a host model like Claude can offload cheap, well-defined NLP work instead of doing it itself. By the end you'll be able to authenticate, list the available tools, and call any of them, including a worked example for the `zerogpu_summarize` tool.

## Video walkthrough

Watch the full demo here:

<iframe title="YouTube video player" />

## Quickstart

### Prerequisites

* An MCP-capable client (Claude Desktop, Claude Code, Cursor, or any runtime that speaks MCP over streamable HTTP).
* A ZeroGPU [API key](https://platform.zerogpu.ai/dashboard) and Project ID.
* A model ID from the [model catalog](/docs/model-catalog) if you plan to override the default model on `zerogpu_chat`.

### Get your ZeroGPU API key

1. Sign in to the [ZeroGPU dashboard](https://platform.zerogpu.ai/dashboard).
2. Open **API Keys** and click **Create key**.
3. Copy the key (starts with `zgpu-api-`) and grab your Project ID (UUID) from the project settings page.

### Connect the server

The ZeroGPU MCP server authenticates with the `x-api-key` header (your API key) on every request; the `x-project-id` header (your Project ID) is optional. Most clients let you declare a remote server with custom headers in a config file. Using placeholders:

```json theme={null}
{
  "mcpServers": {
    "zerogpu": {
      "type": "http",
      "url": "https://mcp.zerogpu.ai/mcp",
      "headers": {
        "x-api-key": "<YOUR_API_KEY>",
        "x-project-id": "<YOUR_PROJECT_ID>"
      }
    }
  }
}
```

In Claude Code you can register the same server from the terminal:

```bash theme={null}
claude mcp add --transport http zerogpu https://mcp.zerogpu.ai/mcp \
  --header "x-api-key: <YOUR_API_KEY>" \
  --header "x-project-id: <YOUR_PROJECT_ID>"
```

Never commit real credentials. Keep them in environment variables or a local, git-ignored config and substitute them at runtime.

### Your first request

Once the server is registered, the client performs the MCP handshake (`initialize`, then `notifications/initialized`) automatically and the ZeroGPU tools become available to the host model. To verify the connection end to end without a client, you can drive the streamable HTTP transport directly with `curl`. First initialize a session and capture the `mcp-session-id` response header:

```bash theme={null}
export ZEROGPU_API_KEY="<YOUR_API_KEY>"
export ZEROGPU_PROJECT_ID="<YOUR_PROJECT_ID>"

curl -sS -D - https://mcp.zerogpu.ai/mcp \
  -H "Content-Type: application/json" \
  -H "Accept: application/json, text/event-stream" \
  -H "x-api-key: $ZEROGPU_API_KEY" \
  -H "x-project-id: $ZEROGPU_PROJECT_ID" \
  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"my-client","version":"1.0.0"}}}'
```

The server replies with its capabilities and returns an `mcp-session-id` header. Pass that session id (plus the same auth headers) on every later call. A quick health check confirms the orchestration API is reachable:

```bash theme={null}
curl -sS https://mcp.zerogpu.ai/mcp \
  -H "Content-Type: application/json" \
  -H "Accept: application/json, text/event-stream" \
  -H "x-api-key: $ZEROGPU_API_KEY" \
  -H "x-project-id: $ZEROGPU_PROJECT_ID" \
  -H "mcp-session-id: <SESSION_ID>" \
  -d '{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"zerogpu_health","arguments":{}}}'
```

Inside the tool result you'll get back overall and per-component status:

```json theme={null}
{
  "status": "ok",
  "components": {
    "worker": { "status": "ok" },
    "redisDeviceRegistry": { "status": "ok", "error": null },
    "redisKeys": { "status": "ok", "error": null },
    "cloudInference": { "status": "ok", "error": null }
  }
}
```

## Usage

The server exposes 14 tools. Every tool name is prefixed with `zerogpu_`, takes a JSON `arguments` object, and returns a single text content block whose body is a JSON string. The list below is the live tool catalog discovered from the server's `tools/list` response; each entry shows what the tool does, which ZeroGPU model or endpoint backs it, its full argument surface, and the shape of the response.

Throughout this section, "client" means any MCP host (Claude, Cursor, your own runtime). When a tool auto-invokes, the host model picks it based on the intent words in your request; you can also call any tool explicitly by name through your client's tool-call interface.

#### `zerogpu_health`

Ping the ZeroGPU Orchestration API to confirm it is reachable and return per-component status. Use it as a pre-flight before a batch of calls, or when other tools start failing.

* **Backed by:** ZeroGPU Orchestration API health endpoint.

| Name | Required | Default | Description |
| - | - | - | - |
| *(none)* | - | - | Takes no arguments. |

Returns an object with a top-level `status` and a `components` map (`worker`, `redisDeviceRegistry`, `redisKeys`, `cloudInference`), each `ok` or reporting an error.

#### `zerogpu_summarize`

Summarize, condense, shorten, recap, or TL;DR any text passage (articles, emails, meeting notes, transcripts, docs, chat logs). Runs on a fast small instruct model on the ZeroGPU edge, far cheaper than summarizing the text with the host model.

* **Backed by:** `llama-3.1-8b-instruct-fast`.

| Name | Required | Default | Description |
| - | - | - | - |
| `text` | yes | - | The passage to summarize (non-empty). |
| `max_tokens` | optional | model default | Upper bound on summary length, `1` to `2048`. |

Returns `summary`, the `model` used, a `usage` token breakdown, and a `savings` object comparing ZeroGPU cost to a frontier-model baseline. See the full worked example below.

#### `zerogpu_classify_iab`

Classify text into the IAB (Interactive Advertising Bureau) ad-tech taxonomy: news articles, pages, ad creative, or blog posts. Cheaper and purpose-built compared to reasoning topics out with the host model.

* **Backed by:** `zlm-v1-iab-classify-edge` (`zlm-v2-iab-classify-edge-enriched` when `enriched=true` or `multilingual=true`).

| Name | Required | Default | Description |
| - | - | - | - |
| `text` | yes | - | Text to classify (non-empty). |
| `enriched` | optional | `false` | Pass `true` for the richer, more granular taxonomy (topics, keywords, intent). |
| `multilingual` | optional | `false` | Pass `true` when the text is not in English. Returns the same enriched shape. |

Returns the predicted IAB category, with scores when available.

```json theme={null}
{ "name": "zerogpu_classify_iab", "arguments": { "text": "The Lakers signed a new point guard ahead of the playoffs.", "enriched": false } }
```

#### `zerogpu_classify_domain`

Classify a website into the IAB taxonomy from its domain name alone, with no page fetch: "what kind of site is espn.com?", bidstream and inventory enrichment, allow/deny list scoring. Use this when all you have is a hostname; when you have the page text, `zerogpu_classify_iab` is more precise per URL.

* **Backed by:** `zlm-v1-iab-domain-classifier`.

| Name | Required | Default | Description |
| - | - | - | - |
| `domain` | yes | - | Hostname, e.g. `nytimes.com`. A full URL is accepted and reduced to its hostname; the normalized value is echoed back on the result. |

Returns scored content categories across IAB 1.0 and 2.2, plus `topics`, `keywords`, and `user_intent`.

```json theme={null}
{ "name": "zerogpu_classify_domain", "arguments": { "domain": "espn.com" } }
```

#### `zerogpu_classify_zero_shot`

Score text against a flat list of candidate labels you supply at call time, for example "is this email tech, politics, or sports?" or "is this review positive, negative, or neutral?". More consistent than reasoning through the classification with the host model.

* **Backed by:** `deberta-v3-small`.

| Name | Required | Default | Description |
| - | - | - | - |
| `text` | yes | - | Text to classify (non-empty). |
| `labels` | yes | - | Array of at least 2 candidate label strings. |
| `threshold` | optional | - | Score cutoff in `[0, 1]` for multi-label filtering. |

Returns per-label scores.

```json theme={null}
{ "name": "zerogpu_classify_zero_shot", "arguments": { "text": "I love how fast this laptop boots up.", "labels": ["positive", "negative", "neutral"] } }
```

#### `zerogpu_classify_structured`

Classify text along several labeled axes at once, for example `{ "sentiment": ["positive","negative"], "topic": ["tech","sports","finance"] }`. Use this instead of `zerogpu_classify_zero_shot` when the labels are grouped into categories rather than a flat list.

* **Backed by:** `gliner2.5-base-v1`.

| Name | Required | Default | Description |
| - | - | - | - |
| `text` | yes | - | Text to classify (non-empty). |
| `schema` | yes | - | Object mapping each axis name to an array of allowed labels. |

Returns one chosen label per axis.

```json theme={null}
{
  "name": "zerogpu_classify_structured",
  "arguments": {
    "text": "Support replied quickly but the fix didn't work.",
    "schema": { "sentiment": ["positive", "negative", "neutral"], "topic": ["support", "billing", "product"] }
  }
}
```

#### `zerogpu_extract_entities`

Pull named entities out of text: people, places, organizations, dates, products, monetary amounts, or any custom entity types you name.

* **Backed by:** `gliner2.5-base-v1`.

| Name | Required | Default | Description |
| - | - | - | - |
| `text` | yes | - | Source text (non-empty). |
| `labels` | yes | - | Array of at least 1 entity type, e.g. `["person","company","date"]`. |
| `threshold` | optional | - | Minimum confidence in `[0, 1]`. |

Returns an entities map keyed by label.

```json theme={null}
{
  "name": "zerogpu_extract_entities",
  "arguments": { "text": "Apple CEO Tim Cook met Sundar Pichai in Cupertino on Monday.", "labels": ["person", "organization", "location"] }
}
```

#### `zerogpu_extract_json`

Extract structured fields out of unstructured text into a JSON object: contact info from a signature, fields from a resume, line items from an invoice, specs from a product page, ticket details. Deterministic on the schema, faster and cheaper than hand-parsing.

* **Backed by:** `gliner2.5-base-v1` (JSON mode).

| Name | Required | Default | Description |
| - | - | - | - |
| `text` | yes | - | Source text (non-empty). |
| `schema` | yes | - | Grouped schema: keys are group names, values are arrays of `field::type::desc` specs. |

Returns the extracted object, grouped exactly as the schema declares.

```json theme={null}
{
  "name": "zerogpu_extract_json",
  "arguments": {
    "text": "Reach Maria Lopez at maria.lopez@acme.io or 415-555-0188.",
    "schema": { "contact": ["name::str::Full name", "email::str::Email address", "phone::str::Phone number"] }
  }
}
```

#### `zerogpu_redact_pii`

Redact, mask, scrub, or anonymize personally identifiable information in text: names, phone numbers, emails, addresses, SSN, credit cards, account numbers, dates of birth. Use this before sharing or logging sensitive text, or before forwarding user input to another model you don't want to expose raw PII to.

* **Backed by:** `gliner2.5-multi-v1`.

| Name | Required | Default | Description |
| - | - | - | - |
| `text` | yes | - | Source text (non-empty). |
| `mask` | optional | generic mask | Set to `"label"` to substitute `[NAME]`, `[EMAIL]`, `[PHONE]`-style placeholders. |

Returns the redacted text.

```json theme={null}
{ "name": "zerogpu_redact_pii", "arguments": { "text": "Email John Smith at john@acme.com about invoice 12345.", "mask": "label" } }
```

#### `zerogpu_extract_pii`

Detect what PII is present in text and return it grouped by category (identity, contact, financial, medical, credentials) without modifying the source. Use this when you need structured data about PII (for redaction policies, audits, or downstream tooling) rather than a masked version.

* **Backed by:** `gliner2.5-multi-v1`.

| Name | Required | Default | Description |
| - | - | - | - |
| `text` | yes | - | Source text (non-empty). |
| `threshold` | optional | - | Minimum confidence in `[0, 1]` to tighten results. |
| `categories` | optional | all | Array filter, e.g. `["identity","contact"]`. |

Returns the PII grouped by category.

```json theme={null}
{ "name": "zerogpu_extract_pii", "arguments": { "text": "Contact Jane Doe at jane@example.com or +1 (415) 555-1212.", "categories": ["identity", "contact"] } }
```

#### `zerogpu_generate_followups`

Generate follow-up, clarifying, or suggested next questions about a passage: articles, interviews, reports, briefs, customer messages.

* **Backed by:** `llama-3.1-8b-instruct-fast`.

| Name | Required | Default | Description |
| - | - | - | - |
| `text` | yes | - | The passage to generate questions about (non-empty). |

Returns a list of suggested questions.

```json theme={null}
{ "name": "zerogpu_generate_followups", "arguments": { "text": "Our Q3 churn rose to 4.1%, mostly in the SMB segment after the price change." } }
```

#### `zerogpu_moderate`

Check text against the 13 standard safety categories before it is published, stored, forwarded, or acted on: user-generated content, comments, support messages, prompts, ad creative, or anything from an untrusted source. Cheaper and more consistent than asking the host model to judge the content itself.

* **Backed by:** `zlm-v1-moderation-edge`, on the [Moderations](/api-reference/moderations) endpoint.

| Name | Required | Default | Description |
| - | - | - | - |
| `input` | yes | - | Text to classify. A single string, or an array of strings for one verdict per element. |

Returns `any_flagged` for a quick gate, a `results` array with one entry per input (each carrying `flagged`, the `flagged_categories` that tripped with their scores, and the full `category_scores` map), and the `model` used. This endpoint reports no token usage, so no `savings` block is returned.

```json theme={null}
{ "name": "zerogpu_moderate", "arguments": { "input": "I will find you and make you regret this." } }
```

#### `zerogpu_embed`

Turn text into vectors for semantic search, RAG retrieval, clustering, deduplication, or similarity scoring.

* **Backed by:** `all-minilm-l6-v2` (`bge-small-en-v1.5` when `model` is set), on the [Embeddings](/api-reference/embeddings) endpoint.

| Name | Required | Default | Description |
| - | - | - | - |
| `input` | yes | - | Text to embed. A single string, or an array of strings for one vector per element. |
| `model` | optional | `all-minilm-l6-v2` | Either `all-minilm-l6-v2` (22.7M, fastest) or `bge-small-en-v1.5` (33M, 512-token context, stronger on retrieval). |

Returns an `embeddings` array (each entry an `index` and its `embedding`), the `count`, the vector `dimensions`, and `usage`. Both models return 384 dimensions, so they are interchangeable in an existing index, but don't mix them within one index. Embeddings bill on input tokens only.

```json theme={null}
{ "name": "zerogpu_embed", "arguments": { "input": ["ZeroGPU runs inference at the edge.", "Frontier models are expensive."] } }
```

#### `zerogpu_chat`

Short, self-contained generative reply for quick rephrasing, simple Q\&A, drafting a short message, or light brainstorming, anything that does not need code, workspace context, or multi-step reasoning. Prefer this over answering the turn with the host model when host-model capability isn't required.

* **Backed by:** `LFM2.5-1.2B-Instruct` (`LFM2.5-1.2B-Thinking` when `thinking=true`).

| Name | Required | Default | Description |
| - | - | - | - |
| `messages` | yes | - | Array of `{ role, content }` turns. `role` is one of `system`, `user`, `assistant`, `tool`. |
| `thinking` | optional | `false` | Route to the Thinking variant; the reasoning trace is returned separately. |
| `model` | optional | `LFM2.5-1.2B-Instruct` | Override the model id (see the [model catalog](/docs/model-catalog)). |
| `max_tokens` | optional | model default | Upper bound on the reply, `1` to `4096`. |
| `temperature` | optional | model default | Sampling temperature in `[0, 2]`. |

Returns the assistant reply (plus a separate reasoning trace when `thinking=true`).

```json theme={null}
{
  "name": "zerogpu_chat",
  "arguments": { "messages": [{ "role": "user", "content": "Explain WebSockets in two sentences." }] }
}
```

### Example: the `zerogpu_summarize` tool

The simplest end-to-end call is summarization. In an MCP client, just ask the host model to summarize a passage and it auto-invokes `zerogpu_summarize`. To call it explicitly over the raw transport, send a `tools/call` request with the session id and auth headers from the handshake:

```bash theme={null}
curl -sS https://mcp.zerogpu.ai/mcp \
  -H "Content-Type: application/json" \
  -H "Accept: application/json, text/event-stream" \
  -H "x-api-key: $ZEROGPU_API_KEY" \
  -H "x-project-id: $ZEROGPU_PROJECT_ID" \
  -H "mcp-session-id: <SESSION_ID>" \
  -d '{
    "jsonrpc": "2.0",
    "id": 3,
    "method": "tools/call",
    "params": {
      "name": "zerogpu_summarize",
      "arguments": {
        "text": "The board met Thursday to review Q3 results. Revenue rose 18% year-over-year to $42M, driven mainly by enterprise renewals and a strong launch in the EU market. Operating margin slipped to 11% from 14% as headcount grew 30% ahead of the new data-center buildout. The CFO flagged rising cloud costs as the top risk for Q4 and proposed a hiring freeze on non-engineering roles until margins recover.",
        "max_tokens": 120
      }
    }
  }'
```

The tool result is a text content block whose body parses to:

```json theme={null}
{
  "summary": "The board reviewed Q3 results, with revenue increasing 18% to $42M due to enterprise renewals and a strong EU launch, but operating margin dropped to 11% from 14% due to a 30% headcount growth ahead of the data-center buildout. The CFO identified rising cloud costs as a top risk for Q4 and proposed a hiring freeze on non-engineering roles until margins recover.",
  "model": "llama-3.1-8b-instruct-fast",
  "usage": {
    "prompt_tokens": 158,
    "completion_tokens": 91,
    "total_tokens": 249
  },
  "savings": {
    "zerogpu_cost_usd": 0.000034,
    "baseline_cost_usd": 0.001839,
    "savings_usd": 0.001805,
    "price_table_version": "2026-05-26-2"
  }
}
```

Every generative tool returns a `usage` and `savings` block like this, so you can measure exactly how much offloading the task to ZeroGPU saved versus running it on a frontier model.

### Patterns and recipes

**Sanitize before the host model sees raw input.** Pipe untrusted text through `zerogpu_redact_pii` first when you don't want personal data captured in the host model's transcript or sent to downstream LLMs. Combine with `zerogpu_extract_pii` if you also need an audit log of what was masked.

**Cheap router in front of the host model.** Use `zerogpu_classify_zero_shot` or `zerogpu_classify_structured` to triage an incoming message (bug / feature / question, urgent / normal, in-scope / out-of-scope) and only escalate the hard cases to the host model. The classifier call costs orders of magnitude less than a full host-model turn.

**Structured extraction over free-form parsing.** For semi-structured text (signatures, invoices, contact blocks), prefer `zerogpu_extract_json` over asking the host model to "parse this into JSON." It's deterministic on the schema, faster, and cheaper. Keep field descriptions short and specific, since the description is what the model uses to find the span.

**Health-check before a batch.** Call `zerogpu_health` once before fanning out a large run of tool calls. If a component reports an error, back off rather than failing each call individually.

### Tool reference table

Every tool at a glance.

| Tool | Purpose | Required arguments |
| - | - | - |
| `zerogpu_health` | Confirm the orchestration API is reachable | *(none)* |
| `zerogpu_summarize` | Summarize / TL;DR a text passage | `text` |
| `zerogpu_classify_iab` | IAB ad-tech taxonomy classification | `text` |
| `zerogpu_classify_domain` | IAB classification from a bare hostname | `domain` |
| `zerogpu_classify_zero_shot` | Score text against a flat label list | `text`, `labels` |
| `zerogpu_classify_structured` | Multi-axis classification by schema | `text`, `schema` |
| `zerogpu_extract_entities` | Named-entity recognition | `text`, `labels` |
| `zerogpu_extract_json` | Extract structured fields into JSON | `text`, `schema` |
| `zerogpu_redact_pii` | Mask / scrub PII in text | `text` |
| `zerogpu_extract_pii` | Detect PII grouped by category | `text` |
| `zerogpu_generate_followups` | Suggest follow-up questions | `text` |
| `zerogpu_moderate` | Safety-check text across 13 categories | `input` |
| `zerogpu_embed` | Embed text as 384-dim vectors | `input` |
| `zerogpu_chat` | Short small-model chat reply | `messages` |

## Troubleshooting

**`missing required headers: x-api-key and x-project-id`** - the request reached the server without the required auth header. Make sure your client config sets `x-api-key` (and `x-project-id` if you scope requests to a project), and that it's sent on the `initialize` call as well as every later request.

**`Request failed with status 401`** - your API key is missing, revoked, or mistyped. Rotate the key in the [dashboard](https://platform.zerogpu.ai/dashboard) and update your client config. Keys must start with `zgpu-api-`.

**`Request failed with status 403`** - the key is valid but doesn't have access to the project, or the project doesn't have access to a requested model. Confirm your `x-project-id` matches the project that owns the key.

**`Request failed with status 429`** - you're being rate-limited. Back off and retry with exponential delay, or move bulk workloads to the [Batch API](/docs/batch), which has separate quotas tuned for high-volume jobs.

**Tools don't appear in the client.** The client never completed the MCP handshake. Confirm the transport is set to streamable HTTP (`type: "http"`), the URL is exactly `https://mcp.zerogpu.ai/mcp`, and the client supports remote MCP servers. Restart or reload the client after editing its config.

**`Bad Request` or session errors on follow-up calls.** Every call after `initialize` must include the `mcp-session-id` returned by the handshake. If you're driving the transport by hand, capture that response header and resend it (along with the auth headers) on each request.

**Empty or low-confidence extraction / classification.** Lower `threshold` to surface more candidates, or check that your label set matches the language of the source text (the underlying models are English-tuned for most label sets). Very short inputs (one or two words) yield lower confidence across the board.

**Schema-shaped tools reject the arguments.** `zerogpu_extract_json` and `zerogpu_classify_structured` expect `schema` as a JSON object whose values are arrays. For `zerogpu_extract_json`, each array item is a `field::type::desc` string; for `zerogpu_classify_structured`, each array is a list of allowed labels. A flat array or a stringified schema will be rejected.

**`zerogpu_chat` returns an empty or off-topic reply.** It's backed by a 1.2B model and is meant for short, self-contained turns. For anything needing code, workspace context, or multi-step reasoning, keep the work on the host model. Set `thinking=true` for short logic or math turns where a reasoning trace helps.

**Wrong tool auto-invoked.** The host model picks based on phrasing. Rephrase to remove trigger words ("summarize", "classify", "redact", "extract"), or call the intended tool explicitly through your client's tool interface.

## Conclusion

The ZeroGPU MCP server turns every edge model into a first-class tool that any MCP client can call, so a host model can hand off summarization, classification, extraction, and short chat to a cheaper, faster model without leaving the conversation. It's a fast way to keep raw PII out of the host model's context, cut token spend on well-defined NLP work, and measure the savings on every call.

<CardGroup>
  <Card title="Model Catalog" icon="layer-group" href="/docs/model-catalog">
    Browse every model the MCP tools route to.
  </Card>

  <Card title="API Reference" icon="code" href="/api-reference/responses">
    Explore the full OpenAI-compatible API surface.
  </Card>

  <Card title="Cookbook" icon="book" href="/cookbook/index">
    Worked examples for classification, extraction, and batch jobs.
  </Card>

  <Card title="Join Discord" icon="discord" href="https://discord.gg/58XZuxKyq6">
    Ask questions and share what you're building.
  </Card>
</CardGroup>


# OpenClaw
Source: https://docs.zerogpu.ai/integrations/openclaw

Route your OpenClaw agent's routine tasks through ZeroGPU's nano language models with the ZeroGPU Router plugin.

OpenClaw is an agentic runtime that runs an AI agent with direct access to your shell, your files, and a plugin system for extending what the agent can do. Plugins install from the ClawHub marketplace with a single `openclaw plugins install` command, and each plugin bundles skills - Markdown instruction files the agent reads and auto-invokes whenever a request matches, or that you trigger by name. Because skills shell out through the agent's built-in Bash tools, a plugin can wire in any local CLI without hosting a server or registering an API. Teams use OpenClaw to automate multi-step work while keeping the heavy reasoning on their primary model and offloading the cheap, repetitive steps elsewhere.

ZeroGPU is an ultra-fast, compute-efficient inference provider for apps and agents. We run purpose-built small and nano language models across an edge-powered network for the high-volume, purpose-specific tasks your app or agent runs constantly. Plug in our OpenAI-compatible API and you're live - zero GPU infrastructure, serverless, auto-scaling by default.

## Overview

This guide walks through installing the `zerogpu-router` plugin, which turns every ZeroGPU CLI command into an OpenClaw skill your agent can call on its own. You'll install the `zerogpu` CLI, authenticate once, add the plugin from ClawHub, and try both auto-invoked and manual skills. By the end your OpenClaw agent can hand off cheap, well-defined NLP work - classification, entity and PII extraction, summarization, and short chat - to ZeroGPU's edge-optimized models, cutting token spend without changing the model that does your agent's reasoning.

## Cookbook

For a runnable, real-world example, see [Build a PII-Safe Inbox Summarizer with Your OpenClaw Assistant](/cookbook/openclaw-inbox-summarizer). It walks through saving a triage workflow to your assistant once, then having it redact PII, summarize, and label every message you forward automatically, all on ZeroGPU's nano models.

<Note>
  For more worked examples, browse the [cookbook index](/cookbook/index) for
  end-to-end walkthroughs of classification, extraction, and PII workflows on
  ZeroGPU's small models.
</Note>

## Video walkthrough

Watch the `redact-pii` skill mask PII in-line inside an OpenClaw agent, live:

<iframe title="Redact PII On-Device with ZeroGPU's Nano Models" />

## Quickstart

### Prerequisites

* An OpenClaw install with plugin API `2026.4.0` or newer.
* Node.js 20 or newer. The `zerogpu` CLI that every skill shells out to runs on Node.
* A ZeroGPU [API key](https://platform.zerogpu.ai/dashboard) (starts with `zgpu-api-`).
* Optional: a model ID from the [model catalog](/docs/model-catalog) if you want to call a model by name. The skills already route each task to the right model, so this is only needed for custom work.

### Get your ZeroGPU API key

1. Sign in to the [ZeroGPU dashboard](https://platform.zerogpu.ai/dashboard).
2. Open **API Keys** and click **Create key**.
3. Copy the key (it starts with `zgpu-api-`). No Project ID is required - `zerogpu login` takes just the key.

### Install the plugin

The plugin does not provision the CLI on its own, so install and log in to `zerogpu` first, then add the plugin from ClawHub:

```bash theme={null}
# 1. Install the CLI the skills shell out to
npm install -g zerogpu-cli

# 2. Log in (prompts for your API key)
zerogpu login

# 3. Install the plugin from ClawHub
openclaw plugins install clawhub:zerogpu-router
```

Pin a specific release with `clawhub:zerogpu-router@2.1.2`. In sandboxed or Docker-based agents, make sure the `zerogpu` binary is present inside the container the agent actually runs in, not just on your host.

### Your first request

Ask your agent in plain language. It picks the matching skill, runs the `zerogpu` CLI locally instead of the host model, and replies with the result:

```text theme={null}
redact the PII in this note before I share it:
"Email John Smith at john@acme.com about invoice 12345."
```

Reply:

```text theme={null}
Email [PERSON] at [EMAIL] about invoice 12345.

model: gliner2.5-multi-v1 · 2 spans masked
```

The raw PII never reaches your primary reasoning model - the small model does the masking, and only the scrubbed text comes back.

## Usage

Every inference skill auto-invokes when your OpenClaw agent detects a matching request, so you rarely name a skill directly: describe the task and the agent picks the route. Three skills (`signin`, `status`, and `cost-savings`) are manual-only and never auto-invoked - you trigger them yourself. This section covers every skill in the plugin: what it does, which ZeroGPU model it routes to, the CLI command it wraps, the arguments it accepts, and the shape of the reply you get back. Each skill returns a structured payload of `{ <task fields>, model, usage, savings }`, which the agent renders as the result plus a trailing `model: <name> · <usage>` line.

#### How the agent picks a skill

Each skill ships with a description the OpenClaw agent reads when deciding whether to invoke it for a given message. You don't have to remember skill names: just phrase the task. The agent keys off intent words like "summarize", "redact", "scrub", "classify by", "extract", or "tag this as", and off the structure of the data you supply (a flat label list implies zero-shot classification; a JSON schema with category axes implies structured classification). Arbitrary user text is passed to the CLI verbatim through a heredoc, so newlines, quotes, parentheses, and other shell metacharacters in your input are handled safely.

Occasionally a reply ends with a savings line:

```text theme={null}
💰 ZeroGPU savings so far: ≈ $2 (18,730 frontier-model tokens offloaded)
```

This surfaces intermittently, not on every call, so a healthy total accumulates quietly in the background. To see the running figure on demand, use the `cost-savings` skill.

If the agent picks the wrong skill for a request, rephrase to drop the ambiguous trigger word, or ask for the specific skill by name.

### Chat and summarization

#### chat

Short, single-turn chat reply for things that don't need host-level reasoning or prior conversation context. Use it when you'd otherwise burn primary-model tokens on a one-liner.

* **Model:** `LFM2.5-1.2B-Instruct`
* **Wraps:** `zerogpu chat`
* **Auto-invokes on:** quick factual answers, one-liners, and basic rephrasings, especially when you've signalled "use a small model."

| Argument | Required | Description |
| - | - | - |
| `text` | yes | The prompt to answer. |
| `-i`, `--instructions <text>` | optional | System instructions to steer tone or behavior. |

**Example prompt**

```text theme={null}
using a small model, explain WebSockets in two sentences like a concise technical writer
```

**Illustrative reply**

```text theme={null}
WebSockets keep a single TCP connection open so a client and server can send messages
to each other at any time, without repeating the HTTP request-response handshake. That
makes them a good fit for live features like chat, notifications, and streaming updates.

model: LFM2.5-1.2B-Instruct · 24 tokens in / 46 out
```

#### chat-thinking

Same shape as `chat`, but the model returns its reasoning trace alongside the answer.

* **Model:** `LFM2.5-1.2B-Thinking`
* **Wraps:** `zerogpu chat_thinking`
* **Auto-invokes on:** short logic, math, or word-problem questions where step-by-step reasoning is useful or explicitly requested.

**Example prompt**

```text theme={null}
show your reasoning: if a train leaves at 3 PM going 60 mph, when does it cover 150 miles?
```

The reply includes the intermediate reasoning steps before the final answer (5:30 PM).

#### summarize

Condense a longer passage into a short summary. Use it when you need the gist of a report, ticket thread, transcript, or document without spending primary-model tokens on the read.

* **Model:** `llama-3.1-8b-instruct-fast`
* **Wraps:** `zerogpu summarize`
* **Auto-invokes on:** "summarize this", "give me the gist", "TL;DR this", "condense this into a few sentences."

**Example prompt**

```text theme={null}
summarize this: The board met Thursday to review Q3 results. Revenue rose 18%
year-over-year to $42M, driven mainly by enterprise renewals and a strong launch in
the EU market. Operating margin slipped to 11% from 14% as headcount grew 30% ahead of
the new data-center buildout. The CFO flagged rising cloud costs as the top risk for Q4
and proposed a hiring freeze on non-engineering roles until margins recover. The board
approved the freeze and asked for a revised 2025 budget by mid-December.
```

**Illustrative reply**

```text theme={null}
Q3 revenue grew 18% YoY to $42M on enterprise renewals and EU growth, but operating
margin fell to 11% due to a 30% headcount increase for the data-center buildout. Citing
cloud costs as the main Q4 risk, the board approved a hiring freeze on non-engineering
roles and requested a revised 2025 budget by mid-December.

model: llama-3.1-8b-instruct-fast · 118 tokens in / 61 out
💰 ZeroGPU savings so far: ≈ $2 (18,730 frontier-model tokens offloaded)
```

For a file, run the CLI directly: `zerogpu summarize "$(cat article.txt)"`.

### Classification

#### classify-iab

Classify text against the **IAB content and audience taxonomy** (standard ad-tech category labels).

* **Model:** `zlm-v1-iab-classify-edge`
* **Wraps:** `zerogpu classify_iab`
* **Auto-invokes on:** "what IAB category is this?", "tag this article for ad targeting", "give me the topic taxonomy."

**Example prompt**

```text theme={null}
what IAB category is this: The Lakers signed a new point guard ahead of the playoffs.
```

**Illustrative reply**

```json theme={null}
{
	"categories": [{ "id": "IAB17-44", "name": "Basketball", "confidence": 0.97 }]
}
```

#### classify-iab-enriched

Enriched IAB classification that returns audience categories **plus** topics, keywords, and inferred user intent.

* **Model:** `zlm-v2-iab-classify-edge-enriched`
* **Wraps:** `zerogpu classify_iab_enriched`
* **Auto-invokes on:** "give me topics, keywords, and intent", any request for richer ad or audience signals than plain IAB labels.

**Example prompt**

```text theme={null}
give me topics, keywords, and intent for: Compare the Tesla Model Y and the Hyundai
Ioniq 5 for a family of four.
```

**Illustrative reply**

```json theme={null}
{
	"categories": [{ "id": "IAB2-1", "name": "Auto Buyers", "confidence": 0.92 }],
	"topics": ["electric vehicles", "family cars"],
	"keywords": ["Tesla Model Y", "Hyundai Ioniq 5"],
	"intent": "comparison-shopping"
}
```

#### classify-zero-shot

Zero-shot classification against an arbitrary list of candidate labels you supply at call time. The model scores every candidate and returns the best fit.

* **Model:** `deberta-v3-small`
* **Wraps:** `zerogpu classify_zero_shot`
* **Auto-invokes on:** "is this positive, negative, or neutral?", "tag this as bug, feature, or question", any ask that names its own flat label set.

| Argument | Required | Default | Description |
| - | - | - | - |
| `text` | yes | - | Text to classify. |
| `-l <label>` / `--labels <a,b,c>` | yes (one) | - | Candidate labels. Repeat `-l` or pass a comma-separated `--labels` list. |
| `-t`, `--threshold <number>` | optional | - | Confidence floor in `[0, 1]` for multi-label output. |

**Example prompt**

```text theme={null}
classify this as billing, bug, feature-request, or account:
"I was charged twice for my October subscription, and the second charge is a different
amount than my plan. Please refund the duplicate and explain the difference."
```

**Illustrative reply**

```json theme={null}
[
	{ "label": "billing", "score": 0.94 },
	{ "label": "account", "score": 0.29 },
	{ "label": "bug", "score": 0.07 },
	{ "label": "feature-request", "score": 0.02 }
]
```

```text theme={null}
model: deberta-v3-small · 61 tokens in
💰 ZeroGPU savings so far: ≈ $2 (19,180 frontier-model tokens offloaded)
```

#### classify-structured

Schema-driven, multi-axis classification. You define each category and its allowed labels, and the model returns one chosen label per category.

* **Model:** `gliner2.5-base-v1`
* **Wraps:** `zerogpu classify_structured`
* **Auto-invokes on:** "classify by sentiment and topic", any request that names multiple classification dimensions with explicit label sets.

| Argument | Required | Description |
| - | - | - |
| `text` | yes | Text to classify. |
| `-s`, `--schema <json>` | yes | Single-quoted JSON object mapping each axis to its allowed labels. |

**Example prompt**

```text theme={null}
classify this by sentiment and topic - "Support replied quickly but the fix didn't work"
```

The agent synthesizes a schema such as `{"sentiment":["positive","negative","neutral"],"topic":["support","billing","product"]}` and calls the skill.

**Illustrative reply**

```json theme={null}
{ "sentiment": "negative", "topic": "support" }
```

### Extraction

#### extract-entities

Custom-label named-entity recognition. You define the entity labels; the model finds matching spans with confidence scores.

* **Model:** `gliner2.5-base-v1`
* **Wraps:** `zerogpu extract_entities`
* **Auto-invokes on:** "extract all people, organizations, and locations", "find every product mention."

| Argument | Required | Default | Description |
| - | - | - | - |
| `text` | yes | - | Source text. |
| `-l <label>` / `--labels <a,b,c>` | yes (one) | - | Entity labels to extract. |
| `-t`, `--threshold <number>` | optional | `0.3` | Minimum confidence in `[0, 1]`. |

**Example prompt**

```text theme={null}
extract the people, organizations, and locations from:
"Apple CEO Tim Cook met with Sundar Pichai in Cupertino on Monday."
```

**Illustrative reply**

```json theme={null}
[
	{ "label": "organization", "text": "Apple", "score": 0.98 },
	{ "label": "person", "text": "Tim Cook", "score": 0.97 },
	{ "label": "person", "text": "Sundar Pichai", "score": 0.96 },
	{ "label": "location", "text": "Cupertino", "score": 0.91 }
]
```

#### extract-json

Pull specific named fields out of free text into a structured JSON object, defined by a per-field schema. Each field is declared as `name::type::description`.

* **Model:** `gliner2.5-base-v1`
* **Wraps:** `zerogpu extract_json`
* **Auto-invokes on:** "extract the contact info as JSON", "parse this invoice", "pull these fields out."

| Argument | Required | Description |
| - | - | - |
| `text` | yes | Source text. |
| `-s`, `--schema <json>` | yes | Single-quoted JSON. Each field is `name::type::description`, grouped under a key. |

**Example prompt**

```text theme={null}
pull the name, email, and phone out of this as JSON:
"Reach Maria Lopez at maria.lopez@acme.io or 415-555-0188."
```

The agent builds a schema like `{"contact":["name::str::Full name","email::str::Email address","phone::str::Phone number"]}`.

**Illustrative reply**

```json theme={null}
{
	"contact": {
		"name": "Maria Lopez",
		"email": "maria.lopez@acme.io",
		"phone": "415-555-0188"
	}
}
```

### PII handling

#### extract-pii

Extract personally identifiable information entities, grouped by category, **without modifying the source text**. Use it when you need structured data about PII (for redaction policies, audits, or downstream tooling) rather than a masked version.

* **Model:** `gliner2.5-multi-v1`
* **Wraps:** `zerogpu extract_pii`
* **Auto-invokes on:** "find all PII", "what personal info is in this?", "list emails, phones, and names."

| Argument | Required | Default | Description |
| - | - | - | - |
| `text` | yes | - | Source text. |
| `-t`, `--threshold <number>` | optional | `0.5` | Minimum confidence. |
| `-c`, `--categories <list>` | optional | `identity,contact` | Comma-separated. Other values: `financial`, `medical`, `credentials`. |

**Example prompt**

```text theme={null}
find the PII in this, including financial:
"Contact Jane Doe at jane@example.com or +1 (415) 555-1212."
```

**Illustrative reply**

```json theme={null}
[
	{
		"category": "identity",
		"label": "person",
		"text": "Jane Doe",
		"score": 0.96
	},
	{
		"category": "contact",
		"label": "email",
		"text": "jane@example.com",
		"score": 0.99
	},
	{
		"category": "contact",
		"label": "phone",
		"text": "+1 (415) 555-1212",
		"score": 0.95
	}
]
```

To mask PII in-line rather than extract it, use `redact-pii`.

#### redact-pii

Detect PII and replace each span in-line with a `[LABEL]` placeholder. Use it before sharing or logging sensitive text, or before forwarding user input to another model you don't want to expose raw PII to.

* **Model:** `gliner2.5-multi-v1`
* **Wraps:** `zerogpu redact_pii`
* **Auto-invokes on:** "redact", "scrub", "mask", "anonymize", or "sanitize this for sharing."

**Example prompt**

```text theme={null}
redact the PII in this CRM note before I share it with the vendor -
"Call with Daniel Okafor (daniel.okafor@brightwave.io, +1 206-555-0147) on Tue.
Send the enterprise quote to their AP team at 400 Pine St, Seattle, WA 98101.
Card on file ends 4412 - do not reference it in email."
```

**Illustrative reply**

```text theme={null}
Call with [PERSON] ([EMAIL], [PHONE_NUMBER]) on Tue. Send the enterprise quote to their
AP team at [ADDRESS]. Card on file ends 4412 - do not reference it in email.

model: gliner2.5-multi-v1 · 5 spans masked
```

The card last-four is left untouched: only spans the model recognizes as PII are replaced. Project-specific identifiers (account numbers, internal ticket IDs) should be handled by your own redaction layer or by `extract-entities` with custom labels.

### Account and savings

These three skills are **manual-only**. The agent never auto-invokes them - trigger them yourself when you need them.

#### signin

Sign in to ZeroGPU and persist your API key so every subsequent skill call works without re-prompting.

* **Wraps:** `zerogpu login`

| Flag | Required | Description |
| - | - | - |
| `--api-key <key>` | optional | API key (must start with `zgpu-api-`). If omitted, you're prompted interactively (masked). |

With no arguments the CLI prompts for the key. For non-interactive setups (CI), pass `--api-key zgpu-api-…`. On success the key is written to the local config file and `ZEROGPU_API_KEY` is upserted into your shell profile so other tools can pick it up. Ask your agent to sign you in, or run `zerogpu login` directly in your terminal.

#### status

Show your current ZeroGPU sign-in status and the masked API key.

* **Wraps:** `zerogpu status`

Exit code is `0` when signed in, `1` when not - handy as a hard fail in scripts before running any inference skill. Ask "am I signed in to ZeroGPU?" or run `zerogpu status`.

#### cost-savings

Show how much you've saved by routing tasks to ZeroGPU instead of the host model - cumulative dollars and tokens offloaded.

* **Wraps:** `zerogpu cost_savings`

Token counts are actual (from the API); the dollar figure estimates what the same work would have cost on the host model. The baseline defaults to `claude-opus-4-8` and is overridable with the `ZEROGPU_SAVINGS_MODEL` env var. Pass `--json` for raw data or `--reset` to clear the history. Ask "how much have I saved with ZeroGPU?" to see the summary.

### Skills reference

Every skill at a glance.

| Skill | Workload | Backing model | Invocation |
| - | - | - | - |
| `chat` | Short small-model chat replies | `LFM2.5-1.2B-Instruct` | auto |
| `chat-thinking` | Short chat with a visible reasoning trace | `LFM2.5-1.2B-Thinking` | auto |
| `summarize` | TL;DRs, abstracts, meeting summaries | `llama-3.1-8b-instruct-fast` | auto |
| `classify-iab` | IAB topic classification | `zlm-v1-iab-classify-edge` | auto |
| `classify-iab-enriched` | IAB categories plus topics, keywords, intent | `zlm-v2-iab-classify-edge-enriched` | auto |
| `classify-zero-shot` | Classify against a flat label list | `deberta-v3-small` | auto |
| `classify-structured` | Multi-axis schema classification | `gliner2.5-base-v1` | auto |
| `extract-entities` | Custom-label named-entity recognition | `gliner2.5-base-v1` | auto |
| `extract-json` | Pull structured fields into grouped JSON | `gliner2.5-base-v1` | auto |
| `extract-pii` | Extract PII grouped by category | `gliner2.5-multi-v1` | auto |
| `redact-pii` | Mask PII in-line with `[LABEL]` placeholders | `gliner2.5-multi-v1` | auto |
| `signin` | Sign in and persist your API key | - | manual |
| `status` | Show current sign-in status | - | manual |
| `cost-savings` | Cumulative dollars and tokens offloaded | - | manual |

For the full flag reference on any command, run `zerogpu <command> --help` in your terminal outside the agent session.

### Patterns and recipes

**Sanitize before the host model sees raw input.** Ask the agent to `redact-pii` untrusted text first when you don't want personal data captured in your primary model's transcript or forwarded to a downstream LLM. Pair it with `extract-pii` when you also need an audit log of exactly what was masked.

**Cheap router in front of your reasoning model.** Use `classify-zero-shot` or `classify-structured` to triage an incoming message (bug / feature / question, urgent / normal, in-scope / out-of-scope) and only escalate the hard cases to your primary model. The classifier call costs orders of magnitude less than a full reasoning turn.

**Structured extraction over free-form parsing.** For semi-structured text (signatures, invoices, contact blocks), prefer `extract-json` over asking your host model to "parse this into JSON." It's deterministic on the schema, faster, and cheaper. Keep field descriptions short and specific - the description is what the model uses to locate the span.

**Tune thresholds for the job.** For NER and PII extraction the defaults (`0.3` and `0.5`) favor recall. Raise `-t` to `0.6` or higher when you need precision (compliance-grade redaction lists); lower it when you'd rather over-extract and filter downstream.

## Troubleshooting

**`zerogpu: command not found`** - the CLI isn't installed or isn't on your `PATH`. Run `npm install -g zerogpu-cli` and restart your shell. If you use a Node version manager (nvm, fnm, volta), make sure the shell that launched OpenClaw has the same Node version active.

**A sandboxed or Docker agent can't find `zerogpu`** - the CLI has to exist inside the container the agent runs in, not just on your host. Add `npm install -g zerogpu-cli` to your image build, or mount the binary into the sandbox.

**Skill returns "You're not signed in yet."** - no credentials on disk. Run the `signin` skill, or `zerogpu login` in your terminal, then confirm with the `status` skill (`zerogpu status`).

**The agent won't use the plugin's skills** - the plugin may not be installed or enabled. Re-run `openclaw plugins install clawhub:zerogpu-router` and confirm it's enabled in your OpenClaw plugin settings. Pin a known-good release with `clawhub:zerogpu-router@2.1.2` if a newer build misbehaves.

**`Request failed with status 401`** - your API key is missing, revoked, or mistyped. Rotate the key in the [dashboard](https://platform.zerogpu.ai/dashboard) and re-run the `signin` skill. Keys must start with `zgpu-api-`.

**`Request failed with status 403`** - the key is valid but doesn't have access to the requested project or model. Confirm the key belongs to a project that owns the model you're routing to.

**`Request failed with status 429`** - you're being rate-limited. Back off and retry with exponential delay, or move heavy workloads to the [Batch API](/docs/batch), which has separate quotas tuned for bulk jobs.

**The wrong skill was auto-invoked** - the agent picks based on phrasing. Rephrase to drop the ambiguous trigger word ("redact", "classify", "extract"), or ask for the specific skill by name.

**Schema quoting errors on `classify-structured` / `extract-json`** - the `-s` flag expects a single-quoted JSON string. On Windows PowerShell, escape inner double quotes or use a here-string. Run the CLI directly (`zerogpu classify_structured … -s '…'`) to isolate quoting issues from the agent.

**Empty or low-confidence results** - lower `-t` to surface more candidates, or check that the labels match the language of the source text (the underlying models are English-tuned for most label sets). Very short inputs return lower confidence across the board.

**No savings line appears** - it's intentionally occasional, not shown on every call. Trigger the `cost-savings` skill (`zerogpu cost_savings`) any time to see the cumulative dollars and tokens offloaded.

## Conclusion

The `zerogpu-router` plugin turns ZeroGPU's nano language models into first-class OpenClaw skills, so your agent can hand off classification, extraction, and short chat to a cheaper, faster model without changing the model that does its reasoning. It's a fast way to keep raw PII out of your primary model's context, cut token spend on well-defined NLP work, and watch the savings add up per call.

<CardGroup>
  <Card title="Model Catalog" icon="layer-group" href="/docs/model-catalog">
    Browse every model the plugin can route to.
  </Card>

  <Card title="API Reference" icon="code" href="/api-reference/responses">
    Explore the full OpenAI-compatible API surface.
  </Card>

  <Card title="Cookbook" icon="book" href="/cookbook/index">
    Worked examples for classification, extraction, and batch jobs.
  </Card>

  <Card title="Join Discord" icon="discord" href="https://discord.gg/58XZuxKyq6">
    Ask questions and share what you're building.
  </Card>
</CardGroup>


# Go
Source: https://docs.zerogpu.ai/integrations/sdk/go

Integrate ZeroGPU into your Go application.

## Basic usage

```go theme={null}
package main

import (
  "bytes"
  "encoding/json"
  "fmt"
  "io"
  "net/http"
)

func main() {
  url := "https://api.zerogpu.ai/v1/responses"

  payload := map[string]any{
    "model": "YOUR_MODEL",
    "input": []map[string]string{
      {
        "role":    "user",
        "content": "Your input text here...",
      },
    },
    "text": map[string]any{
      "format": map[string]string{
        "type": "text",
      },
    },
  }

  payloadBytes, err := json.Marshal(payload)
  if err != nil {
    panic(err)
  }

  req, err := http.NewRequest("POST", url, bytes.NewBuffer(payloadBytes))
  if err != nil {
    panic(err)
  }

  req.Header.Set("content-type", "application/json")
  req.Header.Set("x-api-key", "YOUR_API_KEY")
  req.Header.Set("x-project-id", "YOUR_PROJECT_ID")

  client := &http.Client{}
  res, err := client.Do(req)
  if err != nil {
    panic(err)
  }
  defer res.Body.Close()

  body, _ := io.ReadAll(res.Body)
  fmt.Println(string(body))
}
```

## Using environment variables

```go theme={null}
import "os"

apiKey := os.Getenv("ZEROGPU_API_KEY")
projectID := os.Getenv("ZEROGPU_PROJECT_ID")

req.Header.Set("x-api-key", apiKey)
req.Header.Set("x-project-id", projectID)
```

## Error handling

```go theme={null}
res, err := client.Do(req)
if err != nil {
  log.Fatalf("Request failed: %v", err)
}
defer res.Body.Close()

if res.StatusCode != http.StatusOK {
  body, _ := io.ReadAll(res.Body)
  log.Fatalf("API error %d: %s", res.StatusCode, string(body))
}
```


# JavaScript
Source: https://docs.zerogpu.ai/integrations/sdk/javascript

Integrate ZeroGPU into your JavaScript or Node.js application.

## Official SDK (recommended)

Install the published client from [npm](https://www.npmjs.com/package/zerogpu-api):

```bash theme={null}
npm install zerogpu-api
```

```javascript theme={null}
import { ZerogpuApiClient } from 'zerogpu-api';

const client = new ZerogpuApiClient({
  apiKey: process.env.ZEROGPU_API_KEY,
  projectId: process.env.ZEROGPU_PROJECT_ID,
});

// Many models accept plain string `input`; others accept a message list: see API reference.
const response = await client.responses.createResponse({
  model: process.env.ZEROGPU_MODEL ?? 'YOUR_MODEL',
  input: 'Your input text here...',
  text: { format: { type: 'text' } },
});

console.log(response.output);
```

Source and regeneration workflow: [zerogpu/SDK](https://github.com/zerogpu/SDK) on GitHub.

### Chat completions (`/v1/chat/completions`)

For models that use the chat route, call `client.chat.createChatCompletion({ model, messages, metadata? })`. See [Chat completions](/api-reference/chat-completions).

### Responses with `metadata`

Optional `metadata` on `createResponse` is supported for model-specific options (e.g. PII). See [Responses](/api-reference/responses).

***

## Raw HTTP (`fetch`)

If you prefer not to use the SDK, call the endpoint directly.

```javascript theme={null}
const url = 'https://api.zerogpu.ai/v1/responses';
const headers = {
  'content-type': 'application/json',
  'x-api-key': 'YOUR_API_KEY',
  'x-project-id': 'YOUR_PROJECT_ID'
};
const payload = {
  model: 'YOUR_MODEL',
  input: [
    {
      role: 'user',
      content: 'Your input text here...'
    }
  ],
  text: {
    format: {
      type: 'text'
    }
  }
};

fetch(url, {
  method: 'POST',
  headers,
  body: JSON.stringify(payload)
})
  .then(response => response.json())
  .then(data => console.log(data))
  .catch(error => console.error('Error:', error));
```

## Using environment variables (Node.js)

```javascript theme={null}
const url = 'https://api.zerogpu.ai/v1/responses';
const headers = {
  'content-type': 'application/json',
  'x-api-key': process.env.ZEROGPU_API_KEY,
  'x-project-id': process.env.ZEROGPU_PROJECT_ID
};
const payload = {
  model: 'YOUR_MODEL',
  input: [
    {
      role: 'user',
      content: 'Your input text here...'
    }
  ],
  text: {
    format: {
      type: 'text'
    }
  }
};

const response = await fetch(url, {
  method: 'POST',
  headers,
  body: JSON.stringify(payload)
});

const data = await response.json();
console.log(data.output[0].content[0].text);
```

## Error handling

```javascript theme={null}
try {
  const response = await fetch(url, {
    method: 'POST',
    headers,
    body: JSON.stringify(payload)
  });

  if (!response.ok) {
    throw new Error(`HTTP ${response.status}: ${response.statusText}`);
  }

  const data = await response.json();
  console.log(data.output[0].content[0].text);
} catch (error) {
  console.error('Request failed:', error.message);
}
```


# Python
Source: https://docs.zerogpu.ai/integrations/sdk/python

Integrate ZeroGPU into your Python application.

## Official SDK (recommended)

Install the published client from [PyPI](https://pypi.org/project/zerogpu-api/):

```bash theme={null}
pip install zerogpu-api
```

```python theme={null}
import os
from zerogpu import ZerogpuApi

client = ZerogpuApi(
    api_key=os.environ["ZEROGPU_API_KEY"],
    project_id=os.environ["ZEROGPU_PROJECT_ID"],
)

# Many models accept plain string `input`; others accept a message list: see API reference.
response = client.responses.create_response(
    model=os.environ.get("ZEROGPU_MODEL", "YOUR_MODEL"),
    input="Your input text here...",
    text={"format": {"type": "text"}},
)
print(response)
```

Source and regeneration workflow: [zerogpu/SDK](https://github.com/zerogpu/SDK) on GitHub.

### Chat completions (`/v1/chat/completions`)

For models that use the chat route:

```python theme={null}
from zerogpu import ChatMessage

client.chat.create_chat_completion(
    model="YOUR_MODEL",
    messages=[ChatMessage(role="user", content="Hello")],
)
```

See [Chat completions](/api-reference/chat-completions).

### Responses with `metadata`

Pass `metadata={...}` into `create_response` when the model supports it. See [Responses](/api-reference/responses).

***

## Raw HTTP (`requests`)

If you prefer not to use the SDK, call the endpoint directly.

```python theme={null}
import requests

url = "https://api.zerogpu.ai/v1/responses"
headers = {
    "content-type": "application/json",
    "x-api-key": "YOUR_API_KEY",
    "x-project-id": "YOUR_PROJECT_ID",
}
payload = {
    "model": "YOUR_MODEL",
    "input": [
        {
            "role": "user",
            "content": "Your input text here...",
        }
    ],
    "text": {
        "format": {
            "type": "text"
        }
    },
}

response = requests.post(url, headers=headers, json=payload)
print(response.json())
```

## Using environment variables

```python theme={null}
import os
import requests

url = "https://api.zerogpu.ai/v1/responses"
headers = {
    "content-type": "application/json",
    "x-api-key": os.environ["ZEROGPU_API_KEY"],
    "x-project-id": os.environ["ZEROGPU_PROJECT_ID"],
}
payload = {
    "model": "YOUR_MODEL",
    "input": [
        {
            "role": "user",
            "content": "Your input text here...",
        }
    ],
    "text": {
        "format": {
            "type": "text"
        }
    },
}

response = requests.post(url, headers=headers, json=payload)
result = response.json()

print(result["output"][0]["content"][0]["text"])
```

## Error handling

```python theme={null}
import requests

try:
    response = requests.post(url, headers=headers, json=payload)
    response.raise_for_status()
    result = response.json()
    print(result["output"][0]["content"][0]["text"])
except requests.exceptions.HTTPError as e:
    if e.response.status_code == 401:
        print("Invalid API key")
    else:
        print(f"Error: {e.response.status_code}")
except requests.exceptions.RequestException as e:
    print(f"Request failed: {e}")
```

## Install dependencies

```bash theme={null}
pip install requests
```


# Ruby
Source: https://docs.zerogpu.ai/integrations/sdk/ruby

Integrate ZeroGPU into your Ruby application.

## Basic usage

```ruby theme={null}
require 'net/http'
require 'uri'
require 'json'

uri = URI.parse('https://api.zerogpu.ai/v1/responses')
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = (uri.scheme == 'https')

request = Net::HTTP::Post.new(uri.request_uri)
request['content-type'] = 'application/json'
request['x-api-key'] = 'YOUR_API_KEY'
request['x-project-id'] = 'YOUR_PROJECT_ID'

request.body = {
  model: 'YOUR_MODEL',
  input: [
    {
      role: 'user',
      content: 'Your input text here...'
    }
  ],
  text: {
    format: {
      type: 'text'
    }
  }
}.to_json

response = http.request(request)
puts response.body
```

## Using environment variables

```ruby theme={null}
request['x-api-key'] = ENV['ZEROGPU_API_KEY']
request['x-project-id'] = ENV['ZEROGPU_PROJECT_ID']
```

## Error handling

```ruby theme={null}
response = http.request(request)

case response.code.to_i
when 200
  result = JSON.parse(response.body)
  puts result['output'][0]['content'][0]['text']
when 401
  puts 'Invalid API key'
else
  puts "Error #{response.code}: #{response.body}"
end
```


# Rust
Source: https://docs.zerogpu.ai/integrations/sdk/rust

Integrate ZeroGPU into your Rust application.

## Basic usage

```rust theme={null}
use reqwest::Client;
use serde_json::json;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = Client::new();

    let payload = json!({
        "model": "YOUR_MODEL",
        "input": [
            {
                "role": "user",
                "content": "Your input text here..."
            }
        ],
        "text": {
            "format": {
                "type": "text"
            }
        }
    });

    let response = client
        .post("https://api.zerogpu.ai/v1/responses")
        .header("content-type", "application/json")
        .header("x-api-key", "YOUR_API_KEY")
        .header("x-project-id", "YOUR_PROJECT_ID")
        .json(&payload)
        .send()
        .await?;

    let body = response.text().await?;
    println!("{}", body);
    Ok(())
}
```

## Dependencies (Cargo.toml)

```toml theme={null}
[dependencies]
reqwest = { version = "0.12", features = ["json"] }
serde_json = "1"
tokio = { version = "1", features = ["full"] }
```

## Using environment variables

```rust theme={null}
use std::env;

let api_key = env::var("ZEROGPU_API_KEY").expect("ZEROGPU_API_KEY not set");
let project_id = env::var("ZEROGPU_PROJECT_ID").expect("ZEROGPU_PROJECT_ID not set");

let response = client
    .post("https://api.zerogpu.ai/v1/responses")
    .header("content-type", "application/json")
    .header("x-api-key", &api_key)
    .header("x-project-id", &project_id)
    .json(&payload)
    .send()
    .await?;
```