Dedicated GPU Endpoints

Provision and manage dedicated GPU Rigs through the OpenAI-compatible API.

Use a dedicated GPU Rig through the same OpenAI-compatible API used for standard Tolka Edge inference.

A dedicated Rig is a GPU instance provisioned for your selected model and configuration. Tolka handles provisioning and model serving, while you interact with the Rig through a standard OpenAI-compatible endpoint.

Current availability

Dedicated GPU support is currently available for supported models and GPU configurations shown in the Tolka Edge dashboard.

How it works

Choose a model and GPU from the Dedicated GPU Dashboard.
Deploy your Rig. Tolka provisions the selected GPU and starts the configured inference runtime.
Wait for the Rig to become ready. The dashboard shows the current lifecycle state.
Send requests using the Rig's OpenAI-compatible endpoint.
Terminate the Rig when you no longer need the dedicated GPU.

API endpoint

Once your Rig is running, requests use the standard chat completions endpoint:

POSThttps://api.tolkaedge.com/v1/chat/completions

Example:

curl https://api.tolkaedge.com/v1/chat/completions \
  -H "Authorization: Bearer $TOLKA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen3-14B",
    "messages": [
      {
        "role": "user",
        "content": "Hello from my dedicated Tolka Edge Rig."
      }
    ]
  }'

Rig states

A dedicated Rig can move through the following states:

Status	Description
`deploying`	GPU infrastructure is being provisioned.
`loading`	The model and inference runtime are starting.
`running`	The Rig is ready to accept requests.
`terminating`	The Rig is shutting down.
`terminated`	The Rig has been stopped and GPU billing has ended.
`failed`	Provisioning or startup failed.

Dedicated capacity

While your Rig is running, the allocated GPU is dedicated to your deployment. Your model is served from that dedicated GPU rather than from shared inference capacity.

Concurrency depends on the selected model + GPU configuration. See the model configuration in the dashboard for the supported deployment limits.

Billing

Dedicated Rigs use pay-as-you-go GPU-time billing.

Billing begins when the Rig starts booting and continues while the GPU remains allocated. Usage is tracked to the second.

The applicable GPU rate is shown before deployment.

Terminating a Rig

Terminate your Rig from the Dedicated GPU Dashboard when you are finished.

Once the Rig is terminated:

GPU billing stops.
The inference endpoint is no longer available.
A new deployment is required to start another Rig.

For more information, see Rig Lifecycle.