Architecture Overview

How Tolka Edge provisions dedicated AI Rigs, routes API requests, serves models, and tracks usage.

Tolka Edge separates the customer-facing API from the infrastructure running your model. You interact with a single OpenAI-compatible endpoint while Tolka manages GPU provisioning, model serving, lifecycle state, and billing.

Request lifecycle

Client

OpenAI SDK

Tolka API

Auth + Validation

AI Rig

Dedicated GPU + vLLM

Model

Qwen3-14B

Response

Streamed back

API Gateway

Requests first reach the Tolka API gateway.

The gateway:

Authenticates your API key.
Validates the request.
Identifies your deployed AI Rig.
Forwards the request to the Rig's inference endpoint.
Streams the response back to your client.

Authentication and account data remain behind the Tolka API layer. Your application only needs the Tolka API endpoint and API key.

Dedicated AI Rigs

Each AI Rig runs on a dedicated GPU instance selected from Tolka's supported compute options.

A Rig contains:

A dedicated GPU.
The configured model runtime.
vLLM inference server.
The model and its serving configuration.

Once the Rig is running, requests are sent directly to that Rig rather than being processed on shared inference infrastructure.

Model serving

Tolka uses vLLM to serve supported open models through an OpenAI-compatible API.

For the current S-tier profile, Tolka serves Qwen3-14B-AWQ using AWQ quantization.

vLLM handles:

Request scheduling.
Continuous batching.
KV-cache management.
Token generation.
Streaming responses.

The exact concurrency and context limits depend on the selected model and GPU profile.

Streaming

Tolka forwards streaming responses from the inference server to the client as they are generated.

This allows applications to display tokens incrementally rather than waiting for the entire response.

Billing

Tolka uses pay-as-you-go GPU-time billing for dedicated Rigs.

Billing is based on the actual time your Rig is active and is tracked to the second.

The billing rate is locked when the Rig is deployed. Your usage and spending can be viewed from the dashboard.

Rig lifecycle

A typical dedicated Rig moves through these states:

Deploying

GPU infrastructure is being provisioned and the inference runtime is starting.

Loading

The model is being loaded and the inference server is becoming ready.

Running

The Rig is ready to accept API requests and active GPU time is being billed.

Terminated

The Rig has been shut down and GPU billing has stopped.

You control the Rig lifecycle from the Dedicated GPU Dashboard.

Dedicated by design

Tolka's dedicated model is designed around predictable GPU ownership rather than shared serverless execution.

Your selected Rig runs your deployed model configuration on its allocated GPU, giving you predictable access to the compute capacity assigned to that Rig.