Getting started

Introduction

Autograd begins as an inference platform for agents and grows into a reinforcement learning loop built from their real work.

Live inference preview

The endpoint, model IDs, context windows, and request limits below describe the currently available models.

Why start with inference?

Agent teams need a capable model before they need a training system. Routing inference through Autograd creates immediate value and establishes a consistent record of model calls, tool use, latency, token usage, and model versions.

The SDK adds the signals that are only visible inside your product: whether a task completed, whether a user accepted or corrected the result, and which reward or grader outcome belongs to the run.

Agent→Inference→Reward→RL→Checkpoint

Quickstart

The inference interface follows OpenAI client semantics. Both models use the same base URL and API key; select a model with the request's model field.

python

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.autograd.run/v1",
    api_key=os.environ["AUTOGRAD_API_KEY"],
)

response = client.chat.completions.create(
    model="Qwen/Qwen3.8-27B",
    messages=[
        {"role": "user", "content": "Plan the task."}
    ],
    max_tokens=1024,
)

print(response.choices[0].message.content)

Authentication

The client reads an Autograd API key from the environment. Create and revoke keys in the console; send the key as a bearer token on every inference request.

export AUTOGRAD_API_KEY="sk-..."

Qwen 3.8 27B 262K context

Qwen 3.8 27B is designed for fast agent inference. The context window is a shared token budget: input, chat-template tokens, reasoning, and generated output together must fit within 262,144 tokens.

API model ID: Qwen/Qwen3.8-27B
Model size: 27 billion parameters
Context window: 262,144 tokens
Input budget: Context window minus generated output and chat overhead
Output budget: Set with max_tokens; must fit the remaining context
API surface: OpenAI-compatible Chat Completions
Streaming: Supported with stream: true

Plan output before input.

For example, reserving 4,096 output tokens leaves slightly less than 258,048 tokens for the prompt after chat-template overhead. Requests over the total context window are rejected rather than silently truncated.

Kimi K3 1M model context

Kimi K3 is a frontier reasoning model for complex agent work.

API model ID: moonshotai/Kimi-K3
Upstream model card: moonshotai/Kimi-K3 ↗
Model context: 1,048,576 tokens
Reasoning: Always enabled; returned in reasoning_content
API surface: OpenAI-compatible Chat Completions
Streaming: Supported with stream: true

Preserve the thinking history.

For multi-turn conversations and tool calls, pass the complete prior assistant message back in messages, including reasoning_content and tool_calls.

Chat completions

Send requests to POST /v1/chat/completions. Both buffered JSON responses and server-sent event streams are available through the same endpoint.

Request parameters

model: Required. Use Qwen/Qwen3.8-27B or moonshotai/Kimi-K3.
messages: Required. OpenAI-compatible chat messages.
max_tokens: Maximum generated tokens; constrained by remaining context.
temperature: Sampling temperature. Use 0 for deterministic work.
top_p: Nucleus-sampling probability mass.
stream: Set true to receive server-sent events.
tools: OpenAI-compatible tool definitions supported by both available models.
chat_template_kwargs: For Qwen, set {"enable_thinking": false} to disable reasoning for a request.
reasoning_effort: Kimi K3 supports low, high, and max.

Current limits

API request body: 16 MiB maximum at the edge
Qwen API context: 262,144 total tokens per request
Kimi K3 model context: 1,048,576 tokens; effective serving capacity is workload-dependent
Playground prompt: 12,000 characters
Playground output: 1,024 tokens

The smaller Playground limits keep the shared interactive surface responsive. They do not reduce the context available to API-key requests.

Traces and rewards

Inference captures the execution. Your application supplies the meaning. The proposed SDK links product events and reward signals to the same run.

proposed sdk

with autograd.trace() as run:
    result = agent.invoke(task)
    run.reward("task_success", 1.0)
    run.event("user.accepted")

SDK names and signatures shown here are directional and not yet a compatibility promise.

How the reinforcement loop works

RunCapture structured agent executions through inference.
RewardJoin outcomes, corrections, graders, and training permissions.
ReinforceEvaluate a baseline, train on selected work, and evaluate the candidate.
PromoteApprove a better checkpoint behind the same inference interface.

Next Try both models Choose Qwen 3.8 27B or Kimi K3 in the Playground