Models

Pinned ids and the `latest` alias

There are two kinds of model id:

Pinned ids, such as torchcast-decider-4b-v1-1-api-1, name one exact model. A pinned id never changes meaning: the same id always runs the same model with the same configuration, so its behavior does not shift under you.
The alias torchcast-decider-latest points at the newest stable pinned id. When a new model is released, the alias moves to it. Use the alias to get improvements without code changes; use a pinned id when production behavior must stay fixed until you choose to move.

The first pinned id is torchcast-decider-4b-v1-1-api-1. The retirement example below is illustrative. The current list is always GET /v1/models.

The response names the pinned model

The model field of every successful response is the pinned id that answered, even when the request named the alias. Record it next to each result: probabilities from different models are not directly comparable, and the pinned id tells you which model produced which numbers. A request is never answered by a different model than the one it resolved to. If that model is unavailable, the request fails with 503 model_unavailable instead.

Retirement

A pinned id can be retired. A request to a retired id fails with 410. When there is a successor, replacement_model names it:

Error

{
  "error": {
    "message": "The requested model has been retired.",
    "type": "invalid_request_error",
    "code": "model_retired",
    "replacement_model": "torchcast-decider-4b-v1-1-api-1"
  }
}

When a retired id has no successor, replacement_model is null: choose another active id from GET /v1/models, or use torchcast-decider-latest. The alias is never retired. A request to an id that does not exist fails with 400 invalid_model, and valid_models lists the ids you can call. See Errors.

List the models

GET https://api.torchcast.ai/v1/models lists every model id. It needs no API key, and it is rate limited to 5 requests per second per client IP.

curl https://api.torchcast.ai/v1/models

Models

{
  "data": [
    {
      "id": "torchcast-decider-latest",
      "kind": "alias",
      "target_model": "torchcast-decider-4b-v1-1-api-1",
      "status": "active",
      "replacement_model": null,
      "price_per_million_input_tokens_usd": null,
      "max_questions": 16,
      "max_options": 255,
      "max_levels": 10,
      "max_body_bytes": 29360128
    },
    {
      "id": "torchcast-decider-4b-v1-1-api-1",
      "kind": "pinned",
      "target_model": null,
      "status": "active",
      "replacement_model": null,
      "price_per_million_input_tokens_usd": null,
      "max_questions": 16,
      "max_options": 255,
      "max_levels": 10,
      "max_body_bytes": 29360128
    }
  ]
}

The example shows null for the price so that these docs never state one. A model’s live entry carries its current price.

Field	Meaning
`id`	The id to send as `model`.
`kind`	`alias` or `pinned`.
`target_model`	For an alias, the pinned id it points at now. `null` for pinned ids.
`status`	`active` or `retired`.
`replacement_model`	For a retired id, the id to move to, or `null` when there is none.
`price_per_million_input_tokens_usd`	Price in USD per 1,000,000 billable input tokens, as a decimal string with six decimals. Prices are also shown in the console. See Billing.
`max_questions`, `max_options`, `max_levels`	This model’s limits: questions per request, options per `choice` question, levels per `score` question.
`max_body_bytes`	Largest request body this model accepts, in bytes.

If the model list cannot be loaded, this endpoint answers 503 model_unavailable; retry with backoff. Per-model limits are never above the global limits on Limits, and can be lower. Read them from this endpoint instead of hard-coding them. Each model’s input-length limit, in tokens, is not in this list; see Input length. These are the limits the API checks before calling the model. For a short time after a model is added or updated, they can be the global ceilings while the model’s own limits are still loading. In that window the model itself may reject a request that is within them, with 400 invalid_request; reduce the request and retry.

Get started

Core concepts

Using the API

More

Pinned ids and the `latest` alias

The response names the pinned model

Retirement

List the models

​Pinned ids and the latest alias

​The response names the pinned model

​Retirement

​List the models

Pinned ids and the `latest` alias

The response names the pinned model

Retirement

List the models