Create Evaluation

Creates and queues an evaluation run. If evaluatorId is provided, runs that single evaluator. Otherwise, runs all active evaluators configured for the prompt. Returns 202 since execution is asynchronous.

POST /prompts/{promptId}/evaluations

Create an evaluation

cURL

curl --request POST \
  --url https://api.adaline.ai/v2/prompts/{promptId}/evaluations \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "evaluatorId": "<string>",
  "datasetId": "<string>"
}
'

202

{
  "runId": "<string>",
  "evaluatorId": "<string>",
  "progress": {
    "totalRows": 123,
    "completedRows": 123,
    "failedRows": 123,
    "percent": 50
  },
  "metrics": {
    "passRuns": 123,
    "failRuns": 123,
    "unknownRuns": 123,
    "totalCost": 123,
    "totalLatency": 123,
    "totalTokenCount": 123
  },
  "timeline": {
    "submittedAt": 17450965800000,
    "startedAt": 123,
    "finishedAt": 123
  },
  "datasetId": "<string>"
}

Authorizations

Authorization · string · header · required

Workspace API key. Pass as Authorization: Bearer <key>.

Path Parameters

promptId · string · required

Unique identifier for an entity, between 20 and 80 characters.

Required string length: 20 - 80

Body

application/json

evaluatorId · string

Unique identifier for an entity, between 20 and 80 characters.

Required string length: 20 - 80

datasetId · string

Unique identifier for an entity, between 20 and 80 characters.

Required string length: 20 - 80

Response

202

application/json

Evaluation queued. Execution is asynchronous.

An evaluation run with its status, progress, metrics, and timeline.

runId · string · required

Unique identifier for an entity, between 20 and 80 characters.

Required string length: 20 - 80

evaluatorId · string · required

Unique identifier for an entity, between 20 and 80 characters.

Required string length: 20 - 80

status · enum<string> · required

The current status of an evaluation run.

Available options: queued, running, completed, failed, cancelled, cancelling

progress · Evaluation Progress · object · required

Progress information for an evaluation run.

Show child attributes

metrics · Evaluation Metrics · object · required

Aggregated metrics for an evaluation run.

Show child attributes

timeline · Evaluation Run Timeline · object · required

Timestamp information for an evaluation run.

Show child attributes

datasetId · string

Unique identifier for an entity, between 20 and 80 characters.

Required string length: 20 - 80