To create datasets for evaluations, you need to correctly set up columns that match your prompt’s variables. There are two methods to make datasets ready for evaluations.

​
Manual Setup

Start with an empty dataset and create columns manually:

1

Create an empty dataset

Click the ‘add’ button (+) on the project you want to create a dataset in. Once created, this dataset is available to all resources in the project.

2

Create and rename columns

  • Click “Add column” to create a new column. Double-click the column name to rename it. Your dataset must have at least the columns exactly matching the variable names in your prompt to run evaluations.

  • Column names must match your prompt variables precisely. If your prompt uses {{customer_name}}, create a column named “customer_name”.

3

Create rows

Create rows for each of the test cases you want to run evaluations on.

4

Link dataset to evaluator

Navigate to the evaluator you want to link this dataset to. Click on ‘Select dataset’ and choose your newly created dataset. If you don’t see the dataset, double-check that the column names match exactly as variables in your prompt.

5

Edit anywhere

Once linked, you can edit the dataset from either the variable editor or the dataset page.

​
Automated Setup

1

Choose your evaluator

Choose the evaluator you want to link this dataset with.

2

Create a new dataset

Click “Create new dataset” from the dataset selection dropdown.

3

Auto-configured columns

A new dataset is created with matching column names as per the variables in your playground.

4

Create rows

Create rows for each of your test cases you want to run evaluations on.