Type Alias EvaluatorOutputDefinition

EvaluatorOutputDefinition:
    | unstable.EvaluatorOutputDefinition.Numeric
    | unstable.EvaluatorOutputDefinition.Boolean
    | unstable.EvaluatorOutputDefinition.Categorical

Structured output definition to send when creating an evaluator.

Agent guidance:

dataType is required.
Do not send version; that is an internal storage detail and is not part of the public request contract.
For NUMERIC and BOOLEAN, provide reasoning.description and score.description.
For CATEGORICAL, also provide score.categories and score.shouldAllowMultipleMatches.

Example

{
 *         dataType: "NUMERIC",
 *         dataType: LangfuseAPI.unstable.EvaluatorOutputDataType.Numeric,
 *         reasoning: {
 *             description: "Explain why the answer is correct or incorrect."
 *         },
 *         score: {
 *             description: "Return a score between 0 and 1."
 *         }
 *     }

Example

{
 *         dataType: "BOOLEAN",
 *         dataType: LangfuseAPI.unstable.EvaluatorOutputDataType.Boolean,
 *         reasoning: {
 *             description: "Explain why the output satisfies the requirement."
 *         },
 *         score: {
 *             description: "Return true if the output satisfies the requirement, otherwise false."
 *         }
 *     }

Example

{
 *         dataType: "CATEGORICAL",
 *         dataType: LangfuseAPI.unstable.EvaluatorOutputDataType.Categorical,
 *         reasoning: {
 *             description: "Explain which category best fits the output."
 *         },
 *         score: {
 *             description: "Choose the best category.",
 *             categories: ["correct", "partially_correct", "incorrect"],
 *             shouldAllowMultipleMatches: false
 *         }
 *     }

Type Alias EvaluatorOutputDefinition

Example

Example

Example

Settings