AnthropicEvaluationModel

Namespace: Vercel.AI.Anthropic

public sealed class AnthropicEvaluationModel : IRubricModel

Builds and scores Anthropic evaluation requests on the Messages API.

Inheritance: object

Implements: IRubricModel

View source

Constructors

AnthropicEvaluationModel(AnthropicProvider, string)

public AnthropicEvaluationModel(AnthropicProvider provider, string modelId)

Creates an evaluation model.

Parameters

Name	Type	Description
provider	AnthropicProvider
modelId	string

View source

Properties

ModelId

public string ModelId { get; }

Model id.

Value: string

View source

Provider

public string Provider { get; }

Provider id.

Value: string

View source

SupportedQuestionTypes

public IReadOnlyList<string> SupportedQuestionTypes { get; }

Question types this model accepts.

Value: IReadOnlyList<string>

View source

Methods

BuildSchema(IReadOnlyList<AnthropicEvaluationQuestion>)

public static JsonElement BuildSchema(IReadOnlyList<AnthropicEvaluationQuestion> questions)

Builds the json schema. Choice values are c0, c1, …. Number bounds are omitted.

Parameters

Name	Type	Description
questions	IReadOnlyList<AnthropicEvaluationQuestion>

Returns: JsonElement

View source

DoEvaluateAsync(string, string, CancellationToken)

public Task<RubricResult> DoEvaluateAsync(string rubric, string candidate, CancellationToken cancellationToken)

Scores a candidate against a rubric.

Parameters

Name	Type	Description
rubric	string
candidate	string
cancellationToken	CancellationToken

Returns: Task<RubricResult>

View source

EvaluateAsync(IReadOnlyList<AnthropicEvaluationQuestion>, LanguageModelCallOptions?, CancellationToken)

public Task<AnthropicEvaluationResult> EvaluateAsync(IReadOnlyList<AnthropicEvaluationQuestion> questions, LanguageModelCallOptions? options, CancellationToken cancellationToken)

Evaluates questions and maps q0, q1, … back onto the question names.

Parameters

Name	Type	Description
questions	IReadOnlyList<AnthropicEvaluationQuestion>
options	LanguageModelCallOptions
cancellationToken	CancellationToken

Returns: Task<AnthropicEvaluationResult>

View source

ParseAnswers(IReadOnlyList<AnthropicEvaluationQuestion>, string)

public static IReadOnlyDictionary<string, AnthropicEvaluationAnswer> ParseAnswers(IReadOnlyList<AnthropicEvaluationQuestion> questions, string json)

Maps encoded answers back to criteria labels and checks score and probability bounds.

Parameters

Name	Type	Description
questions	IReadOnlyList<AnthropicEvaluationQuestion>
json	string

Returns: IReadOnlyDictionary<string, AnthropicEvaluationAnswer>

View source