Interface AutoraterConfig

Autorater config used for evaluation.

interface AutoraterConfig {
    autoraterModel?: string;
    flipEnabled?: boolean;
    generationConfig?: GenerationConfig;
    samplingCount?: number;
}

Properties

autoraterModel? flipEnabled? generationConfig? samplingCount?

Properties

OptionalautoraterModel

autoraterModel?: string

The fully qualified name of the publisher model or tuned autorater endpoint to use.

Publisher model format: projects/{project}/locations/{location}/publishers/{publisher}/models/{model}

Tuned model endpoint format: projects/{project}/locations/{location}/endpoints/{endpoint}

OptionalflipEnabled

flipEnabled?: boolean

Optional. Default is true. Whether to flip the candidate and baseline responses. This is only applicable to the pairwise metric. If enabled, also provide PairwiseMetricSpec.candidate_response_field_name and PairwiseMetricSpec.baseline_response_field_name. When rendering PairwiseMetricSpec.metric_prompt_template, the candidate and baseline fields will be flipped for half of the samples to reduce bias.

OptionalgenerationConfig

generationConfig?: GenerationConfig

Configuration options for model generation and outputs.

OptionalsamplingCount

samplingCount?: number

Number of samples for each instance in the dataset. If not specified, the default is 4. Minimum value is 1, maximum value is 32.

Settings

On This Page

Properties