AutoraterConfig

@Serializable
data class AutoraterConfig(val samplingCount: Int? = null, val flipEnabled: Boolean? = null, val autoraterModel: String? = null, val generationConfig: GenerationConfig? = null)

Autorater config used for evaluation.

Constructors

Link copied to clipboard
constructor(samplingCount: Int? = null, flipEnabled: Boolean? = null, autoraterModel: String? = null, generationConfig: GenerationConfig? = null)

Properties

Link copied to clipboard
val autoraterModel: String? = null

The fully qualified name of the publisher model or tuned autorater endpoint to use.

Link copied to clipboard
val flipEnabled: Boolean? = null

Optional. Default is true. Whether to flip the candidate and baseline responses. This is only applicable to the pairwise metric. If enabled, also provide PairwiseMetricSpec.candidate_response_field_name and PairwiseMetricSpec.baseline_response_field_name. When rendering PairwiseMetricSpec.metric_prompt_template, the candidate and baseline fields will be flipped for half of the samples to reduce bias.

Link copied to clipboard

Configuration options for model generation and outputs.

Link copied to clipboard
val samplingCount: Int? = null

Number of samples for each instance in the dataset. If not specified, the default is 4. Minimum value is 1, maximum value is 32.