SlidingWindow

@Serializable
data class SlidingWindow(val targetTokens: Long? = null)

Context window will be truncated by keeping only suffix of it.

Context window will always be cut at start of USER role turn. System instructions and BidiGenerateContentSetup.prefix_turns will not be subject to the sliding window mechanism, they will always stay at the beginning of context window.

Constructors

Link copied to clipboard
constructor(targetTokens: Long? = null)

Properties

Link copied to clipboard
val targetTokens: Long? = null

Session reduction target -- how many tokens we should keep. Window shortening operation has some latency costs, so we should avoid running it on every turn. Should be < trigger_tokens. If not set, trigger_tokens/2 is assumed.