GenerationParams class

Parameters controlling the token sampling and generation process.

Use GenerationParams to fine-tune how the model generates text, including randomness (temperature), sampling constraints (Top-K/Top-P), and architectural limits (max tokens).

Example:

final params = GenerationParams(
  temp: 0.7,
  maxTokens: 1024,
  stopSequences: ['User:', '\n\n'],
  grammar: 'root ::= "yes" | "no"', // Force binary response
);

Constructors

GenerationParams({int maxTokens = 4096, double temp = 0.8, int topK = 40, double topP = 0.9, double minP = 0.0, double penalty = 1.1, double presencePenalty = 0.0, int? seed, List<String> stopSequences = const [], String? grammar, bool grammarLazy = false, ThinkingBudget? thinkingBudget, List<GenerationGrammarTrigger> grammarTriggers = const [], List<String> preservedTokens = const [], String grammarRoot = 'root', bool speculativeDecoding = false, SpeculativeDecodingConfig? speculativeDecodingConfig, bool reusePromptPrefix = defaultReusePromptPrefix, int streamBatchTokenThreshold = defaultStreamBatchTokenThreshold, int streamBatchByteThreshold = defaultStreamBatchByteThreshold})
Creates generation parameters with default values.
const

Properties

grammar → String?
GBNF grammar string for structured output (e.g., "root ::= "hello" | "world"").
final
grammarLazy → bool
Whether grammar should be lazily activated by triggers.
final
grammarRoot → String
Grammar start symbol. Defaults to "root".
final
grammarTriggers → List<GenerationGrammarTrigger>
Lazy grammar activation triggers.
final
hashCode → int
The hash code for this object.
no setterinherited
isSpeculativeDecodingEnabled → bool
Whether speculative decoding is requested by either public API shape.
no setter
maxTokens → int
Maximum number of new tokens to generate.
final
minP → double
Min-P sampling threshold.
final
penalty → double
Penalty applied to tokens that have already appeared in the sequence. 1.0 means no penalty.
final
presencePenalty → double
Additive penalty applied once to tokens that appear in the sampler's recent history.
final
preservedTokens → List<String>
Tokens to preserve during constrained decoding.
final
resolvedSpeculativeDecodingConfig → SpeculativeDecodingConfig?
Resolved speculative decoding configuration, if enabled.
no setter
reusePromptPrefix → bool
Reuses matching prompt prefixes from previous requests in the same native context to reduce prompt ingestion latency.
final
runtimeType → Type
A representation of the runtime type of the object.
no setterinherited
seed → int?
Random seed for the sampler.
final
speculativeDecoding → bool
Enables backend-native speculative decoding when supported.
final
speculativeDecodingConfig → SpeculativeDecodingConfig?
Strategy and knobs for backend-native speculative decoding.
final
stopSequences → List<String>
Strings that end generation when matched, excluding the marker from output.
final
streamBatchByteThreshold → int
Native worker chunk flush threshold by byte size.
final
streamBatchTokenThreshold → int
Native worker chunk flush threshold by token pieces.
final
temp → double
Temperature for sampling (higher = more creative/random, lower = more deterministic). Range is typically 0.0 to 2.0. Native LiteRT-LM CPU/GPU and LiteRT-LM Web treat zero as greedy decoding and use one candidate regardless of topK.
final
thinkingBudget → ThinkingBudget?
Optional llama.cpp-style limit for tokens generated inside a reasoning block.
final
topK → int
Top-K sampling: only sample from the top K most likely tokens. Set to 0 to disable.
final
topP → double
Top-P sampling (nucleus sampling): only sample from tokens whose cumulative probability exceeds P.
final

Methods

copyWith({int? maxTokens, double? temp, int? topK, double? topP, double? minP, double? penalty, double? presencePenalty, int? seed, List<String>? stopSequences, String? grammar, bool? grammarLazy, ThinkingBudget? thinkingBudget, bool clearThinkingBudget = false, List<GenerationGrammarTrigger>? grammarTriggers, List<String>? preservedTokens, String? grammarRoot, bool? speculativeDecoding, SpeculativeDecodingConfig? speculativeDecodingConfig, bool clearSpeculativeDecodingConfig = false, bool? reusePromptPrefix, int? streamBatchTokenThreshold, int? streamBatchByteThreshold}) → GenerationParams
Creates a copy of this GenerationParams with updated fields.
noSuchMethod(Invocation invocation) → dynamic
Invoked when a nonexistent method or property is accessed.
inherited
toString() → String
A string representation of this object.
inherited

Operators

operator ==(Object other) → bool
The equality operator.
inherited

Constants

defaultReusePromptPrefix → const bool
Default prompt prefix reuse behavior for native generation.
defaultStreamBatchByteThreshold → const int
Default native stream batching threshold by byte size.
defaultStreamBatchTokenThreshold → const int
Default native stream batching threshold by token pieces.