• encode() and encodePair() now throw ArgumentError for oversized inputs
• Validation applies to all encoding methods including batch operations

SentencePieceTokenizer - Main tokenizer class

• fromBytes() - Load from protobuf bytes
• fromModelFile() / fromModelFileSync() - Load from .model file
• encode() - Encode single text
• encodeBatch() - Encode multiple texts
• encodeBatchParallel() - Parallel batch encoding using Isolates
• encodePair() - Encode text pairs for sequence classification
• encodePairBatch() - Batch encode text pairs
• decode() / decodeBatch() - Decode token IDs back to text

Encoding class with:

• ids - Token IDs (Int32List)
• tokens - Token strings
• typeIds - Segment type IDs (Uint8List)
• attentionMask - Attention mask (Uint8List)
• specialTokensMask - Special token indicators (Uint8List)
• offsets - Character offsets for each token
• withPadding() / withTruncation() - Post-processing methods
• truncatePair() - Static method for pair truncation

Predefined configurations:

• SentencePieceConfig.llama - Llama-style (BOS only)
• SentencePieceConfig.gemma - Gemma-style (BOS + EOS)

Truncation strategies:

• longestFirst - Truncate longer sequence first
• onlyFirst - Only truncate first sequence
• onlySecond - Only truncate second sequence
• doNotTruncate - No truncation

Padding options:

• Left/right padding direction
• Fixed length or pad to longest
• Pad to multiple of N