dart_sentencepiece_tokenizer 1.1.0 
dart_sentencepiece_tokenizer: ^1.1.0 copied to clipboard
Published 11 days ago • 
brodykim.work
brodykim.workA lightweight, pure Dart implementation of SentencePiece tokenizer. Supports BPE (Gemma) and Unigram (Llama) algorithms.
2 versions
Stable versions of dart_sentencepiece_tokenizer
| Version | Min Dart SDK | Uploaded | Documentation | Archive | |
|---|---|---|---|---|---|
| 1.1.0 | 3.10 | 11 days ago |  |  | |
| 1.0.0 | 3.10 | 12 days ago | | |
Metadata
0
likes
160
points
164
downloads
Publisher
Weekly Downloads
Metadata
A lightweight, pure Dart implementation of SentencePiece tokenizer. Supports BPE (Gemma) and Unigram (Llama) algorithms.
Repository (GitHub)
View/report issues
Topics
#nlp #sentencepiece #tokenizer #machine-learning #llm
Documentation
License
MIT (license)
