dart_sentencepiece_tokenizer 1.3.0 
dart_sentencepiece_tokenizer: ^1.3.0 copied to clipboard
Published 31 days ago • 
brodykim.work
brodykim.workA lightweight, pure Dart implementation of SentencePiece tokenizer. Supports BPE (Gemma) and Unigram (Llama) algorithms.
6 versions
Stable versions of dart_sentencepiece_tokenizer
| Version | Min Dart SDK | Uploaded | Documentation | Archive | |
|---|---|---|---|---|---|
| 1.3.0 | 3.10 | 31 days ago |  |  | |
| 1.2.2 | 3.10 | 36 days ago | | | |
| 1.2.1 | 3.10 | 37 days ago | | ||
| 1.2.0 | 3.10 | 47 days ago | | ||
| 1.1.0 | 3.10 | 60 days ago | | ||
| 1.0.0 | 3.10 | 2 months ago | |
Metadata
3
likes
160
points
212
downloads
Publisher
Weekly Downloads
Metadata
A lightweight, pure Dart implementation of SentencePiece tokenizer. Supports BPE (Gemma) and Unigram (Llama) algorithms.
Repository (GitHub)
View/report issues
Topics
#nlp #sentencepiece #tokenizer #machine-learning #llm
Documentation
License
MIT (license)
