Python wrapper for SentencePiece. This API supports the encoding, decoding, and training of SentencePiece models. For a detailed feature and API comparison with Hugging Face Tokenizers and OpenAI's ...
Each example comes with C/C++ source code, testbench, a README, and Tcl/Python scripts and/or config file. The examples are organized in categories denoted by the directory names: ...
Some results have been hidden because they may be inaccessible to you
Show inaccessible results