discover · compare · collections · requirements · contribute · feed · watching · stats · help with your tokens
karpathy/minbpe
Minimal, clean code for the Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization.
History and source changes · Reports and owner responses · Compare