discover · compare · collections · requirements · contribute · feed · watching · stats · help with your tokens
openai/tiktoken
tiktoken is a fast BPE tokeniser for use with OpenAI's models.
History and source changes · Reports and owner responses · Compare