DevTools
Analyze text tokenization for different AI models and calculate API costs
Uses cl100k_base encoding with byte-pair encoding (BPE) for efficient tokenization
Similar to GPT but with slightly different token boundaries and vocabulary
Google's tokenization approach, often allowing longer token sequences