AutoTokenizer produces wrong token IDs for OLMo2, HyperClovaX, DeepSeek-R1-Distill-Llama, Yi, and others (v5 regression)
May 12, 2026 · #45920
Python
Difficulty: Medium
Labels
bug
Parent Repository
huggingface/transformers
Python repository
160,724 33,246
Ready to work on this? Walk through the full fork-to-PR workflow so your first contribution goes smoothly.