NaN loss with SDPA attention when local_batch_size > 1 on Gemma 4 4B

April 16, 2026 · #1883
View on GitHub
Python Difficulty: Medium

Parent Repository

NVIDIA-NeMo/Automodel

Python repository

447 133
All Issues Back to Automodel

Sign in required

Authenticate to use favourites & bookmarks

5