Start by proving the model can learn: overfit a single small batch to near zero loss. If it cannot, the bug is in the model, the loss, or the optimizer, not in data volume. Then check the learning rate at both extremes, verify labels are aligned with inputs, confirm gradients are flowing, and confirm the optimizer actually received the parameters you think it did.
Why interviewers ask this
This is the most common practical failure in deep learning work, and the answer reveals whether you debug systematically or start flipping hyperparameters. Overfitting a single batch is the canonical first move because it isolates capability from data, and interviewers listen specifically for it. Checking gradient norms and label alignment shows you know where the real bugs usually hide.
How to structure your answer
- Start with the single batch overfit test to isolate the problem.
- Check the learning rate at both extremes.
- Verify gradients flow and parameters are registered with the optimizer.
- Check the data path: label alignment, normalization, shuffling.
Example answer
First move is always the same: take one batch of about eight examples and try to drive the loss to nearly zero on it. A working model memorizes eight examples in a couple of hundred steps. If it cannot, the problem is not data quantity or regularization, it is something broken in the model, the loss, or the optimizer, and I have massively narrowed the search. From there I check the learning rate at both ends, because too high shows up as loss bouncing or going to NaN, and too low looks like a flat line that is actually creeping. I print gradient norms per layer, which immediately tells me whether the signal is dying or whether some layer is getting nothing because it was accidentally frozen or never passed to the optimizer. Then the data path, and honestly this is where the bug usually is: labels shifted by one relative to inputs after a shuffle, normalization applied twice, or a targets tensor that is all one class because of a filtering bug. I always look at a handful of decoded examples with their labels by eye before blaming the architecture.
Walking into this interview soon? GhostPilot listens to your live call, spots the question the moment it is asked, and puts a structured answer on your screen in real time. Try it on your next mock, or grab a $29 Session Pass, no subscription, for the real thing.
See how it worksFollow-up questions to expect
- What would a loss that goes to NaN after a few hundred steps suggest?
- How would you sanity check that your data loader is shuffling correctly?
- What does it mean if training loss drops but validation loss never does?
Related machine learning engineer questions
Your interviewer will ask their own version of this. Paste your actual job description into the free Question Predictor and get the 20 questions that role is most likely to ask, with what each one is really probing.
Predict my questions