A random forest trains many deep trees in parallel on bootstrapped samples with random feature subsets, then averages, which mainly reduces variance. Gradient boosting trains shallow trees sequentially, each fitting the residual errors of the ensemble so far, which mainly reduces bias. Boosting usually wins on accuracy for tabular data but needs careful tuning; forests are far more forgiving out of the box.
Why interviewers ask this
This is the standard checkpoint for whether you understand ensembles mechanically rather than as library calls. The key discriminator is parallel and variance reducing versus sequential and bias reducing, plus the consequence: forests rarely overfit as you add trees, boosting absolutely can. Interviewers often push into learning rate and early stopping next, so be ready to explain why a low learning rate with many rounds beats the reverse.
How to structure your answer
- Contrast parallel bagging against sequential residual fitting.
- Say which error component each one attacks.
- Note that more trees is safe in a forest and not in boosting.
- Give your practical default and the tuning knobs that matter.
Example answer
A forest grows a lot of deep trees independently, each on a bootstrap sample and each considering a random subset of features at every split, then averages them. The trees are individually overfit and decorrelated, so averaging kills variance. Boosting is the opposite shape: shallow trees, built one at a time, each new tree fit to the errors the ensemble is still making, so it chips away at bias. The consequence people miss is what happens as you add trees. In a forest, more trees never really hurts, it just converges. In boosting, more trees eventually overfits, which is why the learning rate and early stopping matter so much. My default on tabular data is still gradient boosting, usually LightGBM or XGBoost, with a low learning rate and early stopping on a validation set, because that combination has beaten everything else I have tried on structured problems. I reach for a forest when I want a solid baseline in ten minutes with almost no tuning, or when I want out of bag estimates without carving out a separate validation split.
Walking into this interview soon? GhostPilot listens to your live call, spots the question the moment it is asked, and puts a structured answer on your screen in real time. Try it on your next mock, or grab a $29 Session Pass, no subscription, for the real thing.
See how it worksFollow-up questions to expect
- Why does a lower learning rate with more rounds usually generalize better?
- What does the subsample parameter do in gradient boosting?
- When would a plain linear model still beat both of these?
Related data scientist questions
Your interviewer will ask their own version of this. Paste your actual job description into the free Question Predictor and get the 20 questions that role is most likely to ask, with what each one is really probing.
Predict my questions