Let the decision be driven by how quickly the data changes a decision. If nobody acts on the data before the next morning, batch is cheaper, simpler to reason about, and far easier to backfill. Choose streaming when latency is part of the product, such as fraud scoring, live inventory, or alerting. Streaming costs you exactly once handling, late data, state management, and permanent on call.
Why interviewers ask this
Interviewers want to know whether you pick architectures for reasons rather than fashion, because a streaming pipeline nobody needed is a permanent operational tax. They are listening for the operational cost of streaming (state, watermarks, replay, schema changes on a running job) and for the point that a fifteen minute micro batch satisfies most so called real time requirements.
How to structure your answer
- Anchor on the decision latency the business actually needs.
- State the default: batch unless latency is part of the product.
- Name the real costs streaming adds beyond the happy path.
- Offer micro batch as the middle ground.
- Give one example of each from your own work.
Example answer
I start by asking who acts on this and how fast. If the answer is an analyst opening a dashboard at 9am, then a nightly or hourly batch job is the right call, because it is cheaper, trivially replayable, and I can rerun last Tuesday without thinking hard about it. I go streaming when latency is genuinely part of the product. On a payments team we streamed transaction events into a fraud model because a decision taken ninety minutes later has no value at all. What I try to make explicit is what streaming costs. You inherit state stores, watermarks and late data policy, and the fact that changing a schema on a running job is a migration rather than a deploy. A lot of the time when a stakeholder says real time they mean not tomorrow, and a fifteen minute micro batch on the warehouse gives them what they want for a fraction of the operational load.
Walking into this interview soon? GhostPilot listens to your live call, spots the question the moment it is asked, and puts a structured answer on your screen in real time. Try it on your next mock, or grab a $29 Session Pass, no subscription, for the real thing.
See how it worksFollow-up questions to expect
- How would you migrate an existing batch pipeline to streaming?
- What does a lambda architecture solve, and what does it cost?
- How do you backfill a streaming pipeline after a bug?
Related data engineer questions
Your interviewer will ask their own version of this. Paste your actual job description into the free Question Predictor and get the 20 questions that role is most likely to ask, with what each one is really probing.
Predict my questions