Collect events through a lightweight endpoint into a durable log such as Kafka or Kinesis, so producers are decoupled from consumers. Validate against a schema registry at the boundary and route failures to a dead letter topic. Land raw events immutably in object storage partitioned by event date, then transform into cleaned and modeled layers. Deduplicate on event id and key everything on event time.
Why interviewers ask this
This is an open ended design question used to see how you structure a system and where you place safeguards. Interviewers look for a durable buffer, schema enforcement at the edge, an immutable raw layer for replay, sensible partitioning, and a story for duplicates and late data. Jumping straight to tool names without describing the flow is the common failure.
How to structure your answer
- Clarify volume, latency requirement, and who consumes the output.
- Describe collection into a durable, replayable log.
- Enforce schema at the boundary with a dead letter path.
- Land raw immutably, then model in layers.
- Cover duplicates, late data, and personal data handling.
Example answer
I would ask first about volume and latency, because a hundred thousand events a day and a hundred thousand a second are different systems, and whether anyone actually needs sub minute data. Assuming a normal product analytics case, a thin collector endpoint writes into Kafka, which gives me durability and lets me replay if a downstream consumer has a bug. At that boundary I validate against a registered schema and push anything invalid to a dead letter topic with the reason, because dropping malformed events silently means you find out months later. Raw events land in object storage partitioned by event date, immutable, and that layer is the source of truth I can always rebuild from. From there a transformation layer cleans, deduplicates on event id, sessionizes, and models into fact and dimension tables. I would handle personal data at ingestion, hashing user identifiers and keeping the mapping in a restricted store, since it is far easier than trying to scrub a lake afterward.
Walking into this interview soon? GhostPilot listens to your live call, spots the question the moment it is asked, and puts a structured answer on your screen in real time. Try it on your next mock, or grab a $29 Session Pass, no subscription, for the real thing.
See how it worksFollow-up questions to expect
- How would you sessionize events, and what defines a session boundary?
- How do you handle bot traffic in the raw layer?
- What changes if the client is a mobile app that goes offline?
Related data engineer questions
Your interviewer will ask their own version of this. Paste your actual job description into the free Question Predictor and get the 20 questions that role is most likely to ask, with what each one is really probing.
Predict my questions