Rule out instrumentation before you look for a business cause, because a tracking or pipeline failure is the most common explanation for a sharp overnight change. Then segment aggressively: platform, app version, geography, browser, new versus returning, and acquisition channel. A drop concentrated in one segment points at a technical cause; a drop spread evenly across every segment points at something external or at your measurement.
Why interviewers ask this
This is the definitive analyst scenario question, and interviewers care about method, not the answer. They want measurement checked first, systematic segmentation to localize the change, awareness of external causes such as holidays or outages, and a habit of confirming the drop is real before anyone escalates it to leadership.
How to structure your answer
- Confirm the number is real: check the pipeline, tracking, and definition.
- Establish exactly when it started, to the hour.
- Segment along platform, version, geography, and channel to localize it.
- Check external causes: releases, outages, holidays, marketing changes.
- Report the finding with a confidence level and next step.
Example answer
My first question is always whether the drop is real, because in my experience the majority of overnight cliffs are measurement. So I check whether the pipeline ran completely, whether anyone changed the metric definition or the dashboard filter, and whether an app release changed the tracking events. A 20% drop that is exactly 20% every hour smells like a data problem, not a human one. Then I look at when it started to the hour, which narrows the suspect list immediately, and I segment. Platform, app version, country, browser, new versus returning. If it is all on Android 8.4.0, that is a release bug and I go straight to the mobile team. If it is spread evenly across everything, I look outward at holidays, an outage at a login provider, or a marketing campaign that ended. Once I had a drop that was entirely one country and it turned out to be an ISP level DNS issue. Then I report what I know, what I ruled out, and what I would check next.
Walking into this interview soon? GhostPilot listens to your live call, spots the question the moment it is asked, and puts a structured answer on your screen in real time. Try it on your next mock, or grab a $29 Session Pass, no subscription, for the real thing.
See how it worksFollow-up questions to expect
- What would you check first if the drop was gradual over two weeks instead?
- How would you distinguish a tracking bug from a real user drop?
- What do you tell leadership before you know the cause?
Related data analyst questions
Your interviewer will ask their own version of this. Paste your actual job description into the free Question Predictor and get the 20 questions that role is most likely to ask, with what each one is really probing.
Predict my questions