A good DAG has small idempotent tasks with clear dependencies, derives its time window from the data interval rather than the wall clock, and keeps heavy logic out of the DAG file since that file is parsed constantly. Avoid passing large data through XCom, avoid long blocking sensors that occupy worker slots, and set sensible retries, timeouts, and alerting on the tasks that matter.
Why interviewers ask this
Orchestration bugs cause quiet correctness problems, not just failures, so interviewers use this to check practical experience. They want top level code hygiene (a slow DAG file degrades the whole scheduler), the data interval point that makes reruns correct, and awareness of resource exhaustion from sensors and oversized XCom payloads.
How to structure your answer
- Argue for small, idempotent, independently retryable tasks.
- Explain why the time window must come from the data interval.
- Warn about heavy top level code in the DAG file.
- Cover XCom size limits and passing references instead of payloads.
- Set retries, timeouts, SLAs, and meaningful alerting.
Example answer
The two things I care about most are task granularity and idempotency. Tasks should be small enough that a failure is cheap to retry and specific enough that the failure tells you what broke, and every one of them should be safe to rerun. Then the time window has to come from the data interval Airflow passes in, never from datetime.now inside the task, otherwise a rerun of last Tuesday processes today's data and you get a subtle wrong answer instead of an error. In review, the thing I flag most often is expensive code at the top level of the DAG file. That file gets parsed on a short interval, so an API call or a database query outside the task function runs constantly and degrades the whole scheduler; I have seen that alone add minutes to scheduling latency across an entire deployment. I also check XCom usage, since it is a metadata table and not a data transport, so pass an S3 path rather than a dataframe.
Walking into this interview soon? GhostPilot listens to your live call, spots the question the moment it is asked, and puts a structured answer on your screen in real time. Try it on your next mock, or grab a $29 Session Pass, no subscription, for the real thing.
See how it worksFollow-up questions to expect
- What is a deferrable operator and what problem does it solve?
- How would you handle a DAG that needs to process a variable number of files?
- How do you test a DAG before it reaches production?
Related data engineer questions
Your interviewer will ask their own version of this. Paste your actual job description into the free Question Predictor and get the 20 questions that role is most likely to ask, with what each one is really probing.
Predict my questions