Data Engineer Interview Questions 2026 — SQL, Spark, System Design
Complete data engineer interview preparation — SQL optimisation, Python, Apache Spark, Airflow, dbt, cloud (AWS/GCP/Azure), and data pipeline system design. Real questions for practice with answers.
What is Data Engineering?
Definition: Data engineering is the practice of designing, building and maintaining data pipelines that move, transform and store data at scale. Data engineers build the infrastructure that data analysts and data scientists rely on to do their work.
Data Engineer: Builds the infrastructure to collect, store and process data (pipelines, ETL, cloud, Spark)
Data Scientist: Builds ML models and statistical analyses on top of the data infrastructure
Advanced SQL for Data Engineers
Data engineer SQL questions go beyond basic queries — they test performance, scalability and optimisation thinking.
| # | Question | Concept | Difficulty |
|---|---|---|---|
| 1 | Explain query execution plan and how to optimise a slow query | EXPLAIN / query optimisation | Hard |
| 2 | What is database partitioning? When would you use it? | Partitioning strategy | Medium |
| 3 | Difference between OLTP and OLAP databases — when to use each | Database design | Medium |
| 4 | What are database indexes? When do they slow down performance? | Indexing | Medium |
| 5 | Write a query to deduplicate a table keeping the latest record per ID | ROW_NUMBER + CTE | Medium |
| 6 | What is a slowly changing dimension (SCD)? Implement Type 2 SCD | Data warehousing | Hard |
| 7 | Explain star schema vs snowflake schema | Data modelling | Medium |
Python & PySpark Questions
Key Python/PySpark Topics
- PySpark DataFrames vs RDDs — when to use each, performance implications
- Spark partitioning — how to optimise shuffle operations
- Broadcast joins in Spark — when and why to use them
- Writing efficient Python ETL scripts with error handling and logging
- Data validation patterns — how to ensure data quality in pipelines
Data Pipeline & System Design
System design questions for data engineers focus on pipeline architecture. Common question format: “Design a data pipeline for [use case].”
Clarify Requirements
What is the data source? Volume (GB/TB/PB per day)? Latency requirement (real-time / near-real-time / batch)? Data consumers (analysts, ML models, dashboards)? SLAs?
Choose Architecture
Batch (daily/hourly ETL): Use Airflow + Spark + data warehouse. Streaming (real-time): Use Kafka + Spark Streaming + OLAP store. Lambda: Both batch and streaming layers.
Design Storage Layer
Raw layer (S3/GCS/ADLS) → Processed layer (Parquet/Delta format) → Serving layer (BigQuery/Snowflake/Redshift). Define partitioning strategy (by date, region, etc.).
Define Transformation Logic
Use dbt for SQL transformations. Define data models: staging → intermediate → marts. Add data quality tests (dbt test, Great Expectations).
Handle Failures & Monitoring
Retry logic, dead letter queues, alerting (PagerDuty, Slack). SLA monitoring — alert if pipeline not completed by X time. Data quality alerts for anomaly detection.
Key Tools Interview Questions
| Tool | Common Question | Quick Answer |
|---|---|---|
| Apache Airflow | What is a DAG in Airflow? | Directed Acyclic Graph — defines task dependencies and execution order in a workflow. Tasks run in order defined by the graph with no cycles. |
| dbt | What is dbt and how does it fit in the modern data stack? | dbt (data build tool) handles the T in ELT — SQL-based transformations in the data warehouse, with version control, testing and documentation. |
| Apache Kafka | What is Kafka and when would you use it? | Distributed event streaming platform. Use for real-time data pipelines, event-driven architectures, and decoupling data producers from consumers. |
| Delta Lake | What is Delta Lake and what problem does it solve? | Open-source storage layer that adds ACID transactions, schema enforcement, and time travel to data lakes. Solves data lake reliability issues. |
| BigQuery | What is BigQuery partitioning and why does it matter? | Partitioning divides tables by a column (usually date). Queries only scan relevant partitions — dramatically reduces cost and improves speed. |
⭐ Key Takeaways
- Data engineering interviews test SQL optimisation, not just query writing — know EXPLAIN plans and indexing
- PySpark is essential for senior roles — practice DataFrames, window functions and joins in PySpark
- System design: be able to design batch and streaming pipeline architectures with storage, transformation and monitoring layers
- Key tool stack 2026: Airflow + dbt + Spark + cloud (AWS/GCP) + BigQuery/Snowflake
- Compare current job postings for the same responsibilities and location; assess fixed pay, conditional bonuses and equity separately.
- Differentiate yourself: get cloud certified (AWS/GCP) and build a public GitHub with pipeline projects
Crack Your Data Engineering Interview
Mock-interview services are coming soon. Booking and payment are currently inactive. View the planned offers or contact us with a question.
Mock interviews — coming soon →Documentation and next steps
Check the relevant documentation for your software version and database dialect. These are practice resources, not verified employer question banks.
Find a related learning guide · About the author · Suggest a correction