We are looking for a skilled and proactive Senior Data Engineer with deep expertise in Databricks to join our data platform team. You will be responsible for designing, building, and optimizing scalable data pipelines and lakehouse architectures that power analytics, reporting, and machine learning across the organization
Responsibilities
- Develop and maintain robust ETL / ELT pipelines using Databricks and Apache Spark
- Design and implement Delta Lake architectures for structured and semi-structured data
- Collaborate with data analysts, scientists, and product teams to deliver clean, reliable datasets
- Optimize performance of Spark jobs and manage cluster resources efficiently
- Automate workflows using Databricks Jobs, Workflows
- Ensure data quality, lineage, and governance using Unity Catalog and monitoring tools
- Document data models, pipeline logic, and architectural decisions
- Participate in code reviews and contribute to engineering best practices
Required skills and experience
4+ years of experience as a Data Engineer or in a similar roleStrong hands‑on experience with Databricks , including Delta Lake, Spark SQLProficiency in Python and SQL for data manipulation and pipeline developmentSolid understanding of Apache Spark internals and performance tuningExperience with cloud platforms (Azure, AWS, GCP)Knowledge of data modeling, partitioning, and lakehouse principlesAbility to work with large-scale datasets and optimize storage and compute costsStrong communication skills and ability to collaborate across teamsNice to have
Experience with Azure Data Factory or AirflowProficiency with modern data warehouses and tools including Snowflake , Synapse, Red GateExposure to data governance frameworks and tools (e.g., Unity Catalog, Purview)Understanding of machine learning workflows and integration with data pipelinesExperience with BI tools (Power BI, Tableau) and supporting analytics teamsContributions to open-source projects or technical blogs#J-18808-Ljbffr