Senior Data Engineer

Responsibilities

• Implement data-facing backend features : analyze, design, and code services that read / write Delta tables, respecting schema evolution and time travel; document APIs and data contracts.
• Validate feasibility of new requirements within the existing medallion architecture; propose changes to bronze / silver / gold layers and indexing / partitioning where needed.
• Design and maintain interfaces to enterprise systems and BI tools; coordinate with interfacing parties and data governance for schema and access approvals.
• Create unit tests and component integration tests for Spark / SQL and service endpoints; establish data quality checks and reconciliation steps.
• Optimize performance and security : tune Spark jobs, improve query plans, enforce access controls, and ensure auditability.
• Maintain build and configuration : manage YAML pipelines in Azure DevOps, version artifacts, and automate deployments to Databricks environments.
• Drive DevOps for data services : monitoring, alerting, and incident-aware coding; assess release impacts on operations and downstream consumers.
• Conduct code reviews via pull requests; lead bug fixing with root cause analysis and preventive controls at data and code levels.

Qualifications

About Cluster Dynamics Reply

Cluster Reply is the Reply Group company specialising in consulting and system integration of Microsoft technologies. As a Microsoft partner, Cluster Reply is active in Germany, Austria and Switzerland and works within the Reply network with sister companies in Brazil, Great Britain, Italy as well as the USA. The company focuses on innovation and supports customers in their digital transformation. The solutions range from on-premises to cloud applications in the areas of modern workplace and security, business applications, applications and infrastructure as well as data and artificial intelligence.