[C3F] Data Platform Engineer

Project the aim youll have

Our customer provides innovative solutions and insights that enable our clients to manage risk and hire the best talent. Their advanced global technology platform supports fully scalable configurable screening programs that meet the unique needs of over 33000 clients worldwide. Headquartered in Atlanta GA they have an internationally distributed workforce spanning 19 countries with about 5500 employees. Our partner perform over 93 million screens annually in over 200 countries and territories.

Position how youll contribute

• Build and operate change-data-capture pipelines from PostgreSQL into Azure using Kafka Connect and Debezium as the core of the platform
• Configure deploy and scale connectors end to end - connector setup task management offsets schema history and snapshot strategy
• Run these pipelines as stateful workloads on Kubernetes (AKS) covering configuration secrets networking and resource tuning
• Monitor and troubleshoot the platform in production: connector failures task rebalances restarts throughput and backpressure message-size limits retries and recovery
• Automate the platform in Python - configuration-driven onboarding of new data sources pipeline orchestration monitoring and alerting recovery workflows and automated testing
• Integrate CDC streams with the wider Azure data stack: Event Hubs ADLS Azure PostgreSQL ADF and Databricks
• Manage platform infrastructure as code so environments are reproducible and changes are reviewable
• Apply data protection requirements to sensitive data flowing through the pipelines - masking hashing access control and retention

Qualifications :

Expectations the experience you need

• Solid commercial experience as a data or platform engineer with hands-on work on streaming or CDC pipelines rather than batch reporting alone
• Practical Kafka knowledge - topics partitions offsets consumer groups and delivery semantics - including at least one Kafka Connect deployment you ran yourself
• Strong SQL and PostgreSQL skills with working knowledge of WAL logical replication replication slots and replication lag
• Working understanding of CDC concepts: initial snapshots inserts updates and deletes event ordering at-least-once delivery and schema evolution
• Confident Python for automation and tooling - orchestration monitoring recovery scripts and automated tests
• Hands-on experience with Azure data services for example Event Hubs ADLS or Azure PostgreSQL
• Comfortable working with Kubernetes as a user: deploying workloads handling configuration and secrets reading logs debugging failing pods
• Ability to debug a running pipeline from metrics and logs - telling throughput problems from backpressure retries or a genuine connector failure

Additional skills the edge you have

• Production experience with Debezium specifically - snapshot strategies on large tables schema history recovery offset loss and bringing connectors back after failure
• Experience operating stateful workloads on AKS: StatefulSets stable worker identity and resource tuning under load
• Infrastructure-as-code and CI/CD for data platform components (Terraform Bicep or similar)
• Hands-on work with Databricks and ADF at production scale
• Experience implementing data protection controls for sensitive data - masking hashing access control and retention policies

Additional Information :

Our offer professional development personal growth

• Flexible employment and remote work
• International projects with leading global clients
• International business trips
• Non-corporate atmosphere
• Language classes
• Internal & external training
• Private healthcare and insurance
• Multisport card
• Well-being initiatives

Remote Work :

Yes

Employment Type :

Full-time

Experience: years
Vacancy: 1