Dataflow Engineer

Dataflow Engineer (k/m)

Location: Kraków/Hybrid
Contract Type: B2B

About the Role

We are looking for a skilled Dataflow Engineer to join our dynamic team. In this role, you will design, build, and maintain robust data pipelines that ensure data flows seamlessly across systems. You will work closely with data architects, analysts, and other engineers to process large-scale datasets efficiently and reliably.

Key Responsibilities

• Design & Build Data Pipelines: Develop and implement scalable pipelines for collecting, transforming, and storing data across various platforms.
• Data Integration: Connect data from cloud platforms (e.g., Google Cloud), SQL/NoSQL databases, APIs, and external services.
• Optimize Data Flow: Troubleshoot and fine-tune existing pipelines to ensure optimal performance.
• Data Transformation (ETL): Transform raw data into usable formats for analytics and reporting.
• Collaboration: Work with cross-functional teams to understand data requirements and deliver solutions; support systems during weekends or off-hours as needed.
• Automation & Scalability: Build automated workflows to handle high-volume data with low latency and high reliability.
• Monitoring & Maintenance: Set up monitoring and alerting for pipelines to minimize downtime and maximize performance.
• Documentation: Maintain clear documentation of data flows, pipeline configurations, and processing logic.

Required Qualifications

• Experience: 4+ years as a Dataflow Engineer, Data Engineer, or similar role with large datasets and distributed systems.
• Technical Skills:
  • Proficient in Python and Java.
  • Hands-on experience with pipeline orchestration tools (e.g., Google Dataflow).
  • Strong knowledge of cloud platforms (Google Cloud: BigQuery, Dataflow).
  • Experience with ETL frameworks, real-time data streaming, and processing.
  • Familiarity with data formats like JSON and Parquet.
  • Knowledge of SQL/NoSQL databases, data governance, quality, and security best practices.
• Problem-Solving: Ability to troubleshoot complex data integration and processing issues.
• Communication Skills: Strong written and verbal communication to collaborate effectively with technical and non-technical stakeholders.

Why Join Us

• Work on cutting-edge data solutions in a collaborative environment.
• Opportunity to handle large-scale, high-impact data projects.
• Professional growth in a fast-evolving data engineering field.

Zapraszamy do odwiedzenia naszej strony www.antal.pl