Data Engineer Hadoop

Czym będziesz się zajmować?

• Design, implement, and maintain large-scale distributed data processing systems using Hadoop, Spark, and related technologies
• Develop scalable and efficient data pipelines using Scala, Spark, Hive, and SQL
• Work with Google Cloud Platform (GCP) services for data ingestion, transformation, storage, and orchestration
• Migrate and process data using GCP tools such as BigQuery, Dataflow, Dataproc, Cloud Storage, Pub / Sub, and Composer (Airflow).
• Collaborate with architects and development teams to define technically sound and scalable solution designs.
• Debug and troubleshoot data processing and code-related issues; communicate findings with the development team
• Create automated data workflows using tools like Airflow and Jenkins as part of CI / CD pipelines.
• Ensure proper version control, testing (unit / integration), and deployment practices in accordance with DevOps methodologies.
• Communicate effectively with business stakeholders and contribute to a collaborative team environment
• Work in Agile methodologies (Scrum / Kanban) and participate in planning, stand-ups, and retrospectives

Kogo poszukujemy?

Strong experience with the Hadoop ecosystem : Hadoop, HDFS, Hive

Proficiency in Apache Spark and Scala

Advanced SQL skills

Experience with Google Cloud Platform (GCP), especially : BigQuery

Cloud Dataflow

Cloud Dataproc

Cloud Storage

Pub / Sub

Cloud Composer amp; Airflow

Familiarity with CI / CD tools such as Jenkins and Git / GitHub

Experience with Airflow or similar orchestration tools

Understanding of big data architecture and data modeling techniques (both relational and non-relational)

Experience in designing and deploying scalable data solutions

Strong debugging and code review skills

Exposure to Enterprise Data Warehousing concepts

Experience with DevOps practices and tools (Ansible, JIRA)

Knowledge of Agile frameworks (Scrum, Kanban)

Czego wymagamy?

Znajomości :

Hadoop

Spark

Języki :

Polski

Angielski

Jakie warunki i benefity otrzymasz?

26300-28400 PLN miesięcznie

B2B - Elastyczne godziny pracy (100%)

Praca zdalna : Możliwa częściowo

Pakiet medyczny, Pakiet sportowy

Kawa / Herbata, Zimne napoje

Parking, Pokój relaksu

Kim jesteśmy?

Verita HR is an international company providing recruitment support within #Fintech, #Finance and #Banking market in EMEA. We connect the most innovative organizations with the best people in the market. We conduct systematic market research, which allows our Digital Teams to be a step ahead of the competition.