Data Engineer

Czym będziesz się zajmować?

• Design, develop, and maintain robust data pipelines using PySpark in a scalable Big Data environment.
• Contribute to system architecture and design discussions.
• Participate in Agile ceremonies, including sprint planning, reviews, and retrospectives.
• Collaborate with Business Analysts to ensure accurate interpretation and implementation of requirements.
• Implement tools and processes that ensure performance, scalability, availability, and accuracy.
• Promote best practices through code reviews, mentoring, and knowledge sharing within the team.
• Provide production support, including troubleshooting and resolving data pipeline issues.
• Maintain clear documentation and participate in planning and status meetings

Kogo poszukujemy?

5+ years of experience in software design and PySpark / Scala development.

Strong experience in developing and optimizing data pipelines using Hadoop components (Hive, Spark, Spark SQL).

Hands-on experience with scheduling tools like Apache Airflow .

Proficiency with Big Data technologies : Apache Spark, YARN, Hive, Python, SQL, RESTful APIs, MapReduce .

Solid understanding of working in Unix / Linux environments.

Experience with version control and CI / CD tools : Git / GitHub, Ansible, Jenkins .

Familiarity with Agile project management tools such as JIRA .

Strong understanding of both relational and non-relational data modeling techniques.

Proven debugging and code analysis skills with the ability to clearly communicate findings to the development team.

Czego wymagamy?

5 lat doświadczenia na podobnym stanowisku

Znajomości :

Scala

Apache Airflow

Hadoop components

Języki :

Polski

Angielski

Jakie warunki i benefity otrzymasz?

25200-28400 PLN miesięcznie

B2B - Elastyczne godziny pracy (100%)

Praca zdalna : Możliwa częściowo

Pakiet medyczny, Pakiet sportowy

Kawa / Herbata, Zimne napoje

Parking, Parking rowerowy, Pokój relaksu

Kim jesteśmy?

Verita HR is an international company providing recruitment support within #Fintech, #Finance and #Banking market in EMEA. We connect the most innovative organizations with the best people in the market. We conduct systematic market research, which allows our Digital Teams to be a step ahead of the competition.