Talent.com
Data Engineer – PySpark

Data Engineer – PySpark

brabersPowiat Krakowski, Województwo małopolskie, Polska
8 days ago
Job description

Job Title :

Data Engineer – PySpark

Work Mode :

Hybrid (3 days a week from client's office in Kraków)

Project Duration : Long Term - B2B

About the Role :

We are looking for an experienced

Data Engineer

with strong

PySpark

and

Python

skills to join a dynamic team. The ideal candidate will have hands-on experience building large-scale data analytics solutions in a big data or cloud environment.

Key Responsibilities :

  • Design, develop, and maintain robust data pipelines using

PySpark

  • Work with large volumes of data using

    • RDDs, DataFrames, and Datasets

    (not just Spark SQL).

  • Develop Spark applications for various data processing use cases, including performance tuning and complex transformations (group, window functions, etc.).
  • Write clean, efficient, and reusable Python code, and troubleshoot / fix bugs to ensure high-quality deliverables.
  • Automate tasks and processes by creating scripts and tools.
  • Translate business requirements into technical solutions.
  • Collaborate effectively with team members in an

    • Agile environment

  • Optionally, work with

    • Airflow, Databricks, and Azure

    Required Experience & Skills :

  • 5–10 years of hands-on experience in the data analytics space as a

    • Data Engineer

    , familiar with ETL, DQ, DM, and reject / recycling concepts.

  • Minimum

    • 3 years of PySpark coding experience

    , building applications with RDDs, DataFrames, and Datasets.

  • Proven experience developing and deploying

    • Spark applications

    for processing large datasets.

  • Strong proficiency in

    • Python

    ; candidates must demonstrate advanced coding skills.

  • Experience with performance tuning and complex transformations.
  • Exposure to

    • CI / CD pipelines

    and Agile methodologies is a plus.

  • Participation in

    • PySpark coding hackathons

    or similar is a bonus.

    Additional Skills :

  • Strong problem-solving and analytical skills.
  • Ability to work independently and as part of a team.
  • Good communication skills.
    • Create a job alert for this search

    Data Engineer • Powiat Krakowski, Województwo małopolskie, Polska