Talent.com
Data Engineer

Data Engineer

CSI GLOBAL LTDKraków, Województwo małopolskie, Polska
9 days ago
Job description

Data Engineer

Cracow, Poland

B2B Contract

ABOUT THE PROJECT

  • The GCDR application (hosted on Hadoop platform) is primarily concerned with entity resolution (matching internal and external party data together to form single view of the customer).
  • GCDR covers all four lines of business across 57+ markets. It also supports other services such as UI, reporting, network generation, APIs etc.
  • The entity resolution and network generation services use Quantexa software, however many other elements are HSBC build.

YOUR RESPONSIBILITIES

  • Develop, program, and maintain applications, creating new features, enhancing existing ones, etc
  • Work with different aspects of Apache Spark ecosystem, including ETL pipelines, data transformations and optimisation
  • Become familiar with the multiple applications and use cases that are supported on this platform
  • Assist in the root cause analysis of production issues

    • SKILLS & EXPERIENCE WE REQUIRE

  • Strong communication skills, with the ability to convey technical detail in a non technical language
  • Background in hands-on technical development, with at least three years of industry experience in a data engineering role or engineering equivalent
  • Proficiency in developing large scale data processing applications using Apache Spark with Scala including ETL pipelines, data transformations and optimization techniques
  • Expertise building and deploying production level data processing batch systems maintained by application support teams.
  • Experience with a variety of modern development tooling (e.g. Git, Gradle, Nexus) and technologies supporting automation and DevOps (e.g. Jenkins, Docker)
  • Experience working in an Agile environment
  • A strong technical communication ability with demonstrable experience of working in rapidly changing client environments.
  • Knowledge of testing libraries of common programming languages (such as Scala Test or equivalent). Knows the difference between different test types (unit test, integration test) and can cite specific examples of what they have written themselves.
    • Create a job alert for this search

    Data Engineer • Kraków, Województwo małopolskie, Polska