Talent.com
Senior Data Engineer (Spark)
Senior Data Engineer (Spark)Addepto • Białystok, Podlaskie, Poland
Senior Data Engineer (Spark)

Senior Data Engineer (Spark)

Addepto • Białystok, Podlaskie, Poland
Ponad 30 dni temu
Opis pracy

technologies-expected :


  • Python
  • SQL
  • Spark
  • Airflow
  • AWS
  • Cloudera
  • Iceberg

technologies-optional :


  • Kubernetes
  • Kafka
  • NiFi
  • Trino
  • Hudi
  • Java
  • Scala
  • Docker
  • Databricks

about-project :


  • As a Senior Data Engineer, you will have the exciting opportunity to work with a team of technology experts on challenging projects across various industries, leveraging cutting-edge technologies. Here are some of the projects we are seeking talented individuals to join:
  • Development and maintenance of a large platform for processing automotive data. A significant amount of data is processed in both streaming and batch modes. The technology stack includes Spark, Cloudera, Airflow, Iceberg, Python, and AWS.
  • Design and development of a universal data platform for global aerospace companies. This Azure and Databricks powered initiative combines diverse enterprise and public data sources. The data platform is at the early stages of the development, covering design of architecture and processes as well as giving freedom for technology selection.
  • Centralized reporting platform for a growing US telecommunications company. This project involves implementing BigQuery and Looker as the central platform for data reporting. It focuses on centralizing data, integrating various CRMs, and building executive reporting solutions to support decision-making and business growth.

responsibilities :


  • Develop and maintain a high-performance data processing platform for automotive data, ensuring scalability and reliability.
  • Design and implement data pipelines that process large volumes of data in both streaming and batch modes.
  • Optimize data workflows to ensure efficient data ingestion, processing, and storage using technologies such as Spark, Cloudera, and Airflow.
  • Work with data lake technologies (e.g., Iceberg) to manage structured and unstructured data efficiently.
  • Collaborate with cross-functional teams to understand data requirements and ensure seamless integration of data sources.
  • Monitor and troubleshoot the platform, ensuring high availability, performance, and accuracy of data processing.
  • Leverage cloud services (AWS) for infrastructure management and scaling of processing workloads.
  • Write and maintain high-quality Python (or Java/Scala) code for data processing tasks and automation.

requirements-expected :


  • At least 5 years of commercial experience implementing, developing, or maintaining Big Data systems, data governance and data management processes.
  • Strong programming skills in Python (or Java/Scala): writing a clean code, OOP design.
  • Hands-on with Big Data technologies like Spark, Cloudera, Kafka, Data Platform, Airflow, NiFi, Docker, and Iceberg.
  • Excellent understanding of dimensional data and data modeling techniques.
  • Experience implementing and deploying solutions in cloud environments.
  • Consulting experience with excellent communication and client management skills, including prior experience directly interacting with clients as a consultant.
  • Ability to work independently and take ownership of project deliverables.
  • Fluent English (at least C1 level).
  • Bachelor’s degree in technical or mathematical studies.

offered :


  • Work in a supportive team of passionate enthusiasts of AI & Big Data.
  • Engage with top-tier global enterprises and cutting-edge startups on international projects.
  • Enjoy flexible work arrangements, allowing you to work remotely or from modern offices and coworking spaces.
  • Accelerate your professional growth through career paths, knowledge-sharing initiatives, language classes, and sponsored training or conferences, including a partnership with Databricks, which offers industry-leading training materials and certifications.
  • Choose your preferred form of cooperation: B2B or a contract of mandate, and enjoy 20 fully paid days off.
  • Participate in team-building events and utilize the integration budget.
  • Celebrate work anniversaries, birthdays, and milestones.
  • Access medical and sports packages, eye care, and well-being support services, including psychotherapy and coaching.
  • Get full work equipment for optimal productivity, including a laptop and other necessary devices.
  • With our backing, you can boost your personal brand by speaking at conferences, writing for our blog, or participating in meetups.
  • Experience a smooth onboarding with a dedicated buddy, and start your journey in our friendly, supportive, and autonomous culture.

benefits :


  • sharing the costs of sports activities
  • private medical care
  • sharing the costs of foreign language classes
  • sharing the costs of professional training & courses
  • remote work opportunities
  • flexible working time
  • fruits
  • integration events
  • no dress code
  • video games at work
  • birthday celebration
  • employee referral program
  • Reimbursement for prescription glasses or contact lenses

Utwórz powiadomienie o ofertach pracy dla tego wyszukiwania

Senior Data Engineer (Spark) • Białystok, Podlaskie, Poland

Podobne stanowiska
Data Engineer – Data Lake (f/m/x)

Data Engineer – Data Lake (f/m/x)

Sii Sp. z o.o. • Białystok, Podlaskie, Poland
You will join an international project within the healthcare and life sciences industry, focused on building and evolving a modern Data Lake platform supporting large-scale data processing and anal...Pokaż więcej
Ostatnia aktualizacja: 14 dni temu • Promowane
Senior Data Scientist / ML Engineer

Senior Data Scientist / ML Engineer

Addepto • Białystok, podlaskie, Polska
Senior Data Scientist / ML Engineer.As a Senior Data Scientist / ML Engineer, you will have the exciting opportunity to work with a team of technology experts on challenging projects across various...Pokaż więcej
Ostatnia aktualizacja: ponad 30 dni temu • Promowane
Data Engineer with Microsoft Fabric (f/m/x)

Data Engineer with Microsoft Fabric (f/m/x)

Sii Sp. z o.o. • Białystok, podlaskie, Polska
Data Engineer with Microsoft Fabric (f/m/x).We are looking for an experienced Data Engineer with Microsoft Fabric to join our Data & Analytics Competence Center – a specialized unit bringing togeth...Pokaż więcej
Ostatnia aktualizacja: 6 godziny temu • Promowane • Nowe!
Data Engineer (Java + data science)

Data Engineer (Java + data science)

Britenet • Białystok, PL
Poszukujemy inżyniera o profilu T-shaped, gdzie głównym filarem jest inżynieria oprogramowania, a kompetencją uzupełniającą jest Data Science.Inżynieria Oprogramowania i Danych (Poziom Mid/Senior):...Pokaż więcej
Ostatnia aktualizacja: 10 godziny temu • Promowane • Nowe!
Senior Business Analyst – Trading & Reference Data (f/m/x)

Senior Business Analyst – Trading & Reference Data (f/m/x)

Sii Sp. z o.o. • Białystok, Podlaskie, Poland
We are looking for an experienced Senior Business Analyst to join an international project delivered for a leading organization operating in the financial market infrastructure and trading environm...Pokaż więcej
Ostatnia aktualizacja: 25 dni temu • Promowane
Embedded Linux/Kernel Engineer (f/m/x)

Embedded Linux/Kernel Engineer (f/m/x)

Sii Sp. z o.o. • Białystok, Podlaskie, Poland
Join Sii and play a key role in delivering enterprise-class data platforms to international clients.As an Embedded Software Engineer, you will be responsible for developing Linux kernel modules, de...Pokaż więcej
Ostatnia aktualizacja: 1 dzień temu • Promowane
Performance Engineer – DPU Systems (f/m/x)

Performance Engineer – DPU Systems (f/m/x)

Sii Sp. z o.o. • Białystok, Podlaskie, Poland
As a Performance Engineer, you will analyze, measure, and optimize the end-to-end performance of DPU-based systems, including computer, networking, storage, and security workloads.You’ll work close...Pokaż więcej
Ostatnia aktualizacja: ponad 30 dni temu • Promowane
Lead Data Engineer – AI & Cloud Technologies (f/m/x)

Lead Data Engineer – AI & Cloud Technologies (f/m/x)

Sii Sp. z o.o. • Białystok, Podlaskie, Poland
We are seeking a Lead Data Engineer to drive the design and delivery of advanced AI and Cloud solutions within the project in the automotive industry.In this role, you will lead the development of ...Pokaż więcej
Ostatnia aktualizacja: 29 dni temu • Promowane
System Analyst – sektor bankowy (f/m/x)

System Analyst – sektor bankowy (f/m/x)

Sii Sp. z o.o. • Białystok, podlaskie, Polska
Dołącz do naszego zespołu analityków systemowych, realizującego projekty w obszarach takich jak inwestycje, płatności krajowe i zagraniczne, kredyty hipoteczne czy bankowość codzienna.Aktualnie sku...Pokaż więcej
Ostatnia aktualizacja: ponad 30 dni temu • Promowane
Lead Data Engineer – AI & Cloud Technologies (f/m/x)

Lead Data Engineer – AI & Cloud Technologies (f/m/x)

Sii Poland • Białystok, Województwo podlaskie, Polska
We are seeking a Lead Data Engineer to drive the design and delivery of advanced AI and Cloud solutions within the project in the automotive industry.In this role, you will lead the development of ...Pokaż więcej
Ostatnia aktualizacja: 20 dni temu • Promowane
Senior Observability Engineer (f/m/x)

Senior Observability Engineer (f/m/x)

Sii Sp. z o.o. • Białystok, Podlaskie, Poland
A global consumer organization is seeking a Senior Observability Engineer.This role goes beyond tooling- you will design, implement, and continuously enhance observability capabilities to support b...Pokaż więcej
Ostatnia aktualizacja: 22 dni temu • Promowane
System Validation Engineer – DPU Team (f/m/x)

System Validation Engineer – DPU Team (f/m/x)

Sii Sp. z o.o. • Białystok, Podlaskie, Poland
Join our Data Processing Unit (DPU) development team, working on state-of-the-art hardware for high-performance computing, storage, security, and networking acceleration.As a System Validation Engi...Pokaż więcej
Ostatnia aktualizacja: ponad 30 dni temu • Promowane
AI Engineer – pharmaceutical industry (f/m/x)

AI Engineer – pharmaceutical industry (f/m/x)

Sii Sp. z o.o. • Białystok, Podlaskie, Poland
We are looking for an experienced AI Engineer to help us responsibly shape and govern the rapidly evolving ecosystem of AI-powered tools.This is a strategic, engineering-focused role for someone wh...Pokaż więcej
Ostatnia aktualizacja: ponad 30 dni temu • Promowane
C/C++ Firmware Engineer (f/m/x)

C/C++ Firmware Engineer (f/m/x)

Sii Sp. z o.o. • Białystok, Podlaskie, Poland
Join Sii and play a key role in delivering enterprise-class data platforms to international clients.As a Firmware Engineer, you will be responsible for low-level firmware development, hardware brin...Pokaż więcej
Ostatnia aktualizacja: 1 dzień temu • Promowane
Business - System Analyst (f/m/x)

Business - System Analyst (f/m/x)

Sii Sp. z o.o. • Białystok, podlaskie, Polska
Dołącz do Sii i pracuj przy nowoczesnych projektach IT realizowanych dla międzynarodowego klienta z branzy medycznej.Poszukujemy osoby na stanowisko Business-System Analyst, która będzie kluczowym ...Pokaż więcej
Ostatnia aktualizacja: ponad 30 dni temu • Promowane
Senior AWS Platform Engineer & AI Automation Lead

Senior AWS Platform Engineer & AI Automation Lead

Transition Technologies PSC • Białystok, Województwo podlaskie, Polska
A technology solutions provider located in Białystok is looking for a skilled developer to manage R&D efforts including cloud application design and implementation based on AWS.Candidates must have...Pokaż więcej
Ostatnia aktualizacja: ponad 30 dni temu • Promowane
Senior AI Engineer

Senior AI Engineer

SoftwareHut Limited Liability Company • Białystok, Polska
At SoftwareHut, we’re expanding our team to support the growth of cutting‑edge artificial intelligence projects for clients across various industries.If you have strong analytical and communication...Pokaż więcej
Ostatnia aktualizacja: 4 godziny temu • Promowane • Nowe!
AI Architect (f/m/x)

AI Architect (f/m/x)

Sii Sp. z o.o. • Białystok, Podlaskie, Poland
As a team member, you will have the possibility to participate in the design and implementation process of the Service Management & Orchestration (SMO) platform.SMO is a fully automated system for ...Pokaż więcej
Ostatnia aktualizacja: ponad 30 dni temu • Promowane