Senior Data Consultant (Hadoop, Spark)

Type of contract: B2B contract
Salary range: 125-163 PLN net/h

What will you do

You will work as a key member of a technical team alongside Engineers Data Analysts and Business Analysts contributing to a collaborative Agile development process while designing developing and maintaining scalable data solutions in a dynamic DevOps environment.

Your tasks

• Define and contribute to software design and development using Pyspark
• Automate testing of new and existing components
• Promote development standards through code reviews and mentoring
• Provide production support and troubleshooting
• Implement tools and processes ensuring performance scalability and monitoring
• Collaborate with Business Analysts to interpret and implement requirements
• Participate in planning sprint reviews and retrospectives
• Contribute to system architecture and design

Requirements

• Experience with Pyspark or Scala development and design
• Experience using scheduling tools such as Airflow
• Knowledge of Hadoop ecosystem including Spark Hive YARN and ETL frameworks
• Strong SQL and RESTful services knowledge
• Experience working on Unix or Linux platforms
• Hands-on experience building data pipelines using Hadoop components
• Experience with Git GitHub Jenkins Ansible and JIRA
• Understanding of big data modelling using relational and non-relational techniques
• Experience debugging code and communicating findings to development teams
• Openness to work 2 days a week from our clients office (Kraków)

Nice to have

• Experience with Elasticsearch
• Experience developing Java APIs
• Experience in data ingestion processes
• Understanding of cloud design patterns
• Exposure to DevOps and Agile methodologies such as Scrum and Kanban
• Experience with Spark streaming
• Experience with Apache Airflow in production
• Experience with Hadoop ecosystem in enterprise environments
• Knowledge of Python backend services
• Experience with Scala for high performance systems
• Experience in data integration and ETL processes
• Knowledge of PL/SQL
• Experience with Linux and Unix system operations

Required Experience:

Senior IC

Employment Type : Contract
Experience: years
Vacancy: 1