Position: Senior Ingestor Big Data Developer
Company: Luxoft
Experience required: 3y
Category: Data Science
English level: intermediate
Published: 2020-03-01T00:00:00+02:00

The goal of the program is to solve the 'billion-mile validation challenge' by providing end-to-end features and service for Level 3+ Autonomous Driving software development and validation that can be deployed as a collaboration eco-system accessible to the automotive market worldwide.
Responsibilities:
• Performance tuning of high data volume ingests into Hadoop ecosystem clusters
• R&D driven feature development around ingest pipeline optimization, filtering, data consistency
• Performance benchmarking on bare metal, hybrid cloud - heavy load, penetration testing
• Develop Airflow central ingest orchestration and ingestor auto deployment
• Define and implement improvements in technologies, infrastructure and software or architecture components
• Design and develop prototypes & proof-of-concept solutions in support of presales activities as well as support the development of standardized analytic offerings across multiple industries
• Establish demos to showcase ingest capabilities for DXC clients and fairs/congresses
Mandatory Skills:
• Bachelor or Master degree required in informatics, business informatics, business administration or equivalent qualification
• Minimum 2+ years of experience with Hadoop distributions and innovative technologies in Hadoop ecosystem (e.g. Spark, Hive, HBase)
• Proven experience in all phases of a Big Data project: Concept & design, development, implementation, change and operation
• Advanced know how in Python (3+ years)
• Hands-on experience in CI/CD, Ansible playbooks & deployment automation, Jenkins (2+ years)
• MapR DB/Key value or Hbase project experience
• Monitoring experience, e.g. Kafka
• Strong customer focus, assertiveness and precise method of operation
• Possesses an understanding of architectural dependencies of technologies in the customer's analytic environments
• Able to communicate and present internally and externally in a confident manner
• Highly motivated team player
Nice-to-Have Skills:
• Experience in MapR streams
• Job scheduling/orchestration, e.g. Airflow
• Docker, Kubernetes (1+ years hands-on experience)
Languages:
English: Upper-intermediate