Position: Senior Data Engineer, Online Genealogy Service
Company: DataArt
Category: Data Science
Experience required: 3y
English level: upper
Published: 2021-04-01T00:00:00+03:00

About the vacancy

The client is an international company that provides an online genealogy service that helps its clients understand their past and family history.

The project’s goal is to create a scalable platform for the further expansion of the client's business. Technologies the team is using regularly at this time include MySQL, ZFS, XML, Ubuntu, Docker, Ruby, Rails, Solr, AWS/Azure, Tensorflow / Python.

The project uses lean-agile approaches in the product development process.

Responsibilities
- Maintenance of data workflow and ingest of our scanned newspaper image data. This involves handling a lot of data throughput in a reliable and consistent way
- An element of DevOps and Systems Administration. The team works with a significant number of physical and virtual servers, handling deployment pipelines, etc.
- An element of data management: managing file systems; managing databases; managing data ingest into Solr and managing Solr at scale
- Management of some internal, web-based tools that our QC team uses to validate images before they’re published

Must have
- 3+ years of experience working in a data engineering or developer role
- Good knowledge of multiple programming languages
- Experience using one or more scripting languages (Python, Ruby, bash)
- Experience with XML processing
- Experience using ZFS for managing snapshots and replications
- Ability and experience handling large amounts of data
- Practical experience with SQL and any DB engines like MSSQL or MySQL
- Comfortable working from the terminal in Linux/Unix
- Experience in the Ubuntu Linux environment
- Hands-on approach to getting stuff done
- Curiosity to learn and widen your skillset
- Excellent communication skills and experienced team worker
- Good spoken and written English
- We don’t need candidates to have a working knowledge of all, or even potentially any of the above technologies but it’s best candidates are aware that this is the environment they’d be working in and are interested/motivated to learn in these areas

Would be a plus
- Experience with Ruby and Ruby on Rails
- Experience with AWS and/or Azure 
- Experience working with Tensorflow.js
- Experience with open source search platforms (Apache Solr is preferred)
- Experience with JavaScript/Node.js
- Focus on quality, with testing experience and a willingness to pair collaboratively
- Experience with Docker, Git, Kubernetes
- Knowledge and experience working with Machine Learning
- Working knowledge of, or an interest in the image data processing

Learn more about our policy of equal opportunities in employment