Position: Senior Full-Stack Engineer w/strong web development experience
Company: Mellivora Software
Experience required: 5y
Category: Java
English level: upper
Published: 2023-05-01T00:00:00+03:00

We are looking for a unicorn who would possess these skills:

📌 Substantial development experience in Go, Java, Rust, or Python
📌Kubernetes, Kafka, RabbitMQ, CI/CD, docker, containerization
📌REST API design (gin, Spring Boot, Rocket, FastAPI, etc), concurrent processing, larger-scale datasets (tens of TB), and rules engine design
📌  Some experience with APM tools (DataDog, NewRelic, etc.) for application monitoring and performance tuning
📌English at C1 or better

Long-term project, full-time, fully remote. Required 2 hours overlap with US EST working hours.

DETAILS

# Senior Software Engineer: Data Mastering

Our client has engaged us to write a high-volume, high-throughput data mastering system capable of processing tens of millions of records per day while achieving data mastering accuracy on par with similar commercial systems.

For this, we plan to launch a greenfield project that will combine REST API design, concurrent processing, larger-scale datasets (tens of TB), and rules engine design. We are looking for a full-stack, senior software developer who is excited to analyze, design, implement, and enhance such a system.

## What the position entails

This position is truly full-stack, which means that you will:

- In conjunction with the rest of the team, analyze existing systems to gather and validate requirements for this project
- Design and implement processing pipelines, including:
- Message brokering engine (Kafka, RabbitMQ)
- Pipeline design (state machine, exception handling, etc.)
- Design and implement a data mastering rules engine including matching, merging, and history retention/representation.
- Design and implement storage mechanisms for high-speed and high-volume storage and retrieval, using some combination of RDBMS, NoSQL, search technology (Opensearch, etc.), and other technologies as appropriate.
- Design and implement REST API interfaces to the system, including:
   - Data representation
   - Authentication and authorization
   - Asynchronous request handling
   - Error and exception handling
 - Design and implement data management UI for admin users
- Implement all of the above using CI/CD, unit and integration testing, containerization, and Kubernetes deployment orchestration


## Required Skills

In the following list of required skills, "substantial" means that you've personally implemented and supported at least two different production projects using the following technologies, and that those technologies have been used by live users at a non-trivial scale. "Some" experience means that you've used the technology, although perhaps not as the primary designer/implementor, in production applications, and/or at meaningful scale.

- Spoken language: You speak English at a C1 level or better
- Programming language: Substantial development experience in Go, Java, Rust, or Python.
- Framework: Substantial REST API development in a web framework designed for convention-based development of API endpoints (gin, Spring Boot, Rocket, FastAPI, etc.) including
- Authorization/authentication
- Schema declaration (such as JSON schema)
- References
- Concurrent processing: Some experience designing and implementing state machine pipelines for high-volume concurrent processing of events or data records. This should include a comfortable working knowledge of Kafka, RabbitMQ, or a similar open-source message broker.
- Database design:
- Substantial experience designing, implementing, and tuning RDBMS schemas to handle hundreds of GB or more of data, ideally using sharding or other horizontal scaling approaches.
- Some experience designing and implementing search engines based on Elasticsearch, Opensearch, Solr, or Lucene, including:
- Scaling
- Faceted search
- Phonetic matching
- Index tuning
- Some experience with multi-tenancy in both the RDBMS and search engine context
- Substantial experience using Docker for development; some experience using Docker and/or Kubernetes for CI/CD and production deployment
- Substantial experience implementing unit, functional, and integration tests for a web service
- Some experience with APM tools (DataDog, NewRelic, etc.) for application monitoring and performance tuning.

We do not require any specific university degree, certifications, or work history for this position. Candidates will be evaluated on their work experience,  development portfolio, and assessment results from an assessment that we will provide during the interview process.

## Team

In this role, you will be part of a remote scrum team with members located potentially anywhere in the world. We communicate through either Slack or Google Chat depending on team composition and preference, and track our activities through Jira and Confluence.

You should be comfortable working in an agile scrum framework, be comfortable with estimating and committing to deliverables, and enjoy working with other team members to ensure that the team's objectives and commitments are well-considered, well-defined, and delivered on time.