Position: Senior Software Engineer
Category: Scala
Experience: 11 years | English: upper

- As Big Data Engineer - I design and implement Real Time Ingestion for billions of events to Data Lake with 10 minutes Time Lag SLA and exactly once Delivery semantics In Spark. First version of System was Kafka to S3(parquet.gzip files in Hive table structure), later-on I plan and implement data and flow migration to S3(delta.io).
- As Big Data engineer /Senior Software developer -  I design and implement row-level deletes over 25pb DataLake on s3(immutable files). This was a spark job, which can scale to thousands instances and support SLA in 24 hours, with an unpredictable number of delete requests like the GDPR/CCPA scenario.
- As Senior Software developer/Big Data Engineer - I am good at optimizing big data infrastructure. Once, I was able to reduce AWS cost to 3 million dollars from 7.
- As a Senior Software developer - I was able to develop Spark Sink for Kafka, with exactly-once delivery semantics. 
- As DevOps - I develop a fully automated terraform module for EKS cluster, deploy there Dremio and Presto in HA mode with proper monitoring and alerting. And automate performance tests to compare these tools with different infrastructure. For example disk options - EBS IO2/gp3 vs directly attached SSD.
- As DevOps - I Implement CI/CD for spark and spark structure streaming jobs. With end2end Integration test in near production environment on YARN. And then migrate them to k8s.
- As Technical lead  - I am proud that I teach at least 1 person as DevOps and 3 as Developers. These people do not have IT experience before, but they all have technical backgrounds. Also I have positive feedback from my team about my help in explaining complicated design parts and complex tools.
- As Technical lead - I am proud of my design decision to make a bet on Spark and K8s(cloud native stack) in its early phase.
E g. I started working with k8s and Spark when they were 1.7 and Apache Spark 1.1. 
And this helps the company to be on the cutting edge(help find good developers) and move faster and reduce legacy from day one. And what is much more important is the SCALE from a few thousands to 38 billions input events during 1 year.
While most of my experience is Software Developer/Technical Lead in the Big Data area.
I also have experience as a technical consultant in Big Data and Devonshire domains.
I'm self-motivated and looking for an opportunity that can really excite me.

My architecture/design skill is based on rich software developer experience. 
As an experienced developer with nine-year+ record of commended performance in back-end /(big)data engineering. Well-versed in all layers of the developed software, with strong working knowledge in the design of distributed and fault-tolerant systems, with monitoring and deploying parts.
I think I am a good candidate for the Technical Lead position - because aside from my rich software development experience, I also have wide knowledge and experience in DevOps, a mostly modern cloud-native tool. But also some more classic ops tooling like chef/ansible, vagrant, etc.

Please find my working experience on LinkedIn.

The technology stack that I am curious during last year:
- GraalVM
- Apache Arrow
- HTAP DB: TiDB.
- New SQL DB: Vitess, CockroachDB
- Big Data OLAP DB: ClickHouse. Pinot(Startree)
- Stream data processing: Akka, Heron/GearPump, Flink
- Big Data ad-hoc query engine: Dremio. 
- ci/cd: JenkinsX, ArgoCD, Flux.
I prefer to avoid work with non-typed programming languages. Not only as Developer, but also as Team Lead too.
I am sure they have a good and even easy to write system, but make it very hard to support and evolve in scale.