Position: BigData engineer Scala/Java/Python
Category: Data Engineer
English level: fluent
Years of experience: 7


Scala, Java,  Python

Apache Flink(Flink SQL, Datastream)

Apache Kafka using both Java and Scala APIs (Confluent platform,  Schema registry, KSQL, Streams API, Processor API,Kafka Connect)

Apache Spark (RDD,Datasets, Dataframes, SQL, Streaming,Structured Streaming)

Apache Airflow (DataProc, Spark, Python, BigQuery and other operators)

Apache Hadoop (Google cloud Dataproc, Amazon EMR, HDP 2.5)
Athena(Presto), Glue, Hive
Apache Beam (Java and Scala API, Dataflow, Flink, Scio)

AWS, Google Cloud platforms
Akka (Actors, Http, Play)
Google Pub/Sub, AWS MSK, RabbitMQ 
Apache Avro, Parquet
DB: Google BigQuery, 
BigTable(Hbase), Apache Phoenix,  MySQL, MS SQL Server,

Development tools: IntelliJ IDEA, Git, Maven, SBT, Docker/ Docker Compose
Agile development approach, Confluence, Jira
A product company, stock options plan, real-time, high load