Position: Senior Data Engineer
Company: AgileEngine
Experience required: 3y
Category: Data Science
English level: intermediate
Published: 2021-01-01T00:00:00+02:00

**What is required?**
4+ years experience in software development
BS in Computer Science or equivalent practical experience
Strong SQL knowledge
Knowledge of SQL Server or other relational database management systems (Oracle, PostgreSQL, MySQL)
Practical experience implementing Apache Spark jobs: knowing its features on partitioning, grouping, joins, importing the data into the cluster, exporting the data from the cluster
Practical experience building ETL/ELT data pipelines from scratch or using existing frameworks like StreamSets DataCollector (SDC), Fivetran, Apache NiFi
Practical experience of using Data Warehouse(s) with Snowflake or other DW-oriented databases (Google BigQuery, Redshift, etc.)
Practical experience with one of the following languages: C#, Python, Java, Scala
Intermediate English

**What will be a plus?**
AWS basics
AWS DMS
Kubernetes basics
Spark Operator

**What will you do?**
Build Apache Spark jobs
Build data pipelines(ETL flows) to connect different sources of data together
Identification of performance bottlenecks
Serve as a go-to Data Engineering consultant for other team members (Bloomberg Tax specifically requires that, since they are mostly SQL/RDBMS/.NET shop)
Optimize SQL queries
Producing high-quality code
Communication with Engineering & Product management
Alignment with existing development teams