Position: Middle Data Engineer
Company: Exadel
Category: Other
Experience required: 2y
English level: intermediate
Published: 2021-03-01T00:00:00+02:00

The company delivers advanced mobile, artificial intelligence, and connected car technologies through its platform, connecting a vibrant network of 350+ insurance companies, 24,000+ repair facilities, OEMs, hundreds of parts suppliers, and dozens of third-party data and service providers. The collective set of Customers’s solutions inform decision-making, enhance productivity, and help clients deliver faster and better experiences for end consumers.  
The Customer’s company was ranked #17 in the Top 100 Digital Companies in Chicago in 2020 by Built in Chicago, an online community for digital technology entrepreneurs in Chicago.  And, was named to Forbes best mid-sized companies to work for in 2019 – an important accolade and retention tool for the 2,600+ full-time company employees (alongside 350 dedicated contractors).  
The Company’s corporate headquarters is in downtown Chicago within the historic Merchandise Mart—a certified LEED (Leadership in Energy and Environmental Design) building that is also known to be a “technology” hub within the broader metro. 

Must Have Qualifications:

2 + years of experience in building, maintaining, and supporting complex data flows with structural and unstructural data 
Proficiency in Python and PySpark
Hands on experience in HDFS, HIVE, and SQOOP   
Familiarity with Apache Kafka and Apache Airflow
Ability to use SQL for data profiling and data validation 
Experience in Unix commands and scripting
Master’s or Bachelor’s degree 

Preferred Qualifications:
Understanding AWS ecosystem and services such as EMR and S3
Experience and understanding of Continuous Integration and Continuous Delivery (CI/CD)
Understanding in performance tuning in distributed computing environment (such as Hadoop cluster or EMR)
Familiarity with BI tools (such as Tableau and MicroStrategy) and commonly use data modeling techniques. 

English level:
Intermediate+

Responsibilities:
Build end to end data flows from sources to fully curated and enhanced data sets. This can include the effort to locate and analyze source data, create data flows to extract, profile, and store ingested data, define and build data cleansing and imputation, map to a common data model, transform to satisfy business rules and statistical computations, and validate data content
Modify, maintain and support existing data pipelines to provide business continuity and fulfill product enhancement requests 
Provide technical expertise to diagnose errors from production support teams
Coordinate within on-site teams as well as work seamlessly with the US team