Position: Senior Big Data Engineer (Poland)
Company: Grid Dynamics
Category: Other
Experience required: 5y
English level: upper
Published: 2023-01-01T00:00:00+02:00

Grid Dynamics is looking for a Senior Big Data Engineer to join the well known brand. To be a part of a multilocation team to build new frameworks and pipelines feeding ML solutions to all those platforms, Marketing, search and gallery, and many others.

**Project description**
Over the past number of years the company engages to a number of partners and built their own ML \ AI solutions - sales replenishments, action IQ, etc. The core idea is being a part of multilocation team to build new frameworks and pipelines feeding ML solutions to all those platforms, Marketing, search and gallery, and many others.
As part of duties:
integrate new data sources, prepare and sanitize data itself, prepare data sets for ML algorithms, improving data governance.
Create more data sets for data sources, for CRm\Marketing (Epsilon-B)
Automate manual providing data to CRM\Marketing and other solutions
Pick up data, utilize, and identify data sources for BSG re-fetching from Data lake
Working closely to DS, create reusable code (DAGs), trying to build utility libraries, frameworks

**Client description**
Client - huge US network of malls and well-known logo

**Details on tech stack**
Standard Big Data stack + Cloud

**Min requirements to the candidate**
3+ years of professional experience
2+ years of hands-on experience with Big Data, Analytics
2+ years of hands-on experience with ETL and ELT data modeling
Retail industry experience preferred
Experience with Big Data tools (Apache Spark, Apache Airflow), Hadoop (HDFS, Hive, YARN),
Proficient in query languages (e.g., SQL, Hive)
Proficient in Python.
Experience working with traditional warehouse and correlation into Hive warehouse on Big Data technologies
Experience with setting data modeling standards in Hive
Proficient in API development and consumption, data modeling, preparation, manipulation, loading and optimization, warehousing techniques, and technologies
Expert level understanding of ETL fundamentals and building efficient data pipelines
Experience with AWS CLI and Linux
Experience implementing security best practices
Experience with SOA, IaaS, and Cloud Computing technologies
Knowledge of setting standards around data dictionary and tagging data assets within Data Lake for business consumption
Good to have experience with data visualization tools (e.g., Tableau)
Experience with Jenkins, GIT, and Airflow
Experience with continuous software integration, test and deployment
Experience with agile software development (e.g., Scrum) and project management tools (e.g., JIRA)
Ability to work within a dynamic programmatic environment with evolving requirements and capability goals
Strong customer service and communication skills
Ability to collaborate and share knowledge with a diverse team of individuals across Technology and Business
Self-motivated.  Capable of working with little or no supervision.

**Nice to have requirements to the candidate**
AWS Cloud Computing technologies
Experience with AWS CLI and Linux
ML:
- coming from personalization
- time serious, modeling elements, classification
- deep learning

**What tasks the candidate will work with?**
As part of the Big Data Analytics team, using Big Data technology, you will be responsible for designing, developing, implementing and supporting data analytics solutions utilizing AWS cloud architecture.
·    Play a role in developing a high-performance data platform, integrating data from a variety of internal and external sources, in order to create certified data domains to support current and future analytics and data science needs
·    Build processes supporting optimal data transformation, data structures, metadata and data flow pipelines to consume data from source systems
·    Assemble large, complex data sets that meet business requirements
·    Identify technical implementation options and issues
·    Support and maintain production data workflows, perform root cause analysis and debug issues
·    Mentor consumers of the data and analytics teams
·    Provide technical coaching to other engineers and partners across the organization
·    Partner and communicate cross-functionally across the enterprise
Evaluate, identify, and offer innovative options and solutions
·    Interact with business teams to gather, interpret and understand their business needs and create detailed design specifications
·    Foster the continuous evolution of best practices within the development team to ensure data quality, standardization and consistency
·    Continuously seek Data Lake platform, ingestion, governance and service improvement opportunities that ensure optimal use of the environment and improve stability

**What is good about project?**
Well known brand
Huge infrastructure and real big data
Strong team
Very good relationship with the client gained over a number of years of partnership