Position: Intermediate (Middle) Big Data Engineer
Company: SoftServe
Category: Data Engineer
Experience required: 2y
English level: upper
Published: 2022-05-01T00:00:00+03:00

**WE ARE**
SoftServe is a global digital solutions company with headquarters in Austin, Texas, founded in 1993 in Ukraine. 13,000+ associates are currently working on 1,100+ projects with clients in the USA, Europe, and APAC region. We are about people who create bold things, make a difference, have fun, and love their work.
We are successfully cooperating with our client in helping people move forward with credit, providing products that responsibly meet their needs. Whether this is offering new ways of accessing credit with a leading retailer or providing tools to facilitate customers’ management of their accounts, our client will continuously strive to help customers responsibly make the most of their credit.
We offer an approach to financial services in touch with people and their lives. It is an approach grounded in customer knowledge and differentiated by our passion to deliver the products, services, tools, and expertise that best meet our customers’ needs.

**YOU ARE**
• Proficient in Big Data integration technologies such as Spark, Scala, AWS Data Pipeline for orchestration, Dremio, Glue, AWS Athena, AWS S3, EMR
• Skilled in API and library design
• Proficient with traditional database SQL technologies (Oracle, SQL Server, DB2)
• Experient in the integration of data from multiple data sources
• Able to write high-performance, reliable and maintainable code
• Pretty knowledgeable of NoSQL database structures theories, principles, and practices
• Confident in CI/CD best practices, multi-threading, and concurrency concepts
• Accustomed to cloud deployment
• Familiar with the fundamentals of Linux scripting language

Nice to have such expertise as:
Python skills
Building applications for cloud environment

**YOU WANT TO WORK WITH**
• Modeling, manipulation, transformation, and loading of the Data Lake technology layers
• Designing and coding of data transformation and loading processes through the various layers of the Data Lake using Big Data open-source technologies such as Kafka, Spark, Scala, and associated AWS ETL tooling
• Owning all of the Data Content, manipulation, business rules, and associated processes within the Data Lake
• Defending quality and simplicity in the system code
• Leading the enforcement of quality within the data landscape and Agile approach
• Monitoring and profiling performance and suggesting/implementing solutions where appropriate
• Recommending and testing new data structures, and physical data layouts, to improve throughput and performance
• Endorsing the new Data Lake technology across the organization to address a broad set of use cases across data science and data warehousing
• Researching, testing, and recommending new layers or products in the Data Lake Stack as these fast-moving technologies develop
•  Educating and training other resources in the organization whenever needed

**TOGETHER WE WILL**
• Model, manipulate, transform and load Data Lake technology layers
• Build knowledge of all data resources within Client and prototype new data sources internally and externally to Client
• Building knowledge of all existing data manipulation within both data warehouse systems and within the business marts
• Design and code data transformation and load processes through the various layers of the Data Lake
• Get a great deal of learning and development opportunities along with our structured career path