Position: Senior Big Data Lead, Global Music Company
Company: DataArt
Category: Data Engineer
Experience required: 3y
English level: intermediate
Published: 2022-02-01T00:00:00+02:00

Our client is a global music and entertainment conglomerate headquartered in the US with offices in over 40 countries. The company has some of the world’s biggest music stars on its roster and provides innovative analytical tools both to their signed artists and to independent musicians.

The client is actively integrating automation, data science, and other modern technologies into its operations and plans to expand the team in various projects, such as processing tools for new artists and tracks, license and rights management, marketing, and A/R activity.

Many projects and applications are built around the concept of a unified data storage that is capable of hosting all sales and consumer data in one place and providing a wide variety of customizable reports to stakeholders at all levels of the company. That helps develop products that best fit the client’s business goals.

Technology stack: Spark 3.0+ + Scala; OOP, DI (Guice); AWS (Lambda, S3, Secrets Manager); GCP (BigTable); Databricks; DataDog.

Responsibilities
- Elicit business needs and requirements, determine the project's scope, team, and timeline
- Lead project implementation from ideation to completion
- Serve as a top-level escalation point for technical and process concerns
- Select and integrate any Big Data tools and frameworks required to provide the requested capabilities
- Implement ETL process
- Monitor performance and advice on any necessary infrastructure changes
- Establish Agile delivery processes in a team

Must have
- Experience architecting complex BigData solutions
- Expertise with Spark (+ Scala)
- Hands-on experience with processing frameworks, or other Big Data technology areas: data ingestion, consolidation, streaming, or batching
- Expertise in the RDBMS, SQL and NoSQL engines
- Hands-on experience waking with Spark
- Cloud practice with AWS (Lambda, S3)
- Spoken English

Would be a plus
- Experience leading a team of 2+ Data Engineers
- Practice implementing Data Lakes, Data Warehousing or analytics systems
- GCP (BigTable)