Summary Seasoned data engineer professional adept at understanding mandates, developing plans, leading and implementing enterprise-wide solutions. Complex problem-solver with an innovative approach. A proven track record with experience in all facets of data engineering from ETL to data consumption.Extensive experience in Hadoop and cloud computing platforms with focus on automated CICD. Strong programming skills in design and implementation using Java J2EE, Scala, SQL, and other scripting languages. Excellent leadership management qualities with excellent communication and interpersonal skills. Skills Design and Create scalable platforms and products in data realm including ETL in hadoop ecosystem, data services and SQL NoSql databases Create and review high level design documentation, architecture diagrams Design and create automated CI CD for various data platform both on-prem and cloud ecosystems Lead and participate in all aspects of SDLC collaborating with product owners, engineering, devops and delivery teams to align and deliver products. Test software products to ensure that the software products developed by the engineering team meet the company s quality and standards Guide and mentor engineering team on technical matters, including design, architecture, coding practices Experience02 2020toCurrentLead Data EngineerTransamerica Life Insurance Company Charlotte,NC, Managing 3 scrum teams of ETL, services and devOps across data platforms Oversaw and develop the Large-scale real data processing pipelines to handle transaction and real time data using kafka and spark to various databases as part of Data Management Platform team. Leading migration from hbase to aerospike, and solr to elasticsearch which including 30TB historical data migration using spark. Collaborate with devOps, network, infra teams to create data plaform architecture and establish project plans and timeline Design and create seamless CICD pipelines and provide solutions to all application both on-prem and cloud applications. And also manage build engineering team Managing entitlement of the applications and tools extensively working with security, risk and architecture teams to define and implement RBACs across data platform. Working closely with security, infrastructure and vendors to remediate software and infrastructure vulnerabilities including in house and third party applications part of data platform. Recipient of Values Champions Award for 2021 at Early Warning11 2012to02 2020Senior Data EngineerChange Healthcare Kennesaw,GA, Developed Batch and streaming spark application in scala to load data kafka, hbase and solr on cloudera based Hadoop platform in DMP Explored with the Spark improving the performance and optimization of the existing algorithms in Hadoop using Spark Context, Spark -SQL, Data Frame, PairRDD s, Spark YARN. Developed ETL pipelines using Streamsets to move data from kenisis kafka to various datastores Extensive experience in designing and implementation of continuous integration, continuous delivery, continuous deployment through Chef. ansible, bamboo, gitlab and harness Setup full CI CD pipelines so that each commit a developer makes will go through standard process of software lifecycle and gets tested well enough before it can make it to the production. Created Streamsets pipelines to consume data from Amazon Kinesis and Redhsift for data processing04 2011to09 2012Software EngineerGeneral Motors West Chester,OH, Involved in analyzing system specifications, designing and development for multiple J2EE wholesale applications Performed a shakeout test to the code migrated to UAT for the new customer setups, defects and for the enhancements. Performed acceptance testing and conducted functional testing for the customer using WebMethods and UNIX environment. Extensively worked on PM Online application project from scratch Conducted full lifecycle software development from planning to deployment and maintenance. Reviewed and modified unit and integration tests to improve software quality and reliability. Performed back-end testing using SQL queries to validate the data in the back-end database. Used SQL to validate backend database changes, deletes and update.12 2007to03 2011Software DeveloperWalt Disney Co. Cincinnati,OH, Delivered code to meet functional or technical specifications. Designed front-end and back-end solutions for test-driven development. Participated in code review meetings, providing input on bugs, inefficiencies and potential solutions to emergent issues. Modified existing software systems to enhance performance and add new features. Performed regression and performance tests for updated systems. Education and TrainingExpected intotoBachelor of Science Electrical EngineeringUniversity of South Alabama-Mobile,ALGPA  SEP  HiETLData EngineerDallas tx-Onsite Need only Local to DALLAS, TX- Onsite 1) Need Data Engineer with heavy Python Development Background.2) Must have worked on UNIXLINUX, Extensive Experience with SQLUnixLinux scripting is a must.3) Strong exp on AWS Cloud, Experience in Automating ETL process with Python and Automation around AWS Data Infrastructure4) AWS Glue is must, with hold on ETL Concepts.5) Good to have Testing exp and familiarity with Test Strategies. Regards,Nisha JhaTechnical RecruiterNisha.j peritussoft.com