For the given job description <<As a Solution Architect - Data Engineering, you will work for an American multinational technology company focusing on various technological domains such as artificial intelligence, online advertising, search engine technology, cloud computing, computer software, quantum computing, e-commerce, and consumer electronics. The job requires expertise in Databricks, a cloud-based data engineering platform.
Key responsibilities:Provide technical leadership in a post-sales capacity to guide strategic customers as they design and implement big data projects, ranging from architectural design to data engineering to model deployment.Identify and drive new initiatives that enable customers to turn their data into actionable business value and align these initiatives with their business outcomes for continued success.Architect production-level workloads, including end-to-end pipeline load performance testing and optimization.Deliver tutorials and training to drive community adoption (including hackathons, conference presentations, etc.)Work closely with Engineering and Product Management to drive the direction of the Databricks product.Contribute to Databricks SME community. Required SkillsHands-on and technical expertise with Apache SparkProven experience of design and implementation experience in big data technologies including Hadoop, NoSQL, MPP, OLTP, OLAP5+ years Experience working as either: Software EngineerData Engineer: query tuning, performance tuning, troubleshooting, and debugging Spark andor other big data solutions.Comfortable programming in Python, Scala, or JavaExperience using and designing solutions on cloud infrastructure and services, such as AWS, Azure, or GCPExperience with Development Tools for CICD, Unit and Integration testing, Automation and Orchestration, REST API, BI tools, and SQL Interfaces. E.g. Jenkins.Excellent communication skills and comfortable presenting to strategic stakeholdersleadership.Experience in customer-facing pre-sales, post-sales, technical architecture guidance, or consulting.Passionate about learning new technologies and making customers successfulData ScienceML Engineer experience in model selection, model lifecycle, hyper-parameter tuning, model serving, and deep learning, using tools like MLFlow desired.
 Life at CapgeminiCapgemini supports all aspects of your well-being throughout the changing stages of your life and career. For eligible employees, we offer:Flexible workHealthcare, including dental, vision, mental health, and well-being programsFinancial well-being programs such as 401(k) and Employee Share Ownership PlanPaid time off and paid holidaysPaid parental leaveFamily-building benefits like adoption assistance, surrogacy, and cryopreservationSocial well-being benefits like subsidized backup childelder care and tutoringMentoring, coaching, and learning programsEmployee Resource GroupsDisaster Relief>> the resume: <<SummaryData Engineeringwith experience in Design, Development, Implementation and support ofData Warehousingfor over8 years. Experienced in completeSoftware Development Life Cycle (SDLC),software Testing Life Cycle (STLC),SDLC methodologies. Extensively worked onInformatica Designer Tools(Source Analyze, Warehouse Designer, Mapping Designer, Mapplet Designer, and Transformation Developer) and Workflow Manager Tools (Task Developer, Worklet and Workflow Designer), Workflow Monitor and informatica Power exchange. Experience in working with using Informatica inSAP HANA,Oracle,MS SQL,TeradataandDB2 environments. Hands on experience in migrating on premise ETL toGoogle Cloud Platform(GCP)using cloud native tools such asBIG Query,Cloud Data Proc,Google Cloud Storage,Composer.Practical understanding of the Data modeling concepts likeStar - Schema Modeling, snowflake Schema Modeling, Fact and Dimension tables. Also experience in Optimizing Database querying, data manipulation using SQL andPL/SQLin Oracle, flat files and SQL server database. Experienced in implementingChange Data Capture (CDC)using informatica Power Center for Oracle and SAP Systems. Experienced in debugging mapping by analyzing the data flow and evaluating transformations. Experienced in performance tuning of data flow through source, target, sessions and mappings,identifying and resolving performance bottlenecks at various stagesusing techniques like Database tuning and Session Partitioning. Experienced in test strategy, developing test plan, details test cases, and writing test scripts by decomposing business requirements, and developing test scenarios to support quality deliverables. Involved in test planning and execution for various Test Phases:Unit Test,System and User Acceptance Testing. Expert inanalyzing, designing, developing, installing, configuringanddeployingMS SQL Server suite of products with Business Intelligence in SQL Server Reporting Services, SQL Server Analysis Services and SQL Server Integration Services. Performed data profiling, data cleansing, data conversion, exception handling and data matching usinginformatica IDQ.Excellent communication skills, good organizational skills, self-motivated, hardworking, ability to grasp quickly and learn fast and open to new technologies.
Skills & ToolsETL Tools Informatica Power Center 10.5.1 Power Exchange 9.6.1/9.0.1, Informatica Data Quality 9.6.1, Power connect for SAP BW, power connect for JMS, power connect for IBM MQ series, power connect for Mainframes, DTS, MDM, ERWIN.Scheduling, TIDAL, UC4, CONTROL-M, AUTOSYS, OpCon.Oracle, SAP HANA, MS SQL Server, Snowflake, MongoDB, Teradata, DB2, AWS.Python, Java, SQL, UNIX, Unix Shell Scripts, HTML, XML, JSON, Microsoft Office, Apache Spark, Kafka, Kubernetes, Hive, ScalaData analysis, Data management, Data warehouse, Big Data, PL/SQL, RDBMS, NoSQL, Vertica, Spark, Kafka, Oozie, Maven.Debugging, Coding, Designing, Quality analysis, ETL, SAP BW.AWS Cloud Tools: EC2, Elastic Load-balancers, Elastic Container Service (Docker Containers), S3, Elastic Beanstalk, Cloud Front, Elastic Files System, RDS, Dynamo DB, DMS, VPC, Direct Connect, Route53, Cloud Watch, Cloud Trail, Cloud Formation, IAM, EMR, ELB, Lambda functions, REST API, Airflow, Data Pipeline, RedShift.Google Cloud Platform: GCP Cloud Storage, Big Query, Composer, Cloud Dataproc, Cloud SQL, Cloud Functions, Cloud Pub/Sub, Dataflow, AI Building Blocks, Looker, Cloud Data Fusion, Dataprep, Firestone.Azure: Azure Storage, Database, Azure Data Factory, Azure Analysis Services.
Education and TrainingNew England CollegeHenniker,NHExpected in12/2017––Master of Science:Computer And Information Systems-GPA:
ExperienceLockheed Martin-Data EngineerEagan,MN,08/2019-CurrentResponsible in Identifying all the Legacy systems, analyze theirdata models, mathematical and.Scientific models and all the business components to be migrated to Arkansas Integrated Eligibility system (ARIES)NexGen solution.Business and Data Mapping Analyst worked with Client to gather business and functional requirements.Experience in building large-scale data pipelines and data-centric applications using Big Data tooling like Hadoop, Spark, Hive and Airflow in a production setting.Experience in working with Rest APIs for data extraction.Designing and coordinating withInformatica Power centeradmin to set up services, users, Groups, database components like tables, indexes, procedures and synonyms, Unix groups and User access for ARIES system.Designing all the components of ARIES system and step by step conversion approach using theRalph KimballandBill Inmomdata warehouse design methodologies to determine the feasibility of the design within the time and cost constraints.Development ofETL(Extract, Transform and Load) code components like mappings, Workflows, Sessions and database objects like stored procedures, functions andUnix shell scriptsfrom business requirements and design plan.Designing and developing the mathematical models and analytical reports inCognos, analyze.Developing integrations usingInformatica CloudData Integration(IICS–CDIService) For Google Analytics.Responsible Identifying the performance bottleneck in the ARIES systems during Integrated,Testing(IT) andSystem Testing(ST) and PT (Performance Testing) phases and implement Performance tuning techniques.Confer with the scrum master, client partners, Business Managers and adhere to theagile.Experience in multiple database technologies such as traditional RDBMS (MS SQL Server, Oracle, MySQL, PostgreSQL), MPP (AWS Redshift, Snowflake, Teradata), Distributed Processing (Spark, Hadoop, EMR), NoSQL (MongoDB, DynamoDB, Cassandra, Neo4J, Titan).Conduct code review sessions with peer developers to ensure code quality.Wrote scripts and processes fordata integrationand bug fixes.UtilizedPythonto handle debugging and automation scripting tasks.Created and implemented complex business intelligence solutions.Create & Managing buckets onS3and store DB and log backup, upload images for CND server.Setup databases onAmazon RDSorEC2 Instancesas per requirements.Hands-on experience with snowflake utilities, SnowSQL, SnowPipe, Big data model techniques using python.Expert in migrating data from various systems into Salesforce CRM using ETL tools -Informatica.Hands on Experience inData Management, Data Security, Data Modeling, Data Quality, Workflow Automation, Formulas & Validations.Collaborated with Legacy team to define data extraction methodologies and data source tracking protocols.UsedSpark-SQL to read parquet data and create tables inHiveusingScala API.Hands on experience with data ingestion tools likeSqoop, Kafka, Flume.Used variousspark transformationsandActionsfor cleansing the input data.Installed and configuredApace Airflowfor workflow management and created workflow in python.Worked on Tableau Desktop versions 7/8.2, Tableau Reader and Server.Build data pipelines in Airflow in GCP for ETL related jobs using different airflow operators.Experience in GCP Dataproc, GCS, Cloud functions, BigQuery.Wells Fargo-BI_ETL DeveloperLoveland,CO,11/2018-07/2019Analyzing the source data coming from Mainframe sources and working with business users and developers to develop the Model.Created xsd and used it inxml generator Transformation.Also calledWSDLfiles using web consumer transformation inInformatica.Effectively and efficiently communicated systems solutions to business problems to team members, business unit representatives, management, and other impacted project teams.Analyze and modify existing stored procedures, functions, and queries in order to integrate them into previously built reporting application utilizingSSISandSQL Server Management Studio.Change ETL process fromimporting flat filesandCOBOLas source for previously built application utilizing informatica Power center 9.6 to read data from existing Reporting Server (SQL Server databases).Interface with report users to determine requirements for new and existing reports.Develop, test, and deployETLjobs with reliable error/exception handling and rollback framework Manage automation of file processing as well as all ETL processes within a jobworkflow.Worked as a Data Analysts to match the current data mappings with old mainframe mappings.Created Design Documents, Context Diagrams on every Projects and did code review.Created Reusable lookup to send emails Notification to User's.This Lookup was used by multiple developer in multiple Projects.STOP and START theNetezzaappliances in case of issues.Prepared best practices documentation for teams in writingNOSQL.Conducted sessions to help explain teams about theNetezzaarchitecture and design.As a part of Informatica decommission objects Project UsedSVNto archive the ETL Code and delete the unwanted mappings and workflows.Extracted data fromflat fileand staged into a single place and applied business logic to load them in theTeradata database.Lead in few complex Projects and Performed Unit testing and createdQA documents.Participated in solution brainstorming and provided technical instruction and coaching to others within organization.Developed technical project deliverables.Created spreadsheets usingPower BIandPower Pivotby importing data from the sources directly.Created visually impactful dashboards in Excel andTableaufor data reporting usingPivotTablesandVLOOKUP.Involved in migrating ETL code lower environments to like dev/ test/Load to production environment.UsedDebuggerfor debugging Mappings.CreatedTidalJobs andRunbooksto schedule jobs in Tidal.Scheduling ETL Jobs usingUC4 scheduler.Migrated ETL code using Deployment Groups from Dev to Prod environments.Created SSIS package for loading the data coming from various interfaces like OMS, Orders, Adjustments and Objectives and also used multiple transformation in SSIS to collect data from various sources.Worked on SSIS Package, DTS Import/Export for transferring data from Database (Oracle and Text format data) to SQL Server.CreatedSSISpackages for File Transfer from one location to the other using FTP task.Manage and document our platform infrastructure. This can go from installing a new Consul server to resolving performance issues in a MongoDB cluster, through setting up a continuous integration pipeline.Cognizant Technology Solutions-Informatica DeveloperHorsham,PA,12/2017-10/2018Involved in all the phases of the development like Analysis, Design, Coding,Unit Testing, System Testing and UAT.Extracted data fromheterogenous sourcesand performed complex transformations to load data into the target systems.Resolved various performance issues by examining the logs, current design andremoving the bottlenecks.Created reusable ETL components which need to be run at the mapping, session and workflow levels.Wrote complex SQL queries and performed extensive data analysis inOracle 11g.Peer reviewed developers code and ensured they fall into the enterprise guidelines.Worked extensively with session parameters, Mapping Parameters, Mapping Variables and Parameter files for Incremental Loading.Created Informatica mappings to load Payment flow, BT cash data, Currency conversion data fromSAP HANA DSL layertoSAP HANA DPL layer/DLL (SAP BA layer),Implemented SAP BW andBOBJwith different SAP data source.UsedQlikand Informatica forETL& reporting.Worked closely withSAP ABAPteam to create logical systems, RFC destinations and to create tRFC port for RFC destinations.Followed best practices that were defined at the enterprise level and also peer reviewed the code for the same.Created and reviewed scripts to create new tables, queries for new enhancements and bug fixes in the existingdata warehouse.Used Debugger and various other techniques like tracing to fix the defects/ errors and data issues.Extensively worked with various Lookup caches likeStatic cache, Dynamic cache and Persistent cache.Involved in developing the Deployment groups for deploying the code between various environment (Dev, QA).Worked in the Data Integration Team to perform data and application integration with a goal of moving more data more effectively, efficiently and with high performance to assist in business-critical projects coming up with huge data extraction.Migrated data from SQL Server toNetezza using NZ_Migrate utility.Experience in integration of various data sources likeOracle, DB2, SQL server, .csv, XML and Flat Filesinto staging area.Developed code to extract, transform, and load (ETL) data from inbound flat files and various databases into outboundflat filesandXMLfiles using complex business logic.Involved in deploying objects from DEV to UAT/PROD during monthly/quarterly releases.Developed Slowly Changing Dimension Mappings forType 1SCDandType 2 SCDMonitored and improved query performance by creating views, indexes and sub queries Extensively involved in enhancing and managing Unix Shell Scripts.Developed workflow dependency inInformaticausing Event Wait Task, Command Wait.Involved inL3 Supportby fixing load failures and defects during peak and off-Peak hours.Working with Informatica andSAP ECC.We are extracting data in Informatica fromSAP ECCvia.Business Content Data sources.Design the ETL architecture for the conversion of source from legacy to newERP (SAP).Built Sqoop scripts to extract data from the SAP.Responsible for data integration (Epicor and SAP) and data management.Validated DW data with standardSAP reports.ObjectOne Information Systems-ETL Informatica DeveloperCity,STATE,08/2013-12/2015· Responsible for developing ETL mappings for the reporting requirement & for data feeds.· Created mapplets and many other reusable components reduce redundancy.· Provided the best solution for their requirement. Help the Clients in designing of the system and process to make it more robust and meaningful.· Created the ETL component and test scripts. Involved in client meetings for their inconsistent requirements.· Sourced data fromSAP HANAusingSAP Adapterin Informatica Target Tables.· End to end processing from the source to target. Load the data from various File systems to theDWHandDM. Maintain the Versioning for the objects by usingVSS.· Successfully conducted Load Testing and Performance Testing.· Resolve the issues coming from the end to end processing. Perform the enhancements if required by the business users. requirement & for data feeds· Involved in preparing Source to Target mappings and Application Design Document. Worked closely with all upstream and downstream application owners to make sure interface agreement documents are clear.· Successfully conducted Load Testing and Performance Testing. Automated regression test suite for actuate reporting and true portal.· UsedALMto track and report system defects and bugs, writing modification request for the bugs in the application and helped developers to track the problem and resolve the technical issues.>>. The result is, Good Fit

Fit assessment: Good Fit