Job description:
novi labs delivers world-class oil & gas forecasts via an end-to-end platform that leverages machine learning and data science. join our talented technical team to help us improve and expand our products and further solidify our market-leading position. benefits of working at novi labs small, talented teamhighly collaborative work environment high autonomy with true project ownershipsmall company with visibility into company decisions and prioritieseasy line-of-sight to how your contributions directly benefit customers role we are looking for a seasoned data engineer that has a passion for sound design and engineering practices and is adept at making reasoned short-term vs long-term trade-offs. in this role you will be leading the buildout of our next generation data and machine learning platform, while helping to support and transition from our current systems. this is a key role on a small team. to be successful, it will be important that you feel comfortable driving projects on your own, but also enjoy brainstorming and collaboration. what youll do here after joining novi, you'll collaborate hand in hand with our team of data scientists and etl specialists. together, we're charting a course towards establishing an innovative greenfield platform that will unlock the power of machine-learning powered analytics products. as a key member of our team, you'll play a pivotal role in cultivating a culture of engineering excellence. your responsibilities will encompass the establishment of standards and the integration of emerging technologies to uphold the zenith of software engineering practices in python. your expertise will contribute to the improvement of our existing analytics platform, encompassing the implementation of novel functionalities and the enhancement of both the scalability and reliability of our data transformation and storage. this will be achieved through precise profiling and optimization techniques, coupled with the strategic design of cloud computing solutions. if you're driven by the prospect of making a profound impact within a collaborative and forward-thinking environment, we invite you to join us at novi. together, we'll continue to elevate the benchmark of data-driven solutions while nurturing your professional growth and development. required skills & experience minimum of 5 years of experience building robust and scalable data pipelinesstrong foundation in software engineering principles in pythonknowledge of the pydata stack (e.g. pandas) that goes deeper than the apisexperience handling, storing and linking multi-modal data typesexperience with the full software development lifecycle (git, ci, docker, test, etc.)strong database knowledge (sql & nosql)strong cloud architecture skills - aws preferred desired skills & experience interest in data science or previous experience working on a data science teamexperience working with gis dataprevious work on an agile teamexperience working for a start-up or small company cultural fit at novi, the culture fit is as important as the technical fit. our team will be a great fit for you if you: have a positive attitude and a growth mindsetenjoy working on small, highly collaborative teamsare self-motivated to improve your technical capacitiesinfluence without authority via well-reasoned ideas and clear communicationare comfortable working cross-functionally to deliver customer impactenjoy team and company successes as much as your owntake ownership of deliverables with strong attention to detailare happy to task switch when necessary to help teams succeedenjoy knowing whats happening across the entire company more about novi professional work environment with a clear understanding of worklife balanceflexible leave policyheathdentalvision insurance401k

Resume:
<<summarydata engineeringwith experience in design, development, implementation and support ofdata warehousingfor over8 years. experience d in completesoftware development life cycle (sdlc),software testing life cycle (stlc),sdlc methodologies. extensively worked oninformatica designer tools(source analyze, warehouse designer, mapping designer, mapplet designer, and transformation developer) and workflow manager tools (task developer, worklet and workflow designer), workflow monitor and informatica power exchange. experience in working with using informatica insap hana, sql,teradataanddb2 environments. hands on experience in migrating on premise etl togoogle cloud platform(gcp)using cloud native tools such asbig query,cloud data proc,google cloud storage,composer.practical understanding of the data modeling concepts likestar - schema modeling, snowflake schema modeling, fact and dimension tables. also experience in optimizing database querying, data manipulation using sql andpl/sqlin oracle, flat files and sql server database. experience d in implementingchange data capture (cdc)using informatica power center for oracle and sap systems. experience d in debugging mapping by analyzing the data flow and evaluating transformations. experience d in performance tuning of data flow through source, target, sessions and mappings,identifying and resolving performance bottlenecks at various stagesusing techniques like database tuning and session partitioning. experience d in test strategy, developing test plan, details test cases, and writing test scripts by decomposing business requirements, and developing test scenarios to support quality deliverables. involved in test planning and execution for various test phases:unit test,system and user acceptance testing. expert inanalyzing, designing, developing, installing, configuringanddeployingms sql server suite of products with business intelligence in sql server reporting services, sql server analysis services and sql server integration services. performed data profiling, data cleansing, data conversion, exception handling and data matching usinginformatica idq.excellent communication skills, good organizational skills, self-motivated, hardworking, ability to grasp quickly and learn fast and open to new technologies. skills & toolsetl tools informatica power center 10.5.1 power exchange 9.6.1/9.0.1, informatica data quality 9.6.1, power connect for sap bw, power connect for jms, power connect for ibm mq series, power connect for mainframes, dts, mdm, erwin.scheduling, tidal, uc4, control-m, autosys, opcon.oracle, sql server, snowflake, mongodb, teradata, db2, aws.python, java, sql, unix shell scripts, html, xml, json, microsoft office, apache spark, kafka, kubernetes, hive, scaladata analysis, data management, data warehouse, /sql, rdbms, nosql, vertica, spark, kafka, oozie, maven.debugging, coding, designing, quality analysis, etl, sap bw.aws cloud tools: ec2, elastic load-balancers, elastic container service (docker containers), s3, elastic beanstalk, cloud front, elastic files system, rds, dynamo db, dms, vpc, direct connect, route53, cloud watch, cloud trail, cloud formation, iam, emr, elb, lambda functions, rest api, airflow, data pipeline, redshift.google cloud platform: gcp cloud storage, big query, composer, cloud dataproc, cloud sql, cloud functions, cloud pub/sub, building blocks, looker, cloud data fusion, dataprep, firestone.azure: azure storage, database, azure data factory, azure analysis services. education and trainingnew england collegehenniker,nhexpected in 12/2017––master of science:computer and information systems-gpa: experience lockheed martin- , 08/2019-currentresponsible in identifying all the legacy systems, analyze theirdata models, mathematical and.scientific models and all the business components to be migrated to arkansas integrated eligibility system (aries)nexgen solution.business and data mapping analyst worked with client to gather business and functional requirements.experience in building large-scale data pipelines and data-centric applications using big data tooling like hadoop, spark, hive and airflow in a production setting.experience in working with rest apis for data extraction.designing and coordinating withinformatica power centeradmin to set up services, users, groups, database components like tables, indexes, procedures and synonyms, unix groups and user access for aries system.designing all the components of aries system and step by step conversion approach using theralph kimballandbill inmomdata warehouse design methodologies to determine the feasibility of the design within the time and cost constraints.development ofetl(extract, transform and load) code components like mappings, workflows, sessions and database objects like stored procedures, functions andunix shell scriptsfrom business requirements and design plan.designing and developing the mathematical models and analytical reports incognos, analyze.developing integrations usinginformatica clouddata integration(iics–cdiservice) for google analytics.responsible identifying the performance bottleneck in the aries systems during integrated,testing(it) andsystem testing(st) and pt (performance testing) phases and implement performance tuning techniques.confer with the scrum master, client partners, business managers and adhere to theagile.experience in multiple database technologies such as traditional rdbms (ms sql server, oracle, mysql, postgresql), mpp (aws redshift, snowflake, teradata), distributed processing (spark, hadoop, emr), nosql (mongodb, dynamodb, cassandra, neo4j, titan).conduct code review sessions with peer developers to ensure code quality.wrote scripts and processes fordata integrationand bug fixes.utilizedpythonto handle debugging and automation scripting tasks.created and implemented complex business intelligence solutions.create & managing buckets ons3and store db and log backup, upload images for cnd server.setup databases onamazon rdsorec2 instancesas per requirements.hands-on experience with snowflake utilities, snowsql, snowpipe, big data model techniques using python.expert in migrating data from various systems into salesforce crm using etl tools -informatica.hands on experience indata management, data security, data modeling, data quality, workflow automation, formulas & validations.collaborated with legacy team to define data extraction methodologies and data source tracking protocols.usedspark-sql to read parquet data and create tables inhiveusingscala api.hands on experience with data ingestion tools likesqoop, kafka, flume.used variousspark transformationsandactionsfor cleansing the input data.installed and configuredapace airflowfor workflow management and created workflow in python.worked on tableau desktop versions 7/8.2, tableau reader and server.build data pipelines in airflow in gcp for etl related jobs using different airflow operators.experience in gcp dataproc, gcs, cloud functions, bigquery.wells fargo-bi_etl , 11/2018- 07/2019analyzing the source data coming from mainframe sources and working with business users and developers to develop the model.created xsd and used it inxml generator transformation.also calledwsdlfiles using web consumer transformation ininformatica.effectively and efficiently communicated systems solutions to business problems to team members, business unit representatives, management, and other impacted project teams.analyze and modify existing stored procedures, functions, and queries in order to integrate them into previously built reporting application utilizingssisandsql server management studio.change etl process fromimporting flat filesandcobolas source for previously built application utilizing informatica power center 9.6 to read data from existing reporting server (sql server databases).interface with report users to determine requirements for new and existing reports.develop, test, and deployetljobs with reliable error/exception handling and rollback framework manage automation of file processing as well as all etl processes within a jobworkflow.worked as a data analysts to match the current data mappings with old mainframe mappings.created design documents, context diagrams on every projects and did code review.created reusable lookup to send emails notification to user's.this lookup was used by multiple developer in multiple projects.stop and start thenetezzaappliances in case of issues.prepared best practices documentation for teams in writingnosql.conducted sessions to help explain teams about thenetezzaarchitecture and design.as a part of informatica decommission objects project usedsvnto archive the etl code and delete the unwanted mappings and workflows.extracted data fromflat fileand staged into a single place and applied business logic to load them in theteradata database.lead in few complex projects and performed unit testing and createdqa documents.participated in solution brainstorming and provided technical instruction and coaching to others within organization.developed technical project deliverables.created spreadsheets usingpower biandpower pivotby importing data from the sources directly.created visually impactful dashboards in excel andtableaufor data reporting usingpivottablesandvlookup.involved in migrating etl code lower environments to like dev/ test/load to production environment.useddebuggerfor debugging mappings.createdtidaljobs andrunbooksto schedule jobs in tidal.scheduling etl jobs usinguc4 scheduler.migrated etl code using deployment groups from dev to prod environments.created ssis package for loading the data coming from various interfaces like oms, orders, adjustments and objectives and also used multiple transformation in ssis to collect data from various sources.worked on ssis package, dts import/export for transferring data from database (oracle and text format data) to sql server.createdssispackages for file transfer from one location to the other using ftp task.manage and document our platform infrastructure. this can go from installing a new consul server to resolving performance issues in a mongodb cluster, through setting up a continuous integration pipeline.cognizant technology solutions- , 12/2017- 10/2018involved in all the phases of the development like analysis, design, coding,unit testing, system testing and uat.extracted data fromheterogenous sourcesand performed complex transformations to load data into the target systems.resolved various performance issues by examining the logs, current design andremoving the bottlenecks.created reusable etl components which need to be run at the mapping, session and workflow levels.wrote complex sql queries and performed extensive data analysis inoracle 11g.peer reviewed developers code and ensured they fall into the enterprise guidelines.worked extensively with session parameters, mapping parameters, mapping variables and parameter files for incremental loading. cash data, currency conversion data fromsap hana dsl layertosap hana dpl layer/dll (sap ba layer),implemented sap bw andbobjwith different sap data source.usedqlikand informatica foretl& reporting.worked closely withsap abapteam to create logical systems, rfc destinations and to create trfc port for rfc destinations.followed best practices that were defined at the enterprise level and also peer reviewed the code for the same.created and reviewed scripts to create new tables, queries for new enhancements and bug fixes in the existingdata warehouse.used debugger and various other techniques like tracing to fix the defects/ errors and data issues.extensively worked with various lookup caches likestatic cache, dynamic cache and persistent cache.involved in developing the deployment groups for deploying the code between various environment ( ).worked in the data integration team to perform data and application integration with a goal of moving more data more effectively, efficiently and with high performance to assist in business-critical projects coming up with huge data extraction.migrated data from sql server tonetezza using nz_migrate utility.experience in integration of various data sources likeoracle, db2, sql server, .csv, xml and flat filesinto staging area.developed code to extract, transform, and load (etl) data from inbound flat files and various databases into outboundflat filesandxmlfiles using complex business logic.involved in deploying objects from dev to uat/prod during monthly/quarterly releases.developed slowly changing dimension mappings fortype 1scdandtype 2 scdmonitored and improved query performance by creating views, indexes and sub queries extensively involved in enhancing and managing unix shell scripts.developed workflow dependency ininformaticausing event wait task, command wait.involved inl3 supportby fixing load failures and defects during peak and off-peak hours.working with informatica andsap ecc.we are extracting data in informatica fromsap eccvia.business content data sources.design the etl architecture for the conversion of source from legacy to newerp (sap).built sqoop scripts to extract data from the sap.responsible for data integration (epicor and sap) and data management.validated dw data with standardsap reports.objectone information systems-etl informatica developercity,state, 08/2013- 12/2015· responsible for developing etl mappings for the reporting requirement & for data feeds.· created mapplets and many other reusable components reduce redundancy.· provided the best solution for their requirement. help the clients in designing of the system and process to make it more robust and meaningful.· created the etl component and test scripts. involved in client meetings for their inconsistent requirements.· sourced data fromsap hanausingsap adapterin informatica target tables.· end to end processing from the source to target. load the data from various file systems to thedwhanddm. maintain the versioning for the objects by usingvss.· successfully conducted load testing and performance testing.· resolve the issues coming from the end to end processing. perform the enhancements if required by the business users. requirement & for data feeds· involved in preparing source to target mappings and application design document. worked closely with all upstream and downstream application owners to make sure interface agreement documents are clear.· successfully conducted load testing and performance testing. automated regression test suite for actuate reporting and true portal.· usedalmto track and report system defects and bugs, writing modification request for the bugs in the application and helped developers to track the problem and resolve the technical issues.