Job description:
novi labs delivers world-class oil & gas forecasts via an end-to-end platform that leverages machine learning and data science. join our talented technical team to help us improve and expand our products and further solidify our market-leading position. benefits of working at novi labs small, talented teamhighly collaborative work environment high autonomy with true project ownershipsmall company with visibility into company decisions and prioritieseasy line-of-sight to how your contributions directly benefit customers role we are looking for a seasoned data engineer that has a passion for sound design and engineering practices and is adept at making reasoned short-term vs long-term trade-offs. in this role you will be leading the buildout of our next generation data and machine learning platform, while helping to support and transition from our current systems. this is a key role on a small team. to be successful, it will be important that you feel comfortable driving projects on your own, but also enjoy brainstorming and collaboration. what youll do here after joining novi, you'll collaborate hand in hand with our team of data scientists and etl specialists. together, we're charting a course towards establishing an innovative greenfield platform that will unlock the power of machine-learning powered analytics products. as a key member of our team, you'll play a pivotal role in cultivating a culture of engineering excellence. your responsibilities will encompass the establishment of standards and the integration of emerging technologies to uphold the zenith of software engineering practices in python. your expertise will contribute to the improvement of our existing analytics platform, encompassing the implementation of novel functionalities and the enhancement of both the scalability and reliability of our data transformation and storage. this will be achieved through precise profiling and optimization techniques, coupled with the strategic design of cloud computing solutions. if you're driven by the prospect of making a profound impact within a collaborative and forward-thinking environment, we invite you to join us at novi. together, we'll continue to elevate the benchmark of data-driven solutions while nurturing your professional growth and development. required skills & experience minimum of 5 years of experience building robust and scalable data pipelinesstrong foundation in software engineering principles in pythonknowledge of the pydata stack (e.g. pandas) that goes deeper than the apisexperience handling, storing and linking multi-modal data typesexperience with the full software development lifecycle (git, ci, docker, test, etc.)strong database knowledge (sql & nosql)strong cloud architecture skills - aws preferred desired skills & experience interest in data science or previous experience working on a data science teamexperience working with gis dataprevious work on an agile teamexperience working for a start-up or small company cultural fit at novi, the culture fit is as important as the technical fit. our team will be a great fit for you if you: have a positive attitude and a growth mindsetenjoy working on small, highly collaborative teamsare self-motivated to improve your technical capacitiesinfluence without authority via well-reasoned ideas and clear communicationare comfortable working cross-functionally to deliver customer impactenjoy team and company successes as much as your owntake ownership of deliverables with strong attention to detailare happy to task switch when necessary to help teams succeedenjoy knowing whats happening across the entire company more about novi professional work environment with a clear understanding of worklife balanceflexible leave policyheathdentalvision insurance401k

Resume:
<<professional summaryenergetic data engineer in developing robust code for high-volume businesses. strong decision-maker with 6 years of experience in data engineering and help firms in designing and executing solutions for complex business problems involving large scale data warehousing, real-time analytics and reporting solutions. ability to translate business questions and concerns into specific quantitative questions that can be answered with available data using sound methodologies. skills python 3.x, r, sqlhadoop, apache sparkhive, pig, kafka, sqoop, oozieteradata, snowflakeamazon s3,emr,lambdagit, jenkins, splunkms officemicrosoft visual c#.net education the university of texas at dallasrichardson,txexpected in4 2020––master of science:business analytics-gpa: work history avanade- , 04/2020-currentcombining data from multiple source systems (profile, systematics, etc.) and multiple platforms (snowflake, onelake, hubs) and computing canonical gold-star metrics “once and for all” to cut operational costs.develop spark jobs to transform data and apply business transformation rules to load/process data across enterprise and application specific layers.experience in building/operating/maintaining fault tolerant and scalable data processing integrations using aws.configured s3 buckets with various life cycle policies to archive the infrequently accessed data based on requirement.good working experience on submitting the spark jobs which shows the metrics of the data which is used for data quality checking.working on building efficient data pipelines that transform high volume data into a format used for analytical, fraud prevention and ml use cases.extensively used splunk search processing language (spl) queries, reports, alerts and dashboards.excellent knowledge of source control management concepts such as branching, merging, /tagging and integration, with tool like git.performing data quality checks like row count, schema validation, hash key validation for all data movement between applications.avanade- , 08/2019- 04/2020responsible for designing and developing various analytical solutions for gaining analytical insights into large data sets by ingesting and transforming these datasets in the big data environment using technologies like spark, sqoop, oozie, hive.scheduling jobs to automate the process for regular executing jobs worked on using oozie.developed oozie workflow schedulers to run multiple hive and pig jobs that run independently with time and data availability.familiar with data architecture including data ingestion pipeline design, hadoop information architecture, data modeling and data mining,machine learning and advanced data processing.all the projects which i have worked for are open source projects and has been tracked using jira.parallel copying of files between various clusters using distcp, kafka in hadoop.ascend learning- , 09/2018- 12/2018acquire, clean, integrate, analyze and interpret disparate datasets using a variety of statistical data analysis and data visualization methodologies, reporting and authoring findings where appropriate.developed linear mixed effects models for boston ed-fi dataset to estimate the teacher contribution to the student test scores.random intercept model which utilized a pre-post-test design and included fixed effects for student demographics and a growth model where students were nested by time (random slope and intercept) and teacher was tested.generated various clustering models for entire west virginia department of education and evaluated cluster’s performance.verizon-big data engineer. 01/2014- 07/2017identify customer's digital analytical needs and engage with customer and principal architects daily, understand the business requirements for big data analytical solutions and break down large scale requirements into detailed system specifications.moving data to hdfs framework using sqoop from teradata, sql server.good working experience on hadoop tools related to data warehousing like hive, pig and also involved in extracting the data from these tools on to the cluster using sqoop.solved performance issues in hive with understanding of joins, groups, bucketing, partitions and working on them using hiveql.deployed programs written in pyspark to run spark mllib, for analytics and reduced customers churn rate by 25%. skills python 3.x, r, sqlhadoop, apache sparkhive, pig, kafka, sqoop, oozieteradata, snowflakeamazon s3,emr,lambdagit, jenkins, splunkms officemicrosoft visual c#.net.