Job description:
job description i am actively seeking an experienced data engineer to join an innovative saas company within the ai & adtech space. the ideal candidate is passionate about data, is highly self-motivated, enjoys collaborative work environments, and has a seasoned background in software development. requirements at least 8 years of professional engineering experience at least 5 years of hands-on data engineering experience with scalaspark at least 3 years of hands-on software development experience with java & spring for rest api development highly proficient in designing, building, deploying, etc. new etl data pipelines, databases, features and systems expert level spark knowledge experience with relational databases cloud technology experience (aws is highly preferred) b.s. or higher in computer science, or related field bonus points background working in a saas product based environment experience andor exposure to the full software development life cycle (sdlc) of a product lead andor mentorship experience this is a full time, direct hire position. the comp & benefits package includes a base, annual bonus, unlimited pto, healthcare coverage, and much more! this position is fully remote anywhere in the united states, except california. visa sponsorship is not available at this time**

Resume:
<<summarystrong it experience in aws cloud computing, bigdata & data warehousing.strong business domain knowledge in sales segmentation & territory planning.strong coding & sql experience in python and redshifthaving an experience in agile and deliver the results based on stories and task assigned on the sprint.implementation knowledge in server-less architecture using aws cloud computing.having good hands on experience in git as well as ci/cd pipeline integration.having good hands on experience in building data model based on requirements. skills programming languages: java, python, unix shell scriptingcloud technologies: amazon web services (aws)version control: aws codecommit, git, github repositoriescontinuous integration/continuous delivery(ci/cd): aws codedeploy aws codepipelinedatabases and sql: redshift, oracle, netezza, nosql (ddb & elasticsearch)big data technologies: hadoop, hive, ozzie, airflow, spark, pyspark, scoop, flume experience 01/2018tocurrentsenior data engineersplunk ,collaborated with product owners to gather requirement to help build a solution for sales segmentation.identified key use cases and associated reference architectures for market segments and industry verticals.worked as part of project teams to coordinate database and pipeline development and determine project scopes and limitations.collaborated with solution architects to define database and analytics engagement strategies for operational territories and key accounts.developed and managed enterprise-wide big data environments.specified user access levels for each database segment to protect database integrity and company information.developed and implemented security initiatives to protect important company data.established hardware requirements and devised storage capacity solutions and choose wisely on the aws services by outweigh the requirement and business use-case. 12/2015to 12/2017big data engineernike inc ,perform as a big data developer and work in various phases like data ingestion, data integration and transformation.create a sqoop import command and pull the data (both tables and all tables in db's) from mysql db to hdfs as part of data ingestioncreates a mapreduce programs as well as spark programs to parse, analyze and implement the solutions based on its customer need.perform operations to filter the records from db, boundary query, incremental update or insert in the sqoop import commands.perform operations to insert the data directly to hive tables, change the delimiters and end line character and change the formats like textfile, avrodatafile, orc file into hdfs systems using sqoop import commands.perform sqoop eval function to evaluate the data in mysql.perform sqoop export function to export the data from hdfs and loaded it into mysql and understand the delimiter of the file, define the no of mapper to perform, etc.create a flume config file to ingest the data from various source systems like (spool directory, netcat, exec, sequence, etc.) and loaded it into avro, hdfs etc.define a proper channels(memory, file ,etc.) and loaded all the data comes from source to sinks.used the interceptor to modify the data comes from source like filtering (regex), include the timestamp, search and replace, etc.create a mapreduce program using eclipse ide and execute it through jar file.create a mapper class and using map method generate the key value pair as (node_with_date, system utilization) by writing in context application.create a reducer class and using reduce method, generate the final output of system utilization per node_with_date.implements custom based writable class to get the two values in the mapoutputvalue class.implements custom based partitioner class to assign a appropriate data to the right reducer(basically try to achieve multi-level group by function with mr limitations).implements pattern matching logic for semi-structured data and routed the good and bad records separately.create a custom based input format as well as record reader to parse the xml file.hands-on in creating the broadcast variables.hands-on in sparksql, via class object in scala, convert df to rdd, temptable schema creation and so on. 09/2009to 04/2015etl developerallegis group ,created and implemented complex business intelligence solutions.created conceptual, logical and physical data models for use in different business areas.developed and managed enterprise-wide data analytics environments.identified, protected and leveraged existing data.monitored multiple databases to keep track of all company inventory.assisted maintenance team with completion of preventive maintenance and unscheduled service needs.monitored installations to ensure compliance with local codes and industry best practices. education and trainingexpected intotomaster of science information technologyamity university,noida, india,gpa:expected in 05/2008totobachelor of engineering college of engineering guindy, anna university,chennai, india,gpa:.