Job description:
who we are invitation homes is a fast-paced evolving publicly traded reit that is pioneering a new industry with over 80,000 single family rental homes and a market capitalization of over $25 billion. we are a team of more than 1,400 associates who come from all walks of life. we call different communities home, but our shared values bind us together. invitation homes is a place where possibility lives. thr property management lp data engineer dallas, tx: develop and maintain data pipelines into data warehouse and data lake, including design, and development. develop and implement strategies to translate business requirements into feasible and acceptable data warehouse and data lake solutions. architect and build new data models to improve data models that feed business intelligence tools, increasing data accessibility, and fostering data-driven decision making. design, build and launch data pipelines to move data to data lake and data warehouse build and maintain framework for auditing, error logging, and master data management for data pipelines. build data expertise and own data quality for the data pipelines. implement processes and systems to monitor data quality, ensuring production data is always accurate and available for key stakeholders and business processes. perform data analysis and assist in the resolution of data issues. identify and resolve defects of complex scope using proper engineering tools and techniques. provide support and maintain existing products and add new features. mentor engineers by providing advice, coaching and educational opportunities. telecommuting permitted from any location within the u.s. salary: 148512year. the senior data engineer is accountable for the maintenance, improvement, and movement of data in the enterprise data warehouse, and other data sources used for enterprise reporting. a successful senior data engineer delivers database design, implementation, and schematics that creates sustainable, competitive advantage for our company. further, a senior data engineer is responsible for the development and deployment of innovative analytics data platforms to support end-user enterprise reporting. working with other data engineers, data analysts, and data scientists, a senior data engineer must have a delivery first mentality and a firm grasp of agile development methodology. lastly, the ideal candidate must demonstrate the ability to work as a team member and team leader. what youll doaccountable for the overall performance and maintenance of our sql server environments:manage the edw relational database for optimized performancedesign scalable etl packages from the business source systemsensure all necessary testing and validation for all planned and unplanned releasesmentor junior data engineers in best practices and documentation.act as our database evangelist in leading innovation activities through exploration, benchmarking and implementation of data technologiesparticipate in solution-based activities; articulate your ideas through well thought out documentation and research best-in-class solutionslead and develop and implement data architecture standardsmanage the necessary testing and validation to ensure proper compliance to data governance and qualityprepare progress reports regarding our database environments status and healthmanage troubleshooting data issues and present solutions to these issues.resolve tactical issues thoroughly while always looking for optimization and documenting lessons learnedunderstand when tactical issues are really systematic issues requiring deeper level of analysis and strategic solutionscooperate with the support team to investigate and resolve data related defectsproactively analyze and evaluate current state of data in order to identify and recommend improvements and optimizationsupport our agile development methodologyconstantly prioritize user stories; balance feature development, defects, and tech debt according to business needbe a champion of continuous improvement and delivery; define the minimum viable product, focus teams on key release milestones, and leverage fast follow-up sprintsenable sprint progress; clarify story details, validate completed stories, and prepare stories for the next sprint who were looking for bachelors degree in computer science, applied mathematics, engineering, or any other technology related field. an equivalent of the same in working experience is also accepted for the position.a candidate for the position will have as at least 5 years of working experience as a database engineering or a database engineering administrator within a fast-paced a complex business setting.design, implement and maintain sql server databasesdesign, implement and maintain etl processes using sql server ssisprovide ongoing maintenance support through sql query tuning and optimizationstrong working and conceptual knowledge of building and maintaining physical and logical data modelsstrong working and conceptual knowledge of reporting and visualization tools such as ssrs, powerbi, tableau, or other business intelligence tools.experience working in saas, iaas, and paasexcellent customer service and interpersonal skills; ability to relate to and get along with othersprofessional verbal and written communication skillsstrong organizational and time-management skillsability to multi-task and maintain flexibility and creativity in a variety of situationsability to analyze and resolve problemsability to set and meet goals and consistently meet deadlinesability to maintain confidentiality why invitation homes invitation homes offers the below to each new associate we stand for flexibility, opportunity, and a home that people can make their own. its as true for our associates as it is for our residents. just like we help our residents live freer, we liberate our associates careers, too. our associates know that at invitation homes goals matter, potential is unlocked, and careers thrive. invitation homes isnt just a work place it is a possibility place. competitive pay and an annual bonus program for all associatesgenerous paid time off plans including vacation accrual, sick time, volunteer time, and standard and floating holidays401k with matching company contributionsawesome work environment with casual dressteam events and gatheringsemployee resource groups: together with women, asian alliance, black collective, juntos, gen next, and open invitation. invitation homes truly is where possibility lives, pour a new foundation here! salary range $98,640.00 - $170,976.00 compensation and benefits to attract and retain top talent, we're pleased to offer competitive compensation and benefits, including: annual bonus programhealth, dental, vision, and life insurancelong-term and short-term disability insurancegenerous paid time off plans include vacation accrual, sick time, standard holidays and floating holidays401(k) with company matching contributionsawesome work environment with casual dressteam events and gatherings (pre- and post-covid) invitation homes is an equal opportunity employer committed to fostering a diverse, inclusive and innovative environment with the best associates. we are committed to . if you have a disability or special need that requires accommodation, please contact us at . to all recruitment agencies: invitation homes does not accept agency resumes. please do not forward resumes to invitation homes employees. invitation homes is not responsible for any fees related to unsolicited resumes.

Resume:
<<summarydynamic and motivated it professional with around 7 years of experience as a big data engineer with expertise in designing data intensive applications using hadoop ecosystem , big data analytical , cloud data engineering , data warehouse / data mart, data visualization , reporting , and data quality solutions .in - depth knowledge of hadoop architecture and its components like yarn , hdfs, name node, data node, job tracker, application master, resource manager , task tracker and map reduce programming paradigm.extensive experience in hadoop led development of enterprise level solutions utilizing hadoop components such as apache spark, mapreduce, hdfs, sqoop, pig, hive, hbase, oozie, flume, nifi, kafka, zookeeper, and yarn.profound experience in performing data ingestion, data processing (transformations, enrichment, and aggregations). -depth understanding of mapreduce programming paradigm and spark execution framework.experience d with the spark improving the performance and optimization of the existing algorithms in hadoop using spark context , spark-sql , dataframe api , spark streaming, mllib , pair rdd 's and worked explicitly on pyspark and scala .handled ingestion of data from different data sources into hdfs using sqoop, flume and perform transformations using hive, map reduce and then loading data into hdfs.managed sqoop jobs with incremental load to populate hive external tables. experience in importing streaming data into hdfs using flume sources, and flume sinks and transforming the data using flume interceptors.experience in oozie and workflow scheduler to manage hadoop jobs by direct acyclic graph ( dag ) of actions with control flows.implemented the security requirements for hadoop and integrating with kerberos authentication infrastructure- kdc server setup, creating realm /domain, managing.experience of partitions, bucketing concepts in hive and designed both managed and external tables in hive to optimize performance .experience with different file formats like avro , parquet , orc , json and xml .expertise in creating, debugging, scheduling and monitoring jobs using airflow and oozie.experience d with using most common operators in airflow - python operator, bash operator, google cloud storage download operator, google cloud storage object sensor.hands-on experience in handling database issues and connections with sql and nosql databases such as mongodb , hbase , cassandra , sql server , and postgresql .created java apps to handle data in mongodb and hbase. used phoenix to create sql layer on hbase.experience in designing and creating rdbms tables, views, user created data types, indexes, stored procedures, cursors, triggers and transactions.expert in designing etl data flows using creating mappings/workflows to extract data from sql server and data migration and transformation from oracle/access/excel sheets using sql server ssis .expert in designing parallel jobs using various stages like join, merge, lookup, remove duplicates, filter, dataset, lookup file set, complex flat file, modify, aggregator, xml.hands-on experience with amazon ec2, amazon s3, amazon rds, vpc, iam, amazon elastic load balancing, auto scaling, cloudwatch, sns, ses, sqs, lambda, emr and other services of the aws family.created and configured new batch job in denodo scheduler with email notification capabilities and implemented cluster setting for multiple denodo node and created load balance for improving performance activity.instantiated, created, and maintained ci/cd (continuous integration & deployment) pipelines and apply automation to environments and applications.worked on various automation tools like git, terraform, ansible. experience d in fact dimensional modeling ( star schema, snowflake schema ), transactional modeling and scd (slowly changing dimension)experience d with json based restful web services, and xml/qml based soap web services and also worked on various applications using python integrated ides like sublime text and pycharm .efficient cloud engineer with years of experience assembling cloud infrastructure. utilizes strong managerial skills by negotiating with vendors and coordinating tasks with other it team members. implements best practices to create cloud functions, applications and databases. skills·big data technologies:hadoop, mapreduce, hdfs, sqoop, pig, hive, hbase, oozie, flume, nifi, kafka, zookeeper, yarn, apache spark, mahout, sparklib·databases:oracle, mysql, sql server, mongodb, cassandra, dynamodb, postgresql, teradata, cosmos.·programming:python, pyspark, scala, java, c++, shell script, perl script, sql·cloud technologies:aws, microsoft azure·frameworks:django rest framework, mvc, hortonworks·tools:pycharm, eclipse, visual studio, sql plus, sql developer, toad, sql navigator, query analyzer, sql server management studio, sql assistance, eclipse, postman·versioning tools:svn, git, github·operating systems:windows 7/8/xp/2008/2012, ubuntu linux, macos·network security:kerberos·database modelling: modeling, star schema modeling, snowflake modeling·monitoring tool:apache airflow·visualization/ reporting:tableau, ggplot2, matplotlib, ssrs and power bi·machine learning techniques:linear & logistic regression, classification and regression trees, random forest, associative rules, nlp and clustering. education and trainingpurdue universitywest lafayette,inexpected in 02/2022––post graduate:data engineering-gpa:university of texas at austinaustin,txexpected in 09/2021––post graduate:data science and business analytics-gpa: ,expected in 12/2009––bachelor of arts:business administration and management-gpa: experience deloitte- , 01/2022- 02/2022designed and setup enterprise data lake to provide support for various uses cases including analytics, processing, storing and reporting of voluminous, rapidly changing datadesigned and developed security framework to provide fine grained access to objects in aws s3 using aws lambda, dynamodbset up and worked on kerberos authentication principals to establish secure network communication on cluster and testing of hdfs, hive, pig and mapreduce to access cluster for new usersused aws emr to transform and move large amounts of data into and out of other aws data stores and databases, such as amazon simple storage service (amazon s3) and amazon dynamodbimport the data from different sources like hdfs/hbase into spark rdd and perform computations using pyspark to generate the output responsecreating lambda functions with boto3 to deregister unused amis in all application regions to reduce the cost for ec2 resourcesimporting & exporting database using sql server integrations services (ssis) and data transformation services (dts packages)coded teradata bteq scripts to load, transform data, fix defects like scd 2 date chaining, cleaning up duplicatesconducted data blending, data preparation using alteryx and sql for tableau consumption and publishing data sources to tableau serverimplemented aws step functions to automate and orchestrate the amazon sagemaker related tasks such as publishing data to s3, training ml model and deploying it for predictionintegrated apache airflow with aws to monitor multi-stage ml workflows with the tasks running on amazon sagemakerenvironment: aws emr, s3, rds, redshift, lambda, boto3, dynamodb, amazon sagemaker, apache spark, hbase, apache kafka, hive, sqoop, map reduce, snowflake, apache pig, python, ssrs, tableauassessed organization technology infrastructure and managed cloud migration process.bank of america corporation- , 01/2016- 11/2019worked on azure data factory to integrate data of both on-prem (my sql, cassandra) and cloud (blob storage, azure sql db) and applied transformations to load back to azure synapsemanaged, configured and scheduled resources across the cluster using azure kubernetes serviceinvolved in developing data ingestion pipelines on azure hdinsight spark cluster using azure data factory and spark sqldevelop dashboards and visualizations to help business users analyze data as well as providing data insight to upper management with a focus on microsoft products like sql server reporting services (ssrs) and power biperformed the migration of large data sets to databricks (spark), create and administer cluster, load data, configure data pipelines, loading data from adls gen2 to databricks using adf pipelinescreated various pipelines to load the data from azure data lake into staging sqldb and followed by to azure sql dbworked extensively on azure data factory including data transformations, integration runtimes, azure key vaults, triggers and migrating data factory pipelines to higher environments using arm templatesingested data in mini-batches and performs rdd transformations on those mini-batches of data by using spark streaming to perform streaming analytics in data bricksenvironment: azure sql dw, databrick, azure synapse, cosmos db, adf, ssrs, power bi, azure data lake, arm, azure hdinsight, blob storage, apache spark.cumming llc-big data engineer / , 10/2013- 12/2015ansibledenododenodocloudwatchavropysparkpysparkpysparkmllibdataframenifinifiinteracted with business partners, business analysts and product owner to understand requirements and build scalable distributed data solutions using hadoop ecosystemdeveloped spark streaming programs to process near real time data from kafka, and process data with both stateless and state full transformationsworked with hive data warehouse infrastructure-creating tables, data distribution by implementing partitioning and bucketing, writing and optimizing the hql queriesbuilt and implemented automated procedures to split large files into smaller batches of data to facilitate ftp transfer which reduced 60% of execution timeworked on developing etl processes (data stage open studio) to load data from multiple data sources to hdfs using flume and sqoop, and performed structural modifications using map reduce, hived eveloping spark scripts, udfs using both spark dsl and spark sql query for data aggregation, querying, and writing data back into rdbms through sqoopwritten multiple mapreduce jobs using java api, pig and hive for data extraction, transformation and aggregationavrom multiple file formats including parquet, avro, xml, json, csv, orcfile and other compressed file formats codecs like gzip, snappy, lzostrong understanding of partitioning, bucketing concepts in hive and designed both managed and external tables in hive to optimize performancedeveloped pig udfs for manipulating the data according to business requirements and also worked on developing custom pig loadersdeveloping etl pipelines in and out of data warehouse using combination of python and snowflakes snowsql writing sql queries against snowflakeexperience in report writing using sql server reporting services (ssrs) and creating various types of reports like drill down, parameterized, cascading, conditional, table, matrix, chart and sub reportsused datastax spark connector which is used to store the data into cassandra database or get the data from cassandra databasewrote oozie scripts and setting up workflow using apache oozie workflow engine for managing and scheduling hadoop jobsworked on implementation of a log producer in scala that watches for application logs, transform incremental log and sends them to a kafka and zookeeper based log collection platformused hive to analyze data ingested into hbase by using hive-hbase integration and compute various metrics for reporting on the dashboardtransformed tpyspark using aws glue dynamic frames with pyspark; cataloged the transformed the data using crawlers and scheduled the job and crawler using workflow featureworked on installing cluster, commissioning & decommissioning of data node, name node recovery, capacity planning, and slots configurationdeveloped data pipeline programs with spark scala apis, data aggregations with hive, and formatting data (json) for visualization, and generatipysparkenvironment: aws, cassandra, pyspark, apache spark, hbase, apache kafka, hive, sqoop, flume, apache oozie, zookeeper, etl, udf, map reduce, snowflake, apache pig, python, java, ssrsonfidentialdeveloped and implemented hadoop code while observing coding standards.optimized and tuned hadoop environments and modified hardware to meet prescribed performance thresholds.developed new functions and applications to conduct analyses.created graphs and charts detailing data analysis results.tested, validated and reformulated models to foster accurate prediction of outcomes.fiserv-python developercity,state, 09/2012- 10/2013aws, s3, ec2, lambda, ebs, iam, datadog, cloudtrail, cli, ansible, mysql, python, git, jenkins, dynamodb, cloud watch, docker, kubernetesleveraged open communication, collective decision-making and thorough reviews to create performant and scalable systems.worked with server-side and front-end technologies and leveraged common design patterns to code dynamic and user-friendly systems.implemented new api routes, architected new orm structures and refactored code to boost application performance.harnessed version control tools to coordinate project development and individual code submissions.introduced cloud-based technologies into python development to expand on-premise deployment options.wrote clear and clean code for use in projects.resolved customer issues by establishing workarounds and solutions to debug and create defect fixes.