Category: Python Developer

summary skills jessica claire 100 montgomery st 10th floor555 4321000 resumesampleexamplecom dynamic motivated professional around 7 years experience big data engineer expertise designing data intensive applications using hadoop ecosystem big data analytical cloud data engineering data warehouse data mart data visualization reporting data quality solutions indepth knowledge hadoop architecture components like yarn hdfs name node data node job tracker application master resource manager task tracker map reduce programming paradigm extensive experience hadoop led development enterprise level solutions utilizing hadoop components apache spark mapreduce hdfs sqoop pig hive hbase oozie flume nifi kafka zookeeper yarn profound experience performing data ingestion data processing transformations enrichment aggregations strong knowledge architecture distributed systems parallel processing indepth understanding mapreduce programming paradigm spark execution framework experienced spark improving performance optimization existing algorithms hadoop using spark context sparksql dataframe api spark streaming mllib pair rdd worked explicitly pyspark scala handled ingestion data different data sources hdfs using sqoop flume perform transformations using hive map reduce loading data hdfs managed sqoop jobs incremental load populate hive external tables experience importing streaming data hdfs using flume sources flume sinks transforming data using flume interceptors experience oozie workflow scheduler manage hadoop jobs direct acyclic graph dag actions control flows implemented security requirements hadoop integrating kerberos authentication infrastructure kdc server setup creating realm domain managing experience partitions bucketing concepts hive designed managed external tables hive optimize performance experience different file formats like avro parquet orc json xml expertise creating debugging scheduling monitoring jobs using airflow oozie experienced using common operators airflow python operator bash operator google cloud storage download operator google cloud storage object sensor handson experience handling database issues connections sql nosql databases mongodb hbase cassandra sql server postgresql created java apps handle data mongodb hbase used phoenix create sql layer hbase experience designing creating rdbms tables views user created data types indexes stored procedures cursors triggers transactions expert designing etl data flows using creating mappingsworkflows extract data sql server data migration transformation oracleaccessexcel sheets using sql server ssis expert designing parallel jobs using various stages like join merge lookup remove duplicates filter dataset lookup file set complex flat file modify aggregator xml handson experience amazon ec2 amazon s3 amazon rds vpc iam amazon elastic load balancing auto scaling cloudwatch sns ses sqs lambda emr services aws family created configured new batch job denodo scheduler email notification capabilities implemented cluster setting multiple denodo node created load balance improving performance activity instantiated created maintained cicd continuous integration deployment pipelines apply automation environments applications worked various automation tools like git terraform ansible experienced fact dimensional modeling star schema snowflake schema transactional modeling scd slowly changing dimension experienced json based restful web services xmlqml based soap web services also worked various applications using python integrated ides like sublime text pycharm efficient cloud engineer years experience assembling cloud infrastructure utilizes strong managerial skills negotiating vendors coordinating tasks team members implements best practices create cloud functions applications databases big data technologies hadoop mapreduce hdfs sqoop pig hive hbase oozie flume nifi kafka zookeeper yarn apache spark mahout sparklib databases oracle mysql sql server mongodb cassandra dynamodb postgresql teradata cosmos programming python pyspark scala java c c shell script perl script sql cloud technologies aws microsoft azure frameworks django rest framework mvc hortonworks tools pycharm eclipse visual studio sqlplus sql developer toad sql navigator query analyzer sql server management studio sql assistance eclipse postman versioning tools svn git github operating systems windows 78xp20082012 ubuntu linux macos network security kerberos database modelling dimension modeling er modeling star schema modeling snowflake modeling monitoring tool apache airflow visualization reporting tableau ggplot2 matplotlib ssrs power bi machine learning techniques linear logistic regression classification regression trees random forest associative rules nlp clustering experience aws data engineer 012022 022022 deloitte rosslyn nv designed setup enterprise data lake provide support various uses cases including analytics processing storing reporting voluminous rapidly changing data designed developed security framework provide fine grained access objects aws s3 using aws lambda dynamodb set worked kerberos authentication principals establish secure network communication cluster testing hdfs hive pig mapreduce access cluster new users used aws emr transform move large amounts data aws data stores databases amazon simple storage service amazon s3 amazon dynamodb import data different sources like hdfshbase spark rdd perform computations using pyspark generate output response creating lambda functions boto3 deregister unused amis application regions reduce cost ec2 resources importing exporting database using sql server integrations services ssis data transformation services dts packages coded teradata bteq scripts load transform data fix defects like scd 2 date chaining cleaning duplicates conducted data blending data preparation using alteryx sql tableau consumption publishing data sources tableau server implemented aws step functions automate orchestrate amazon sagemaker related tasks publishing data s3 training ml model deploying prediction integrated apache airflow aws monitor multistage ml workflows tasks running amazon sagemaker environment aws emr s3 rds redshift lambda boto3 dynamodb amazon sagemaker apache spark hbase apache kafka hive sqoop map reduce snowflake apache pig python ssrs tableau assessed organization technology infrastructure managed cloud migration process daca engineer 012016 112019 bank america corporation arcadia ca worked azure data factory integrate data onprem sql cassandra cloud blob storage azure sql db applied transformations load back azure synapse managed configured scheduled resources across cluster using azure kubernetes service involved developing data ingestion pipelines azure hdinsight spark cluster using azure data factory spark sql develop dashboards visualizations help business users analyze data well providing data insight upper management focus microsoft products like sql server reporting services ssrs power bi performed migration large data sets databricks spark create administer cluster load data configure data pipelines loading data adls gen2 databricks using adf pipelines created various pipelines load data azure data lake staging sqldb followed azure sql db worked extensively azure data factory including data transformations integration runtimes azure key vaults triggers migrating data factory pipelines higher environments using arm templates ingested data minibatches performs rdd transformations minibatches data using spark streaming perform streaming analytics data bricks environment azure sql dw databrick azure synapse cosmos db adf ssrs power bi azure data lake arm azure hdinsight blob storage apache spark biodata engineerhadoop developer cumming llc aliso viejo ca ansibledenododenodocloudwatchavropysparkpysparkpysparkmllibdataframenifinifi 102013 122015 interacted business partners business analysts product owner understand requirements build scalable distributed data solutions using hadoop ecosystem developed spark streaming programs process near real time data kafka process data stateless state full transformations worked hive data warehouse infrastructurecreating tables data distribution implementing partitioning bucketing writing optimizing hql queries built implemented automated procedures split large files smaller batches data facilitate ftp transfer reduced 60 execution time worked developing etl processes data stage open studio load data multiple data sources hdfs using flume sqoop performed structural modifications using map reduce hive developing spark scripts udfs using spark dsl spark sql query data aggregation querying writing data back rdbms sqoop written multiple mapreduce jobs using java api pig hive data extraction transformation aggregationavrom multiple file formats including parquet avro xml json csv orcfile compressed file formats codecs like gzip snappy lzo strong understanding partitioning bucketing concepts hive designed managed external tables hive optimize performance developed pig udfs manipulating data according business requirements also worked developing custom pig loaders developing etl pipelines data warehouse using combination python snowflakes snowsql writing sql queries snowflake experience report writing using sql server reporting services ssrs creating various types reports like drill parameterized cascading conditional table matrix chart sub reports used datastax spark connector used store data cassandra database get data cassandra database wrote oozie scripts setting workflow using apache oozie workflow engine managing scheduling hadoop jobs worked implementation log producer scala watches application logs transform incremental log sends kafka zookeeper based log collection platform used hive analyze data ingested hbase using hivehbase integration compute various metrics reporting dashboard transformed tpyspark using aws glue dynamic frames pyspark cataloged transformed data using crawlers scheduled job crawler using workflow feature worked installing cluster commissioning decommissioning data node name node recovery capacity planning slots configuration developed data pipeline programs spark scala apis data aggregations hive formatting data json visualization generatipyspark environment aws cassandra pyspark apache spark hbase apache kafka hive sqoop flume apache oozie zookeeper etl udf map reduce snowflake apache pig python java ssrs onfidential developed implemented hadoop code observing coding standards optimized tuned hadoop environments modified hardware meet prescribed performance thresholds developed new functions applications conduct analyses created graphs charts detailing data analysis results tested validated reformulated models foster accurate prediction outcomes pyohon developer fiserv city state 092012 102013 aws s3 ec2 lambda ebs iam datadog cloud trail cli ansible mysql python git jenkins dynamodb cloud watch docker kubernetes leveraged open communication collective decisionmaking thorough reviews create performant scalable systems worked serverside frontend technologies leveraged common design patterns code dynamic userfriendly systems implemented new api routes architected new orm structures refactored code boost application performance harnessed version control tools coordinate project development individual code submissions introduced cloudbased technologies python development expand onpremise deployment options wrote clear clean code use projects resolved customer issues establishing workarounds solutions debug create defect fixes education training post graduate data engineering 022022 purdue university west lafayette pcot graduate data science business analytics 092021 university texas austin austin tx bachelor arts business administration management 122009 califonia state university fullerton ca