Category: Python Developer

summary skills jessica claire 100 montgomery st 10th floor555 4321000 resumesampleexamplecom dynamic motivated professional around 15 years experience principal software developer expertise designing data intensive applications using spark ecosystem big data analytical cloud data engineering data warehouse data mart data visualization reporting data quality solutions profound experience performing data ingestion data processing transformations enrichment aggregations strong knowledge architecture distributed systems parallel processing indepth understanding spark execution framework experienced spark improving performance optimization existing algorithms hadoop using spark context sparksql dataframe api worked explicitly scala handled ingestion data different data sources hdfs using sqoop perform transformations using hive map reduce loading data hdfs experience partitions bucketing concepts scala worked fine tuning sparkscala sql queries optimize performance experience different file formats like avro parquet json xml expertise creating debugging scheduling monitoring jobs using airflow experienced using common operators airflow python operator bash operator handson experience handling database issues connections sql databases netezza oracle redshift postgresql experience designing creating rdbms tables views user created data types stored procedures expert designing etl data flows using creating mappingsworkflows extract data netezza oracle servers expert designing parallel jobs using various stages like join merge lookup remove duplicates filter dataset lookup file set complex flat file modify aggregator xml handson experience amazon ec2 amazon s3 amazon rds vpc iam amazon elastic load balancing auto scaling cloudwatch sns ses sqs lambda emr services aws family created configured new batch job denodo scheduler email notification capabilities implemented cluster setting multiple denodo node created load balance improving performance activity instantiated created maintained cicd continuous integration deployment pipelines apply automation environments applications worked code versioning automation tools like git experienced json based restful web services efficient cloud engineer years experience assembling cloud infrastructure utilizes strong managerial skills negotiating vendors coordinating tasks team members implements best practices create cloud functions applications databases big data technologies hadoop mapreduce hdfs sqoop pig hive hbase oozie flume nifi kafka zookeeper yarn apache spark mahout sparklib databases oracle mysql sql server mongodb cassandra dynamodb postgresql teradata cosmos operating systems windows 78xp20082012 ubuntu linux macos network security kerberos database modelling dimension modeling er modeling star schema modeling snowflake modeling monitoring tool apache airflow visualization reporting tableau ggplot2 programming python pyspark scala java c c matplotlib ssrs power bi shell script perl script sql cloud technologies aws microsoft azure frameworks django rest framework mvc hortonworks tools pycharm eclipse visual studio sqlplus sql developer toad sql navigator query analyzer sql server management studio sql assistance eclipse postman versioning tools svn git github experience aws data engineer deloitte rosslyn ca machine learning techniques linear logistic regression classification regression trees random forest associative rules nlp clustering 012022 022022 designed setup enterprise data lake provide support various uses cases including analytics processing storing reporting voluminous rapidly changing data responsible maintaining quality reference data source performing operations cleaning transformation ensuring integrity relational environment working closely stakeholders solution architect designed developed security framework provide fine grained access objects aws s3 using aws lambda performed endtoend architecture implementation assessment various aws services like amazon emr redshift s3 used aws emr transform move large amounts data aws data stores databases amazon simple storage service amazon s3 amazon rds used spark sql scala python interface automatically converts rdd case classes schema rdd import data different sources like s3 hdfs spark rdd perform computations using sparkscalasparksql generate output response creating automated alert notifications identify notify idle emr ec2 clusters application regions reduce cost ec2 emr resources developed reusable framework leveraged future migrations automates etl rdbms systems data lake utilizing spark data sources hive data objects developed grafana dashboards based log stash data integrated different source target systems elasticsearch near real time log analysis monitoring end end transactions environment aws emr s3 rds redshift lambda apache spark hive sqoop map reduce python assessed organization technology infrastructure managed cloud migration process implemented cloud policies managed technology requests maintained service availability daca engineer splunk memphis tn 012016 112019 worked azure data factory integrate data onprem sql cassandra cloud blob storage azure sql db applied transformations load back azure synapse managed configured scheduled resources across cluster using azure kubernetes service monitored spark cluster using log analytics ambari web ui transitioned log storage cassandra azure sql datawarehouse improved query performance involved developing data ingestion pipelines azure hdinsight spark cluster using azure data factory spark sql also worked cosmos db sql api mongo api develop dashboards visualizations help business users analyze data well providing data insight upper management focus microsoft products like sql server reporting services ssrs power bi performed migration large data sets databricks spark create administer cluster load data configure data pipelines loading data adls gen2 databricks using adf pipelines created various pipelines load data azure data lake staging sqldb followed azure sql db created databrick notebooks streamline curate data various business use cases also mounted blob storage databrick utilized azure logic apps build workflows schedule automate batch jobs integrating apps adf pipelines services like http requests email triggers etc worked extensively azure data factory including data transformations integration runtimes azure key vaults triggers migrating data factory pipelines higher environments using arm templates ingested data minibatches performs rdd transformations minibatches data using spark streaming perform streaming analytics data bricks environment azure sql dw databrick azure synapse cosmos db adf ssrs power bi azure data lake arm azure hdinsight blob storage apache spark adept troubleshooting identifying current issues providing effective solutions managed performance monitoring tuning identifying repairing issues within database realm identified key use cases associated reference architectures market segments industry verticals designed surveys opinion polls assessment tools collect data created graphs charts detailing data analysis results tested validated reformulated models foster accurate prediction outcomes recommended data analysis tools address business issues developed new functions applications conduct analyses cleaned manipulated raw data collaborated solution architects define database analytics engagement strategies operational territories key accounts biodata engineerhadoop developer general dynamics bossier city 102013 122015 ansibledenododenodocloudwatchavropysparkpysparkpysparkmllibdataframenifinifi interacted business partners business analysts product owner understand requirements build scalable distributed data solutions using hadoop ecosystem developed spark streaming programs process near real time data kafka process data stateless state full transformations worked hive data warehouse infrastructurecreating tables data distribution implementing partitioning bucketing writing optimizing hql queries built implemented automated procedures split large files smaller batches data facilitate ftp transfer reduced 60 execution time worked developing etl processes data stage open studio load data multiple data sources hdfs using flume sqoop performed structural modifications using map reduce hive developing spark scripts udfs using spark dsl spark sql query data aggregation querying writing data back rdbms sqoop written multiple mapreduce jobs using java api pig hive data extraction transformation aggregationavrom multiple file formats including parquet avro xml json csv orcfile compressed file formats codecs like gzip snappy lzo strong understanding partitioning bucketing concepts hive designed managed external tables hive optimize performance developed pig udfs manipulating data according business requirements also worked developing custom pig loaders developing etl pipelines data warehouse using combination python snowflakes snowsql writing sql queries snowflake experience report writing using sql server reporting services ssrs creating various types reports like drill parameterized cascading conditional table matrix chart sub reports used datastax spark connector used store data cassandra database get data cassandra database wrote oozie scripts setting workflow using apache oozie workflow engine managing scheduling hadoop jobs worked implementation log producer scala watches application logs transform incremental log sends kafka zookeeper based log collection platform used hive analyze data ingested hbase using hivehbase integration compute various metrics reporting dashboard transformed tpyspark using aws glue dynamic frames pyspark cataloged transformed data using crawlers scheduled job crawler using workflow feature worked installing cluster commissioning decommissioning data node name node recovery capacity planning slots configuration developed data pipeline programs spark scala apis data aggregations hive formatting data json visualization generatipyspark environment aws cassandra pyspark apache spark hbase apache kafka hive sqoop flume apache oozie zookeeper etl udf map reduce snowflake apache pig python java ssrs onfidential developed implemented hadoop code observing coding standards optimized tuned hadoop environments modified hardware meet prescribed performance thresholds developed new functions applications conduct analyses created graphs charts detailing data analysis results tested validated reformulated models foster accurate prediction outcomes pyohon developer fiserv city state 092012 102013 aws s3 ec2 lambda ebs iam datadog cloud trail cli ansible mysql python git jenkins dynamodb cloud watch docker kubernetes leveraged open communication collective decisionmaking thorough reviews create performant scalable systems worked serverside frontend technologies leveraged common design patterns code dynamic userfriendly systems implemented new api routes architected new orm structures refactored code boost application performance harnessed version control tools coordinate project development individual code submissions introduced cloudbased technologies python development expand onpremise deployment options wrote clear clean code use projects resolved customer issues establishing workarounds solutions debug create defect fixes education post graduate data engineering 022022 training purdue university west lafayette pcot graduate data science business analytics 092021 university texas austin austin tx bachelor arts business administration management 122009 califonia state university fullerton ca