Job description:
role: lead data engineerlocation: remoteduration: long term 8 to 10 years of experience working in data engineering, business intelligence or related field delivering large scale data platforms and services. ideal candidate will have the following skills: a successful history of manipulating, processing and extracting value from large, disconnected datasets. expertise in data modeling, data access and data storage techniques. the candidate should have foundational knowledge of dimensional modeling to build a data warehouse strong experience building transformational logic using sql on relational sql and nosql databases. skills using dbt framework for data transformation is plus robust programming skills and strong experience with python to ingest, transform and load the data experience building and optimizing big data data pipelines, architectures and data sets using tools like airflow. experience with aws cloud services: ec2, emr, rds, redshift

Resume:
<<summary7+ years of it industry experience in in designing and developing data mining/analytics solutions, data centric integration, developing and maintaining business analytics.adept in data query, data migration, data analysis, predictive modeling, machine learning, data mining, and data visualization implementations with extensive use of sql, python, r, java, and unix shell scripting with platform of toad, oracle developer, jupyter notebook, pycharm, r-studio, tableau, hadoop with spark.experience in predictive analytic procedures used in supervised learning (classification, regression, decision trees, random forest, svm, neural networks), unsupervised learning (clustering-k-means and pca), and reinforcement learning.solid theory background for machine learning, data mining, text mining, graph mining, statistics modeling, nlp and deep learning.expert in natural language processing like pos tagging, parsing, named entity recognition, relationship extraction, information retrieval, sentimental classification, machine translation, etc.solid knowledge of deep learning algorithms like cnn, rnn, lstm, gru, etc. for text mining and image processing. sql server, teradata and mysql using a lot of subqueries, joins, aggregate functions, analytical functions, and temporary tables, etc.worked on big data analytics, hadoop ecosystems (hadoop, sqoop, hive, pig, mapreduce) and spark for big data migration, cleaning, transformation, processing, query and analysisfamiliar with software lifecycle includes requirement collecting/documentation, development, and testing for unit, smoke , integration, system, non-functional testing regarding performance, scalability, usability, enduration, load, and volume testing, and regression testing, multivariate testing (a/b testing) and system maintenance.work with business domain experts and application developers to identify data relevant analysis / mining and, develop new predictive / analytical modeling methods and/or tools in financial (like loan and foreign exchange), product, customer, sales domains, etc.experience in data aggregation and reduction techniques of large data sets with high performance and parallel computing for high performance analytical projects.involved in diagnosing and resolving predictive / analytical model performance issues, monitoring analytical system performance · and implementing efficiency improvements.conversant with ms sql/ oracle plsql and rdbms. contributed in data definitions for new database file/table development and/or changes to existing ones as needed for analysis and mining purpose.experience d oracle pl/sql developer for designing, developing, debugging , maintaining and administrating database in oracle rdbms. solid experience in pl/sql and sql programming and performance tuning.familiarity with oracle data warehousing features such as star & snowflake data modeling schemas, materialized views, bitmap indexes, index organized tables, external tables etc. and oltp system using b-tree index, hash join, etc.experience d in front-end developing using java, javascript, c++ and back-end developing using c.professional in integrating and maintaining code using version control tools pvcs, svn, cvs.solid experience and knowledge in etl and data warehousing concepts.data processing experience in designing, implementing transformation processes using etl tool.involved in all aspects of etl- requirement gathering with standard interfaces to be used by operational sources, data cleaning, data loading strategies, etl mappings designing, documentation and etl jobs performance testing.using unix bash/corn shell scripting to do backend process operation, system resources checking, job scheduling, batch data loading, performance tuning and reporting.conversant with project management deliverables and sdlc phases – waterfall and agile.a self-starter, team player, excellent communicator, prolific researcher.expert technical documentation skills. strong interpersonal and communication skills (both written and oral); ability to communicate with people in a wide variety of areas and at various levels from technical specialists to senior management. skills roles: data scientist, data analyst, business system analyst, oracle plsql developerdata visualization: d3.js, tableau, r visualization packages, microsoft exceldata analytics tools/programming: python (numpy, scipy, pandas, scikit-learn, gensim, keras), r(caret, weka, ggplot ), matlab, microsoft sql server, oracle plsqlmachine learning algorithms: classification, regression, clustering, feature engineeringbig data tools: hadoop, mapreduce, sqoop, pig, hive, nosql, cassandra, sparkothers:deep learning, nlp, topic modeling, sklearn, graph mining, text mining, c++ , java, javascript, asp, shell scripting experience 08/2011to 09/2016data scientistlockheed martin corporation– ,actively develop predictive models and strategies for effective fraud detection for credit and customer banking activities using kmeans clustering using python.assisted senior data scientist to do text mining on customer review/comment data, using topic modeling and sentimental classification, using deep learning algorithms like cnn, rnn, lstm, remediate according financial products using python.assisted senior quantitative analyst in assessing risk management of financial derivative products like foreign exchange products, bonds, funds, etc. using machine learning techniques for providing appropriate investment recommendations using collaborative filtering recommender system using python.mentored sophisticated organizations on large scale customer data and analytics using advanced machine learning and statistical models relying for issuing loan using random forest using r.performed k-means clustering in order to understand customer backgrounds and segment the customers based on the customer transaction behavior information for customized product offering, improve existing profitable relationships and to avoid customer churn, etc using r.worked on interactive dashboards for building story and presenting to business using tableau.implementing hadoop to provision big data analytics platforms for customer data. used mapreduce, sqoop, hive and spark to migrate and analyze large call-quality-data datasets from multiple data sources like integrated funds transfer system like fedwire, chips, swift for securities, treasury or derivatives, and web-based cash management systems egifts, giftsweb edd for fraud detection and risk management, for accounts based on positive pay, and automated cash handling, balance reporting, etc.installed and configured hadoop cluster in test and production environments; moving data from oracle 9i database to hdfs and vice-versa using sqoop; collecting and aggregating large amounts of log data using apache flume and staging data in hdfs for further analysis; developed multiple mapreduce jobs in java for data cleaning and preprocessing; writing pig scripts to transform raw data from several data sources into forming baseline data; solved performance issues in hive and pig scripts with understanding of joins, group and aggregation and how does it translate to mapreduce jobs; developed oozie workflow for scheduling and coordinating the etl process; using spark for further data analysis/mining.experience in using sequence files, rcfile, avro and har file formats.work with data analytics team to develop time series and optimization.involved in development and maintenance of oracle database using plsql and back-end development using c/c++ for intra-net management system for employee management system (ems) and agent pay-out system (aps). 04/2009to 08/2011business system analystaccenture– ,project summary:this project is in application service group for mercury system in canon usa, inc. , which mainly in charge of the new item request, item disclosure between companies, item data import from other canon americas companies to canon usa. canon americas master inquiry, model tree maintenance, model configurations, and camera/video merchandise maintenance for canon americas systems includes- s21 for canon usa merchandise master, s98 for panama, cci21 for canada, s85 for mexico, chili, brazil, and argentina, ideal for latin americas countries.hardware/software:windows vista/nt/xp/7, linux, oracle 11g/10g/9i/8i, sql plus , oracle sql developer, toad, microsoft sql server management studio 2008,microsoft visual studio 6.0,odbc/jdbc , microsoft iis 5.1.1, putty, cygwin, winmerge, vpn, itg project management system, pqedit, iis, autosys-pc-xware 5.1.0, ms word, excel, access, project, visioresponsibilities:operational support for canon americas mercury system, includes data adjustment/research, batch data loading, system migration, technical and functional specification documentation, reports, business process alignment, workflow stuck, and reconciliation, etc.break/fix any issues or bugs collected from client and development regarding setup, performance, functionality, and workflow stuck, etc.system enhancement regarding functionality and performance, etc.reproduce and review existing oracle 9i schema objects includes tables, temporary tables, views, materialized views, indexes, triggers, procedures, functions, packages based on customer requirement and system upgrade using toad and oracle sql developer tools.review and analyze asp code for umc mercury application web development for data research and system feature fix and enhancement using visual interdev 6.0.query real-time data regarding canon americas new item request, item status inquiry, item data disclosure and import, model configuration and maintenance, warranty maintenance, and model configuration inquiry , etc. using complex sql queries on oracle 9i canon mercury database.using unix bash/corn shell scripting to do backend process operation, system resources checking, job scheduling, batch data loading, performance tuning and reporting.maintain scheduled day and night batch jobs for mercury system using autosys-pc-xware and unix box and check the mq series using pq edit.implement client, session, action, module, service, instance level end-to-end application tracing using sql trace with tkprof, and explain plan to check execution plans for high-load and top sql statement.using cygwin/ ftp- putty with unix bash shell to make a tunnel for oracle database connection.using tortoise svn for code check-out, update, and release–comparison, etc.tracking and documenting tickets for development and reproduction using itg ticket tracking system.assisted qa and build team to be involved in unit, smoke, integration, system, uat, non-functional testing regarding performance, scalability, usability, enduration, load, and volume testing, and regression testing and maintenance using soup ui and seapine qa wizard pro for product release.data loading using imp/exp, data pump, and external tables from americas mercury system to s21- ross-cusa retails system, and global mercury system.2008to 04/2009oracle plsql developercognizant technology solutions– ,project summary:nyu langone medical center, a world-class patient- one of the nation’s premier centers for excellence in healthcare, biomedical research, and medical education. the project is to develop new oracle database objects on online health information managment system on findwdev instance/server for 29 nyu medical school departments using in clinical, education, research, and foundational areas, etc. provide further data to be loaded into dwh for historical record, decision support and datamart for reports, and cube for ui display.hardware/software:unix, oracle 11g, oracle developer 11g, oracle ebs 11, erp r12, ibm datastage 8.0 (designer, director, manager, parallel extender), sql server management studio 2008, toad for oracle 9.0, tsql & pl/sql, xml, erwin, microsoft visio, autosys, ibm data stage 8, oracle reports 11g.responsibilities:independently develop oracle database objects includes tables, views, materialized views, indexes, triggers, functions, procedures, packages, etc.cooperated with ba, sme to collect and document database design requirements and do data modeling with db architect using erwin and microsoft visio.assisted dba for job scheduling, data loading, and performance tuning using oem, sql tuning/access advisor, hints, explain plan , sql trace and v$ performance views under unix.write complicated queries using a lot of aggregate functions, joins, analytical functions, subqueries , etc. to provide real-time data from oracle db for client and ui development supporting.checking execution plan using explain plan together with sql trace with tkprof using trcsess under unix to realized end-to-end application tracing.add optimization hints into high–load and top sql statements to change the optimization goal, access method, join method, join order, and parallelization, etc.designed and developed etl processes using datastage designer to load data from oracle to staging database and from staging to the target data warehouse.worked with datastage manager for importing metadata from repository, new job categories and creating new data elements.used datastage stages namely hash file, sequential file, transformer, aggregate, sort, datasets, join, lookup, change capture, funnel, peek, row generator stages in accomplishing the etl coding.job scheduling using autosys. coorporated with qa team for debugging, unit, system, , regression testing for new iso release production.working on linux system for batch data loading, job scheduling, and system resource checking, etc.assisted back-end developer for reviewing and debugging c program for health information management systems.involved in web development of online health information management systems using java.reviewed and reproduced online java reports of health information management system, checked db references in it for intelligent decision support system education expected in2016totomaster of science:computer scienceuniversity of california-los angeles,cagpa:gpa:3.78courses: -statistics programming -databases and knowledge bases -graphs and network flows -language and thought -current topics in computer theory(machine learning algorithm) -computer science classics(basic data science) -data mining and big data analytics -system securityexpected in2010totomaster of science:electrical engineeringuniversity of bridgeport-bridgeport,ctgpa:gpa:3.62courses: computer networks -database management systems -data and computer communications -data structuresexpected in2007totobachelor of science:telecommunications engineeringjilin university-changchun,jilingpa:gpa: 3.50.