Migrating from Apache Storm to Apache Flink
Published: 2021-01-26 19:39:50

---
title: Migrating from Apache Storm to Apache Flink
published: true
date: 2021-01-14 16:28:00 UTC
tags: 
canonical_url: 
---

#  Migrating from Apache Storm to Apache Flink

The first thing you need to do is to not just pick up and dump to a new system, but to see what can be reconfigured, refactored or reimagined.   For some routing, transformation or simple ingest type applications or solution parts you may want to use Apache NiFi.

For others Spark or Spark Streaming can quickly meet your needs.   For simple Thing to Kafka or Kafka to Thing flows, a flow with Kafka Connect is appropriate.   For things that need to run in individual devices, containers, pods you may want to move a small application to NiFi Stateless.    There are also sometimes a simple Kafka Stream application will meet your needs.

For many use cases you can replace a compiled application with some solid Flink SQL code.   For some discussions, check this [out](https://dev.to/tspannhw/did-the-user-really-ask-for-exactly-once-fault-tolerance-3fek).

For some really good information on how to migrate Storm solutions to Flink,  Cloudera has a well documented solution for you:

[https://docs.cloudera.com/csa/1.2.0/stormflink-migration/topics/csa-stormflink-migration-process.html](https://docs.cloudera.com/csa/1.2.0/stormflink-migration/topics/csa-stormflink-migration-process.html)

## Conceptual

[https://docs.cloudera.com/csa/1.2.0/stormflink-migration/topics/csa-stormflink-concept.html](https://docs.cloudera.com/csa/1.2.0/stormflink-migration/topics/csa-stormflink-concept.html)

[![](https://lh3.googleusercontent.com/-OAZvhCJkjMs/YABvc9A_QCI/AAAAAAAAcZ8/undGq992x7wpqcNw0WUeWY90ivUFGH5yACLcBGAsYHQ/w640-h537/image.png)](https://lh3.googleusercontent.com/-OAZvhCJkjMs/YABvc9A_QCI/AAAAAAAAcZ8/undGq992x7wpqcNw0WUeWY90ivUFGH5yACLcBGAsYHQ/image.png)
  
  

## Architecture

[https://docs.cloudera.com/csa/1.2.0/stormflink-migration/topics/csa-stormflink-architecture.html](https://docs.cloudera.com/csa/1.2.0/stormflink-migration/topics/csa-stormflink-architecture.html)

  

[![](https://lh3.googleusercontent.com/-I5SYOf-CqBQ/YABvhLBqYQI/AAAAAAAAcaE/3OElq6PjeVwMyFo3GV4pfu9a8OSZZRyyACLcBGAsYHQ/w616-h640/image.png)](https://lh3.googleusercontent.com/-I5SYOf-CqBQ/YABvhLBqYQI/AAAAAAAAcaE/3OElq6PjeVwMyFo3GV4pfu9a8OSZZRyyACLcBGAsYHQ/image.png)
  
  

## Redistribution

[https://docs.cloudera.com/csa/1.2.0/stormflink-migration/topics/csa-stormflink-redistribution.html](https://docs.cloudera.com/csa/1.2.0/stormflink-migration/topics/csa-stormflink-redistribution.html)

[![](https://lh3.googleusercontent.com/-zym046I_Hu8/YABvQ2C717I/AAAAAAAAcZ4/Sv_1FI8U1sQmNfgi6ggZuYyhIo6D_ah9QCLcBGAsYHQ/w640-h571/image.png)](https://lh3.googleusercontent.com/-zym046I_Hu8/YABvQ2C717I/AAAAAAAAcZ4/Sv_1FI8U1sQmNfgi6ggZuYyhIo6D_ah9QCLcBGAsYHQ/image.png)

## References

- [https://docs.cloudera.com/csa/1.2.0/development/topics/csa-application-logic.html](https://docs.cloudera.com/csa/1.2.0/development/topics/csa-application-logic.html)  
- [https://www.datainmotion.dev/2020/12/smart-stocks-with-flank-nifi-kafka.html](https://dev.to/tspannhw/smart-stocks-with-flank-nifi-kafka-flink-sql-4ee9)
- [https://www.datainmotion.dev/2019/11/introducing-mm-flank-apache-flink-stack.html](https://dev.to/tspannhw/introducing-mm-flank-an-apache-flink-stack-for-rapid-streaming-development-from-edge-2-ai-5c12)
- [https://www.datainmotion.dev/2020/07/flank-in-cloud-huge-cloudera-data.html](https://dev.to/tspannhw/flank-in-the-cloud-huge-cloudera-data-platform-public-cloud-updates-july-2020-data-flow-releases-3l27)
- [https://www.datainmotion.dev/2020/05/flank-low-code-streaming-populating.html](https://dev.to/tspannhw/flank-low-code-streaming-populating-kafka-topics-with-flinksql-joins-in-real-time-3jna)
- [https://www.datainmotion.dev/2020/10/running-flink-sql-against-kafka-using.html](https://dev.to/tspannhw/running-flink-sql-against-kafka-using-a-schema-registry-catalog-480a)
- [https://github.com/tspannhw/EverythingApacheNiFi](https://github.com/tspannhw/EverythingApacheNiFi)
- [https://github.com/tspannhw/ApacheConAtHome2020](https://github.com/tspannhw/ApacheConAtHome2020)
- [https://github.com/tspannhw/SmartStocks](https://github.com/tspannhw/SmartStocks)
- [https://github.com/tspannhw/ClouderaFlinkSQLForPartners](https://github.com/tspannhw/ClouderaFlinkSQLForPartners)
- [https://github.com/tspannhw/SmartWeather](https://github.com/tspannhw/SmartWeather)