Basic Understanding of Cloudera Flow Management - Apache NiFi
Published: 2020-12-13 20:19:03

---
title: Basic Understanding of Cloudera Flow Management - Apache NiFi
published: true
date: 2020-12-01 20:12:00 UTC
tags: 
canonical_url: 
---

# ** Basic Understanding of Cloudera Flow Management - Apache NiFi**

  

### **Topics:**

- NiFi Cluster Architecture
- Content Repository
- EncryptedContentRepository and other options
- Provenance Repository
- FlowFile Repository
- FlowFile, Attributes, Process Groups, Connections, Flow Controllers
- Controller Services
- Custom Properties
- Common Attributes (uuid, filename, path, file size, ...)
- Expression Language
- Flow Routing
- Testing and Test Data Generation
- Relationships
- Ports
- Bulletins
- flow.xml.gz
- Input Port
- Output Port
- Empty Queues
- Setting Warning Levels
- Funnels
- Copy on Write
- RecordPath 
- Using Record Processors (Readers/Writers)
- NiFi Toolkit
- NiFi CLI
- NiFi REST API
- NiFi Registry Integration
- Handling Errors
- Parameter Context / Parameters
- Summary / Cluster / Bulletins
- Reporting Tasks
- Sizing NiFi Cluster on # of records \* size / amount of time
- Configuration Files (Changing RAM)
- Understanding NiFi logs
- How to add custom processors
- JVM 
- Back pressure
- Prioritized Queues
- Load Balancing
- Load Balancing Strategies
- Prioritization
- Monitoring a Flow
- Using Search
- Using Documentation
- Classloader
- Site-to-Site Communication / Remote Process Groups
- Extensions
- Scheduling
- Tailing Files
- Reading sFTP/FTP Files
- Wait and Notify
- RetryFlowFile Pattern
- NiFi Calcite SQL 
- Using Jolt
- Using JsonPath
- Using Kerberos
- Using SSL
- Making REST Calls
- Receiving REST Calls
- Working with Websockets
- Working with TCP/IP, UDP, Sockets
- Working with Files, Logs, Syslog
- Producing and Consuming Kafka
- Working with HDFS
- Reading/Writing Hive
- Reading/Writing Impala/Kudu
- Reading/Writing HBase
- Integration with Ranger
- Integration with Knox
- Integration with Atlas
- LookupRecord
- Working with Caches
- Restarting Flows
- Pass by Reference
- Working with XML
- Working with JSON
- Working with AVRO
- Working with Schema Registry
- Using Regular Expressions
- Funnels

  

  

[![](https://lh3.googleusercontent.com/-jI5rvbD863A/X8aX7rBrMOI/AAAAAAAAb-Q/WJ7P964vF-8P_5tW9mPppPSj_xBAGsk7gCLcBGAsYHQ/w558-h293/image.png)](https://lh3.googleusercontent.com/-jI5rvbD863A/X8aX7rBrMOI/AAAAAAAAb-Q/WJ7P964vF-8P_5tW9mPppPSj_xBAGsk7gCLcBGAsYHQ/image.png)
  
  

### Must read:

  

- [https://github.com/tspannhw/EverythingApacheNiFi](https://github.com/tspannhw/EverythingApacheNiFi)
- [https://nifi.apache.org/docs/nifi-docs/html/overview.html](https://nifi.apache.org/docs/nifi-docs/html/overview.html)
- [https://nifi.apache.org/docs/nifi-docs/html/getting-started.html](https://nifi.apache.org/docs/nifi-docs/html/getting-started.html)
- [https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html](https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html)
- [https://nifi.apache.org/docs/nifi-docs/html/record-path-guide.html](https://nifi.apache.org/docs/nifi-docs/html/record-path-guide.html)
- [https://nifi.apache.org/docs/nifi-docs/html/nifi-in-depth.html](https://nifi.apache.org/docs/nifi-docs/html/nifi-in-depth.html)
- [https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html](https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html)
- [https://nifi.apache.org/docs/nifi-docs/rest-api/index.html](https://nifi.apache.org/docs/nifi-docs/rest-api/index.html)
- [https://nifi.apache.org/docs/nifi-docs/html/user-guide.html](https://nifi.apache.org/docs/nifi-docs/html/user-guide.html)
- [https://nifi.apache.org/docs/nifi-docs/html/toolkit-guide.html](https://nifi.apache.org/docs/nifi-docs/html/toolkit-guide.html)
- [https://www.datainmotion.dev/2020/06/no-more-spaghetti-flows.html](https://dev.to/tspannhw/no-more-spaghetti-flows-2emd)