Project: Apache NiFi (App Definition and Development / Streaming & Messaging)
File: README.md

<!-- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> This modules provides AWS related extensions for NiFi Registry. AWS account credentials and an S3 bucket. The AWS extensions will be automatically included when you build NiFi Registry and will be installed at the `${NIFIREGHOME}/ext/aws` directory. If you wish to build NiFi Registry without including the AWS extensions, specify the `skipAws` system property: ``` cd nifi-registry mvn clean install -DskipAws ``` To add AWS extensions to an existing NiFi Registry, build the extension with the following command: ``` cd nifi-registry mvn clean install -f nifi-registry-extensions/nifi-registry-aws ``` The extension zip will be created as `nifi-registry-extensions/nifi-registry-aws/nifi-registry-aws-assembly/target/nifi-registry-aws-assembly-xxx-bin.zip`. Unzip the file into arbitrary directory so that NiFi Registry can use, such as `${NIFIREGHOME}/ext/aws`. For example: ``` mkdir -p ${NIFIREGHOME}/ext/aws unzip -d ${NIFIREGHOME}/ext/aws nifi-registry-extensions/nifi-registry-aws/nifi-registry-aws-assembly/target/nifi-registry-aws-assembly-xxx-bin.zip ``` In order to use this extension, the following NiFi Registry files need to be configured. The extension dir property will be automatically configured when building with the `include-aws` profile (i.e. when not specifying -DskipAws). To manually specify the property when adding the AWS extensions to an existing NiFi registry, configure the following property: ``` nifi.registry.extension.dir.aws=./ext/aws/lib ``` Uncomment the extensionBundlePersistenceProvider for S3: ``` <!-- <extensionBundlePersistenceProvider> <class>org.apache.nifi.registry.aws.S3BundlePersistenceProvider</class> <property name="Region">us-east-1</property> <property name="Bucket Name">my-bundles</property> <property name="Key Prefix"></property> <property name="Credentials Provider">DEFAULT_CHAIN</property> <property name="Access Key"></property> <property name="Secret Access Key"></property> <property name="Endpoint URL"></property> </extensionBundlePersistenceProvider> --> ``` NOTE: Remember to remove, or comment out, the FileSystemBundlePersistenceProvider since there can only be one defined. Of particular note, is the `AUTH` environment variable which is set to `tls`. Additionally, the user must provide an the DN as provided by an accessing client certificate in the `INITIALADMINIDENTITY` environment variable. This value will be used to seed the instance with an initial user with administrative privileges. Finally, this command makes use of a volume to provide certificates on the host system to the container instance. docker run --name nifi-registry \ -v /path/to/tls/certs/localhost:/opt/certs \ -p 18443:18443 \ -e AUTH=tls \ -e KEYSTORE_PATH=/opt/certs/keystore.jks \ -e KEYSTORE_TYPE=JKS \ -e KEYSTORE_PASSWORD=QKZv1hSWAFQYZ+WU1jjF5ank+l4igeOfQRp+OSbkkrs \ -e TRUSTSTORE_PATH=/opt/certs/truststore.jks \ -e TRUSTSTORE_PASSWORD=rHkWR1gDNW3R9hgbeRsT3OM3Ue0zwGtQqcFKJD2EXWE \ -e TRUSTSTORE_TYPE=JKS \ -e INITIALADMINIDENTITY='CN=AdminUser, OU=nifi' \ -d \ apache/nifi-registry:latest In this configuration, the user will need to provide certificates and the associated configuration information. Optionally, if the LDAP provider of interest is operating in LDAPS or START_TLS modes, certificates will additionally be needed. Of particular note, is the `AUTH` environment variable which is set to `ldap`. Additionally, the user must provide a DN as provided by the configured LDAP server in the `INITIALADMINIDENTITY` environment variable. This value will be used to seed the instance with an initial user with administrative

Finally, this command makes use of a volume to provide certificates on the host system to the container instance. For a minimal, connection to an LDAP server using SIMPLE authentication: docker run --name nifi-registry \ -v /path/to/tls/certs/localhost:/opt/certs \ -p 18443:18443 \ -e AUTH=ldap \ -e KEYSTORE_PATH=/opt/certs/keystore.jks \ -e KEYSTORE_TYPE=JKS \ -e KEYSTORE_PASSWORD=QKZv1hSWAFQYZ+WU1jjF5ank+l4igeOfQRp+OSbkkrs \ -e TRUSTSTORE_PATH=/opt/certs/truststore.jks \ -e TRUSTSTORE_PASSWORD=rHkWR1gDNW3R9hgbeRsT3OM3Ue0zwGtQqcFKJD2EXWE \ -e TRUSTSTORE_TYPE=JKS \ -e INITIALADMINIDENTITY='cn=nifi-admin,dc=example,dc=org' \ -e LDAPAUTHENTICATIONSTRATEGY='SIMPLE' \ -e LDAPMANAGERDN='cn=ldap-admin,dc=example,dc=org' \ -e LDAPMANAGERPASSWORD='password' \ -e LDAPUSERSEARCH_BASE='dc=example,dc=org' \ -e LDAPUSERSEARCH_FILTER='cn={0}' \ -e LDAPIDENTITYSTRATEGY='USE_DN' \ -e LDAP_URL='ldap://ldap:389' \ -d \ apache/nifi-registry:latest The following, optional environment variables may be added to the above command when connecting to a secure LDAP server configured with START_TLS or LDAPS -e LDAPTLSKEYSTORE: '' -e LDAPTLSKEYSTORE_PASSWORD: '' -e LDAPTLSKEYSTORE_TYPE: '' -e LDAPTLSTRUSTSTORE: '' -e LDAPTLSTRUSTSTORE_PASSWORD: '' -e LDAPTLSTRUSTSTORE_TYPE: '' In this configuration, the user will need to provide certificates and associated configuration information. Of particular note, is the `AUTH` environment variable which is set to `oidc`. Additionally, the user must provide a in the `INITIALADMINIDENTITY` environment variable. This value will be used to seed the instance with an initial user with administrative privileges. docker run --name nifi-registry \ -v $(pwd)/certs/localhost:/opt/certs \ -p 18443:18443 \ -e AUTH=oidc \ -e KEYSTORE_PATH=/opt/certs/keystore.p12 \ -e KEYSTORE_TYPE=PKCS12 \ -e KEYSTORE_PASSWORD=PLACEHOLDER \ -e TRUSTSTORE_PATH=/opt/certs/truststore.p12 \ -e TRUSTSTORE_PASSWORD=PLACEHOLDER \ -e TRUSTSTORE_TYPE=PKCS12 \ -e INITIALADMINIDENTITY=PLACHOLDER_USER \ -e NIFIREGISTRYSECURITYUSEROIDCDISCOVERYURL=http://OIDC_SERVER/.well-known/openid-configuration \ -e NIFIREGISTRYSECURITYUSEROIDCCONNECTTIMEOUT=10000 \ -e NIFIREGISTRYSECURITYUSEROIDCREADTIMEOUT=10000 \ -e NIFIREGISTRYSECURITYUSEROIDCCLIENTID=CLIENT_ID \ -e NIFIREGISTRYSECURITYUSEROIDCCLIENTSECRET=CLIENT_SECRET \ -e NIFIREGISTRYSECURITYUSEROIDCPREFERREDJWSALGORITHM=RS256 \ -e NIFIREGISTRYSECURITYUSEROIDCADDITIONALSCOPES=profile \ -e NIFIREGISTRYSECURITYUSEROIDCCLAIMIDENTIFYINGUSER=preferredusername \ -d \ apache/nifi-registry:latest The following, optional environment variables can be used to configure the database. | nifi-registry.properties entry | Variable | |-|-| | nifi.registry.db.url | NIFIREGISTRYDB_URL | | nifi.registry.db.driver.class | NIFIREGISTRYDB_CLASS | | nifi.registry.db.driver.directory | NIFIREGISTRYDB_DIR | | nifi.registry.db.username | NIFIREGISTRYDB_USER | | nifi.registry.db.password | NIFIREGISTRYDB_PASS | | nifi.registry.db.maxConnections | NIFIREGISTRYDBMAXCONNS | | nifi.registry.db.sql.debug | NIFIREGISTRYDBDEBUGSQL | The following, optional environment variables may be added to configure flow persistence provider. | Environment Variable | Configuration Property | |--|--| | NIFIREGISTRYFLOWSTORAGEDIR | Flow Storage Directory | | NIFIREGISTRYFLOW_PROVIDER | (Class tag); valid values: git, file | | NIFIREGISTRYGIT_REMOTE | Remote to Push | | NIFIREGISTRYGIT_USER | Remote Access User | | NIFIREGISTRYGIT_PASSWORD | Remote Access Password | | NIFIREGISTRYGIT_REPO | Remote Clone Repository | The following, optional environment variables may be added to configure extension bundle persistence provider. | Environment Variable | Configuration Property | ||-| | NIFIREGISTRYBUNDLESTORAGEDIR | Extension Bundle Storage Directory | | NIFIREGISTRYBUNDLE_PROVIDER | (Class tag); valid values: file, s3 | | NIFIREGISTRYS3_REGION | Region | | NIFIREGISTRYS3BUCKETNAME | Bucket Name | | NIFIREGISTRYS3KEYPREFIX | Key Prefix | | NIFIREGISTRYS3CREDENTIALSPROVIDER | Credentials Provider | | NIFIREGISTRYS3ACCESSKEY | Access Key | | NIFIREGISTRYS3SECRETACCESS_KEY | Secret Access Key | | NIFIREGISTRYS3ENDPOINTURL | Endpoint URL | n keys/user1-ks.p12 -passin pass:password -out keys/user1-key.pem -passout pass:password openssl pkcs12 -in keys/user1-ks.p12 -passin pass:password -out keys/user1-cert.pem -nokeys echo echo "New keys written to ${WD}/keys" echo "Copy to NiFi Registry test keys dir by running: " echo " cp -f \"$WD/keys/*\" /path/to/nifi-registry/nifi-registry-core/nifi-registry-web-api/src/test/resources/keys/" ``` You should now have a `/tmp/test-keys-YYYYMMDD-HHMMSS/keys` directory with all the necessary keys for testing with various tools. You can verify the contents of a keystore using the following command: keytool -list -v -keystore "$WD/keys/registry-ks.jks" -storepass password If you are satisfied with the results, you can copy the files from `/tmp/test-keys-YYYYMMDD-HHMMSS/keys` to this directory: cp -f "$WD/keys/*" /path/to/nifi-registry/nifi-registry-core/nifi-registry-web-api/src/test/resources/keys/