Chapter 17. Running batch operations Data Grid Operator provides a Batch CR that lets you create Data Grid resources in bulk. Batch CR uses the Data Grid command line interface (CLI) in batch mode to carry out sequences of operations. Note Modifying a Batch CR instance has no effect. Batch operations are "one-time" events that modify Data Grid resources. To update .spec fields for the CR, or when a batch operation fails, you must create a new instance of the Batch CR. 17.1. Running inline batch operations Include your batch operations directly in a Batch CR if they do not require separate configuration artifacts. Procedure Create a Batch CR. Specify the name of the Data Grid cluster where you want the batch operations to run as the value of the spec.cluster field. Add each CLI command to run on a line in the spec.config field. apiVersion: infinispan.org/v2alpha1 kind: Batch metadata: name: mybatch spec: cluster: infinispan config: | create cache --template=org.infinispan.DIST_SYNC mycache put --cache=mycache hello world put --cache=mycache hola mundo Apply your Batch CR. Wait for the Batch CR to succeed. 17.2. Creating ConfigMaps for batch operations Create a ConfigMap so that additional files, such as Data Grid cache configuration, are available for batch operations. Prerequisites For demonstration purposes, you should add some configuration artifacts to your host filesystem before you start the procedure: Create a /tmp/mybatch directory where you can add some files. Create a Data Grid cache configuration. Procedure Create a batch file that contains all commands you want to run. For example, the following batch file creates a cache named "mycache" and adds two entries to it: create cache mycache --file=/etc/batch/mycache.xml put --cache=mycache hello world put --cache=mycache hola mundo Important The ConfigMap is mounted in Data Grid pods at /etc/batch . You must prepend all --file= directives in your batch operations with that path. Ensure all configuration artifacts that your batch operations require are in the same directory as the batch file. Create a ConfigMap from the directory. 17.3. Running batch operations with ConfigMaps Run batch operations that include configuration artifacts. Prerequisites Create a ConfigMap that contains any files your batch operations require. Procedure Create a Batch CR that specifies the name of a Data Grid cluster as the value of the spec.cluster field. Set the name of the ConfigMap that contains your batch file and configuration artifacts with the spec.configMap field. cat > mybatch.yaml<<EOF apiVersion: infinispan.org/v2alpha1 kind: Batch metadata: name: mybatch spec: cluster: infinispan configMap: mybatch-config-map EOF Apply your Batch CR. Wait for the Batch CR to succeed. 17.4. Batch status messages Verify and troubleshoot batch operations with the status.Phase field in the Batch CR. Phase Description Succeeded All batch operations have completed successfully. Initializing Batch operations are queued and resources are initializing. Initialized Batch operations are ready to start. Running Batch operations are in progress. Failed One or more batch operations were not successful. Failed operations Batch operations are not atomic. If a command in a batch script fails, it does not affect the other operations or cause them to rollback. Note If your batch operations have any server or syntax errors, you can view log messages in the Batch CR in the status.Reason field. 17.5. Example batch operations Use these example batch operations as starting points for creating and modifying Data Grid resources with the Batch CR. Note You can pass configuration files to Data Grid Operator only via a ConfigMap . The ConfigMap is mounted in Data Grid pods at /etc/batch so you must prepend all --file= directives with that path. 17.5.1. Caches Create multiple caches from configuration files. echo "creating caches..." create cache sessions --file=/etc/batch/infinispan-prod-sessions.xml create cache tokens --file=/etc/batch/infinispan-prod-tokens.xml create cache people --file=/etc/batch/infinispan-prod-people.xml create cache books --file=/etc/batch/infinispan-prod-books.xml create cache authors --file=/etc/batch/infinispan-prod-authors.xml echo "list caches in the cluster" ls caches Create a template from a file and then create caches from the template. echo "creating caches..." create cache mytemplate --file=/etc/batch/mycache.xml create cache sessions --template=mytemplate create cache tokens --template=mytemplate echo "list caches in the cluster" ls caches 17.5.2. Counters Use the Batch CR to create multiple counters that can increment and decrement to record the count of objects. You can use counters to generate identifiers, act as rate limiters, or track the number of times a resource is accessed. echo "creating counters..." create counter --concurrency-level=1 --initial-value=5 --storage=PERSISTENT --type=weak mycounter1 create counter --initial-value=3 --storage=PERSISTENT --type=strong mycounter2 create counter --initial-value=13 --storage=PERSISTENT --type=strong --upper-bound=10 mycounter3 echo "list counters in the cluster" ls counters 17.5.3. Protobuf schema Register Protobuf schema to query values in caches. Protobuf schema ( .proto files) provide metadata about custom entities and controls field indexing. echo "creating schema..." schema --upload=person.proto person.proto schema --upload=book.proto book.proto schema --upload=author.proto book.proto echo "list Protobuf schema" ls schemas 17.5.4. Tasks Upload tasks that implement org.infinispan.tasks.ServerTask or scripts that are compatible with the javax.script scripting API. echo "creating tasks..." task upload --file=/etc/batch/myfirstscript.js myfirstscript task upload --file=/etc/batch/mysecondscript.js mysecondscript task upload --file=/etc/batch/mythirdscript.js mythirdscript echo "list tasks" ls tasks Additional resources Data Grid CLI Guide

Commands:
apiVersion: infinispan.org/v2alpha1 kind: Batch metadata: name: mybatch spec: cluster: infinispan config: | create cache --template=org.infinispan.DIST_SYNC mycache put --cache=mycache hello world put --cache=mycache hola mundo
apply -f mybatch.yaml
wait --for=jsonpath='{.status.phase}'=Succeeded Batch/mybatch
mkdir -p /tmp/mybatch
cat > /tmp/mybatch/mycache.xml<<EOF <distributed-cache name="mycache" mode="SYNC"> <encoding media-type="application/x-protostream"/> <memory max-count="1000000" when-full="REMOVE"/> </distributed-cache> EOF
create cache mycache --file=/etc/batch/mycache.xml put --cache=mycache hello world put --cache=mycache hola mundo
ls /tmp/mybatch batch mycache.xml
create configmap mybatch-config-map --from-file=/tmp/mybatch
cat > mybatch.yaml<<EOF apiVersion: infinispan.org/v2alpha1 kind: Batch metadata: name: mybatch spec: cluster: infinispan configMap: mybatch-config-map EOF
apply -f mybatch.yaml
wait --for=jsonpath='{.status.phase}'=Succeeded Batch/mybatch
echo "creating caches..." create cache sessions --file=/etc/batch/infinispan-prod-sessions.xml create cache tokens --file=/etc/batch/infinispan-prod-tokens.xml create cache people --file=/etc/batch/infinispan-prod-people.xml create cache books --file=/etc/batch/infinispan-prod-books.xml create cache authors --file=/etc/batch/infinispan-prod-authors.xml echo "list caches in the cluster" ls caches
echo "creating caches..." create cache mytemplate --file=/etc/batch/mycache.xml create cache sessions --template=mytemplate create cache tokens --template=mytemplate echo "list caches in the cluster" ls caches
echo "creating counters..." create counter --concurrency-level=1 --initial-value=5 --storage=PERSISTENT --type=weak mycounter1 create counter --initial-value=3 --storage=PERSISTENT --type=strong mycounter2 create counter --initial-value=13 --storage=PERSISTENT --type=strong --upper-bound=10 mycounter3 echo "list counters in the cluster" ls counters
echo "creating schema..." schema --upload=person.proto person.proto schema --upload=book.proto book.proto schema --upload=author.proto book.proto echo "list Protobuf schema" ls schemas
echo "creating tasks..." task upload --file=/etc/batch/myfirstscript.js myfirstscript task upload --file=/etc/batch/mysecondscript.js mysecondscript task upload --file=/etc/batch/mythirdscript.js mythirdscript echo "list tasks" ls tasks