Chapter 5. Migrating applications to Data Grid 8 5.1. Marshalling in Data Grid 8 Marshalling capabilities are significantly refactored in Data Grid 8 to isolate internal objects and user objects. Because Data Grid now handles marshalling of internal classes, you no longer need to handle those internal classes when configuring marshallers with embedded or remote caches. 5.1.1. ProtoStream marshalling By default, Data Grid 8 uses the ProtoStream API to marshall data as Protocol Buffers, a language-neutral, backwards compatible format. Protobuf encoding is a schema-defined format that is now a default standard for many applications and allows greater flexibility when transcoding data in comparison with JBoss Marshalling, which was the default in Data Grid 7. Because the ProtoStream marshaller is based on the Protobuf format, Data Grid can convert to other encodings without first converting to a Java object. When using JBoss Marshalling, it is necessary to convert keys and values to Java objects before converting to any other format. As part of your migration to Data Grid 8, you should start using ProtoStream marshalling for your Java classes. From a high-level, to use the ProtoStream marshaller, you generate SerializationContextInitializer implementations with the ProtoStream processor. First, you add @Proto annotations to your Java classes and then use a ProtoStream processor that Data Grid provides to generate serialization contexts that contain: .proto schemas that provide a structured representation of your Java objects as Protobuf message types. Marshaller implementations to encode your Java objects to Protobuf format. Depending on whether you use embedded or remote caches, Data Grid can automatically register your SerializationContextInitializer implementations. Nested ProtoStream annotations Data Grid 8.2 upgrades to ProtoStream 4.4.0.Final, which requires migration in some cases. In versions, the ProtoStream API did not correctly nest message types with the result that the messages were generated as top-level only. If you have Protobuf-encoded entries in persistent cache stores, you should modify your Java classes so that ProtoStream annotations are at top-level. This ensures that the nesting in your persisted messages matches the nesting in your Java classes, otherwise data incompatibility issues can occur. For example, if you have nested Java classes such as the following: class OuterClass { class InnerClass { @ProtoField(1) int someMethod() { } } } You should adapt the classes so that InnerClass is no longer a child of OuterClass : class InnerClass { @ProtoField(1) int someMethod() { } } Marshalling with Data Grid Server You should use only Protobuf encoding for remote caches in combination with the ProtoStream marshaller for any custom types. Other marshaller implementations, such as JBoss marshalling, require you to use different cache encodings that are not compatible with the Data Grid CLI, Data Grid Console, or with Ickle queries. Cache stores and ProtoStream In Data Grid 7.x, data that you persist to a cache store is not compatible with the ProtoStream marshaller in Data Grid 8. You must use the StoreMigrator utility to migrate data from any Data Grid 7.x cache store to a Data Grid 8 cache store. 5.1.2. Alternative marshaller implementations Data Grid does provide alternative marshaller implementations to ProtoStream help ease migration from older versions. You should use those alternative marshallers only as an interim solution while you migrate to ProtoStream marshalling. Note For new projects Red Hat strongly recommends you use only ProtoStream marshalling to avoid any issues with future upgrades or migrations. Deserialization Allow List In keeping with Red Hat's commitment to using inclusive language the term "white list" has been changed to "allow list" for configuring serialization of your Java classes. Data Grid 8.1 <cache-container> <serialization> <white-list> <class>org.infinispan.test.data.Person</class> <regex>org.infinispan.test.data.*</regex> </white-list> </serialization> </cache-container> Data Grid 8.2 <cache-container> <serialization> <allow-list> <class>org.infinispan.test.data.Person</class> <regex>org.infinispan.test.data.*</regex> </allow-list> </serialization> </cache-container> JBoss marshalling In Data Grid 7, JBoss Marshalling is the default marshaller. In Data Grid 8, ProtoStream marshalling is the default. Note You should use JavaSerializationMarshaller instead of JBoss Marshalling if you have a client requirement to use Java serialization. If you must use JBoss Marshalling as a temporary solution during migration to Data Grid 8, do the following: Embedded caches Add the infinispan-jboss-marshalling dependency to your classpath. Configure Data Grid to use the JBossUserMarshaller , for example: <serialization marshaller="org.infinispan.jboss.marshalling.core.JBossUserMarshaller"/> Add your classes to the list of classes that Data Grid allows for deserialization. Remote caches Data Grid Server does not support JBoss Marshalling and the GenericJBossMarshaller is no longer automatically configured if the infinispan-jboss-marshalling module is on the classpath. You must configure Hot Rod Java clients to use JBoss Marshalling as follows: RemoteCacheManager .marshaller("org.infinispan.jboss.marshalling.commons.GenericJBossMarshaller"); hotrod-client.properties Additional resources Cache Encoding and Marshalling 5.2. Migrating applications to the AutoProtoSchemaBuilder annotation versions of Data Grid use the MessageMarshaller interface in the ProtoStream API to configure marshalling. Both the MessageMarshaller API and the ProtoSchemaBuilder annotation are deprecated as of Data Grid 8.1.1, which corresponds to ProtoStream 4.3.4. Using the MessageMarshaller interface involves either: Manually creating Protobuf schema. Adding the ProtoSchemaBuilder annotation to Java classes and then generating Protobuf schema. However, these techniques for configuring ProtoStream marshalling are not as efficient and reliable as the AutoProtoSchemaBuilder annotation, which is available starting with Data Grid 8.1.1. Simply add the AutoProtoSchemaBuilder annotation to your Java classes and to generate SerializationContextInitializer implementations that include Protobuf schema and associated marshallers. Red Hat recommends that you start using the AutoProtoSchemaBuilder annotation to get the best results from the ProtoStream marshaller. The following code examples demonstrate how you can migrate applications from the MessageMarshaller API to the AutoProtoSchemaBuilder annotation. 5.2.1. Basic MessageMarshaller implementation This example contains some fields that use non-default types. The text field has a different order and the fixed32 field conflicts with the generated Protobuf schema type because the code generator uses int type by default. SimpleEntry.java public class SimpleEntry { private String description; private Collection<String> text; private int intDefault; private Integer fixed32; // public Getter, Setter, equals and HashCode methods omitted for brevity } SimpleEntryMarshaller.java import org.infinispan.protostream.MessageMarshaller; public class SimpleEntryMarshaller implements MessageMarshaller<SimpleEntry> { @Override public void writeTo(ProtoStreamWriter writer, SimpleEntry testEntry) throws IOException { writer.writeString("description", testEntry.getDescription()); writer.writeInt("intDefault", testEntry.getIntDefault()); writer.writeInt("fix32", testEntry.getFixed32()); writer.writeCollection("text", testEntry.getText(), String.class); } @Override public SimpleEntry readFrom(MessageMarshaller.ProtoStreamReader reader) throws IOException { SimpleEntry x = new SimpleEntry(); x.setDescription(reader.readString("description")); x.setIntDefault(reader.readInt("intDefault")); x.setFixed32(reader.readInt("fix32")); x.setText(reader.readCollection("text", new LinkedList<String>(), String.class)); return x; } } Resulting Protobuf schema syntax = "proto2"; package example; message SimpleEntry { required string description = 1; optional int32 intDefault = 2; optional fixed32 fix32 = 3; repeated string text = 4; } Migrated to the AutoProtoSchemaBuilder annotation SimpleEntry.java import org.infinispan.protostream.annotations.ProtoField; import org.infinispan.protostream.descriptors.Type; public class SimpleEntry { private String description; private Collection<String> text; private int intDefault; private Integer fixed32; @ProtoField(number = 1) public String getDescription() {...} @ProtoField(number = 4, collectionImplementation = LinkedList.class) public Collection<String> getText() {...} @ProtoField(number = 2, defaultValue = "0") public int getIntDefault() {...} @ProtoField(number = 3, type = Type.FIXED32) public Integer getFixed32() {...} // public Getter, Setter, equals and HashCode methods and convenient constructors omitted for brevity } SimpleEntryInitializer.java import org.infinispan.protostream.GeneratedSchema; import org.infinispan.protostream.annotations.AutoProtoSchemaBuilder; @AutoProtoSchemaBuilder(includeClasses = { SimpleEntry.class }, schemaFileName = "simple.proto", schemaFilePath = "proto", schemaPackageName = "example") public interface SimpleEntryInitializer extends GeneratedSchema { } Important observations Field 2 is defined as int which the ProtoStream marshaller in versions did not check. Because the Java int field is not nullable the ProtoStream processor will fail. The Java int field must be required or initialized with a defaultValue . From a Java application perspective, the int field is initialized with "0" so you can use defaultValue without any impact as any put operation will set it. Change to required is not a problem from the stored data perspective if always present, but it might cause issues for different clients. Field 3 must be explicitly set to Type.FIXED32 for compatibility. The text collection must be set in the correct order for the resulting Protobuf schema. Important The order of the text collection in your Protobuf schema must be the same before and after migration. Likewise, you must set the fixed32 type during migration. If not, client applications might throw the following exception and fail to start: In other cases, you might observe incomplete or inaccurate results in your cached data. 5.2.2. MessageMarshaller implementation with custom types This section provides an example migration for a MessageMarshaller implementation that contains fields that ProtoStream does not natively handle. The following example uses the BigInteger class but applies to any class, even a Data Grid adapter or a custom class. Note The BigInteger class is immutable so does not have a no-argument constructor. CustomTypeEntry.java import java.math.BigInteger; public class CustomTypeEntry { final String description; final BigInteger bigInt; // public Getter, Setter, equals and HashCode methods and convenient constructors omitted for brevity } CustomTypeEntryMarshaller.java import org.infinispan.protostream.MessageMarshaller; public class CustomTypeEntryMarshaller implements MessageMarshaller<CustomTypeEntry> { @Override public void writeTo(ProtoStreamWriter writer, CustomTypeEntry testEntry) throws IOException { writer.writeString("description", testEntry.description); writer.writeString("bigInt", testEntry.bigInt.toString()); } @Override public CustomTypeEntry readFrom(MessageMarshaller.ProtoStreamReader reader) throws IOException { final String desc = reader.readString("description"); final BigInteger bInt = new BigInteger(reader.readString("bigInt")); return new CustomTypeEntry(desc, bInt); } } CustomTypeEntry.proto syntax = "proto2"; package example; message CustomTypeEntry { required string description = 1; required string bigInt = 2; } Migrated code with an adapter class You can use the ProtoAdapter annotation to marshall a CustomType class in a way that generates Protobuf schema that is compatible with Protobuf schema that you created with MessageMarshaller implementations. With this approach, you: Must not add annotations to the CustomTypeEntry class. Create a CustomTypeEntryAdapter class that uses the @ProtoAdapter annotation to control how the Protobuf schema and marshaller is generated. Include the CustomTypeEntryAdapter class with the @AutoProtoSchemaBuilder annotation. Note Because the AutoProtoSchemaBuilder annotation does not reference the CustomTypeEntry class, any annotations contained in that class are ignored. The following example shows the CustomTypeEntryAdapter class that contains ProtoStream annotations for the CustomTypeEntry class: CustomTypeEntryAdapter.java import java.math.BigInteger; import org.infinispan.protostream.annotations.ProtoAdapter; import org.infinispan.protostream.annotations.ProtoFactory; import org.infinispan.protostream.annotations.ProtoField; @ProtoAdapter(CustomTypeEntry.class) public class CustomTypeEntryAdapter { @ProtoFactory public CustomTypeEntry create(String description, String bigInt) { return new CustomTypeEntry(description, new BigInteger(bigInt)); } @ProtoField(number = 1, required = true) public String getDescription(CustomTypeEntry t) { return t.description; } @ProtoField(number = 2, required = true) public String getBigInt(CustomTypeEntry t) { return t.bigInt.toString(); } } The following example shows the SerializationContextInitializer with AutoProtoSchemaBuilder annotations that reference the CustomTypeEntryAdapter class: CustomTypeEntryInitializer.java import org.infinispan.protostream.GeneratedSchema; import org.infinispan.protostream.annotations.AutoProtoSchemaBuilder; @AutoProtoSchemaBuilder(includeClasses = { CustomTypeEntryAdapter.class }, schemaFileName = "custom.proto", schemaFilePath = "proto", schemaPackageName = "example") public interface CustomTypeAdapterInitializer extends GeneratedSchema { } Migrated code without an adapter class Instead of creating an adapter class, you can add ProtoStream annotations directly to the CustomTypeEntry class. Important In this example, the generated Protobuf schema is not compatible with data in caches that was added via the MessageMarshaller interface because the BigInteger is a separate message. Even if the adapter field writes the same String, it is not possible to unmarshall the data. The following example shows the CustomTypeEntry class that directly contains ProtoStream annotations: CustomTypeEntry.java import java.math.BigInteger; public class CustomTypeEntry { @ProtoField(number = 1) final String description; @ProtoField(number = 2) final BigInteger bigInt; @ProtoFactory public CustomTypeEntry(String description, BigInteger bigInt) { this.description = description; this.bigInt = bigInt; } // public Getter, Setter, equals and HashCode methods and convenient constructors omitted for brevity } The following example shows the SerializationContextInitializer with AutoProtoSchemaBuilder annotations that reference the CustomTypeEntry and BigIntegerAdapter classes: CustomTypeEntryInitializer.java import org.infinispan.protostream.GeneratedSchema; import org.infinispan.protostream.annotations.AutoProtoSchemaBuilder; import org.infinispan.protostream.types.java.math.BigIntegerAdapter; @AutoProtoSchemaBuilder(includeClasses = { CustomTypeEntry.class, BigIntegerAdapter.class }, schemaFileName = "customtype.proto", schemaFilePath = "proto", schemaPackageName = "example") public interface CustomTypeInitializer extends GeneratedSchema { } When you generate the Protobuf schema from the preceding SerializationContextInitializer implementation, it results in the following Protobuf schema: CustomTypeEntry.proto syntax = "proto2"; package example; message BigInteger { optional bytes bytes = 1; } message CustomTypeEntry { optional string description = 1; optional BigInteger bigInt = 2; }

Commands:
class OuterClass { class InnerClass { @ProtoField(1) int someMethod() { } } }
class InnerClass { @ProtoField(1) int someMethod() { } }
<cache-container> <serialization> <white-list> <class>org.infinispan.test.data.Person</class> <regex>org.infinispan.test.data.*</regex> </white-list> </serialization> </cache-container>
<cache-container> <serialization> <allow-list> <class>org.infinispan.test.data.Person</class> <regex>org.infinispan.test.data.*</regex> </allow-list> </serialization> </cache-container>
<serialization marshaller="org.infinispan.jboss.marshalling.core.JBossUserMarshaller"/>
.marshaller("org.infinispan.jboss.marshalling.commons.GenericJBossMarshaller");
infinispan.client.hotrod.marshaller = GenericJBossMarshaller
public class SimpleEntry { private String description; private Collection<String> text; private int intDefault; private Integer fixed32; // public Getter, Setter, equals and HashCode methods omitted for brevity }
import org.infinispan.protostream.MessageMarshaller; public class SimpleEntryMarshaller implements MessageMarshaller<SimpleEntry> { @Override public void writeTo(ProtoStreamWriter writer, SimpleEntry testEntry) throws IOException { writer.writeString("description", testEntry.getDescription()); writer.writeInt("intDefault", testEntry.getIntDefault()); writer.writeInt("fix32", testEntry.getFixed32()); writer.writeCollection("text", testEntry.getText(), String.class); } @Override public SimpleEntry readFrom(MessageMarshaller.ProtoStreamReader reader) throws IOException { SimpleEntry x = new SimpleEntry(); x.setDescription(reader.readString("description")); x.setIntDefault(reader.readInt("intDefault")); x.setFixed32(reader.readInt("fix32")); x.setText(reader.readCollection("text", new LinkedList<String>(), String.class)); return x; } }
syntax = "proto2"; package example; message SimpleEntry { required string description = 1; optional int32 intDefault = 2; optional fixed32 fix32 = 3; repeated string text = 4; }
import org.infinispan.protostream.annotations.ProtoField; import org.infinispan.protostream.descriptors.Type; public class SimpleEntry { private String description; private Collection<String> text; private int intDefault; private Integer fixed32; @ProtoField(number = 1) public String getDescription() {...} @ProtoField(number = 4, collectionImplementation = LinkedList.class) public Collection<String> getText() {...} @ProtoField(number = 2, defaultValue = "0") public int getIntDefault() {...} @ProtoField(number = 3, type = Type.FIXED32) public Integer getFixed32() {...} // public Getter, Setter, equals and HashCode methods and convenient constructors omitted for brevity }
import org.infinispan.protostream.GeneratedSchema; import org.infinispan.protostream.annotations.AutoProtoSchemaBuilder; @AutoProtoSchemaBuilder(includeClasses = { SimpleEntry.class }, schemaFileName = "simple.proto", schemaFilePath = "proto", schemaPackageName = "example") public interface SimpleEntryInitializer extends GeneratedSchema { }
Exception ( ISPN004034: Unable to unmarshall bytes )
import java.math.BigInteger; public class CustomTypeEntry { final String description; final BigInteger bigInt; // public Getter, Setter, equals and HashCode methods and convenient constructors omitted for brevity }
import org.infinispan.protostream.MessageMarshaller; public class CustomTypeEntryMarshaller implements MessageMarshaller<CustomTypeEntry> { @Override public void writeTo(ProtoStreamWriter writer, CustomTypeEntry testEntry) throws IOException { writer.writeString("description", testEntry.description); writer.writeString("bigInt", testEntry.bigInt.toString()); } @Override public CustomTypeEntry readFrom(MessageMarshaller.ProtoStreamReader reader) throws IOException { final String desc = reader.readString("description"); final BigInteger bInt = new BigInteger(reader.readString("bigInt")); return new CustomTypeEntry(desc, bInt); } }
syntax = "proto2"; package example; message CustomTypeEntry { required string description = 1; required string bigInt = 2; }
import java.math.BigInteger; import org.infinispan.protostream.annotations.ProtoAdapter; import org.infinispan.protostream.annotations.ProtoFactory; import org.infinispan.protostream.annotations.ProtoField; @ProtoAdapter(CustomTypeEntry.class) public class CustomTypeEntryAdapter { @ProtoFactory public CustomTypeEntry create(String description, String bigInt) { return new CustomTypeEntry(description, new BigInteger(bigInt)); } @ProtoField(number = 1, required = true) public String getDescription(CustomTypeEntry t) { return t.description; } @ProtoField(number = 2, required = true) public String getBigInt(CustomTypeEntry t) { return t.bigInt.toString(); } }
import org.infinispan.protostream.GeneratedSchema; import org.infinispan.protostream.annotations.AutoProtoSchemaBuilder; @AutoProtoSchemaBuilder(includeClasses = { CustomTypeEntryAdapter.class }, schemaFileName = "custom.proto", schemaFilePath = "proto", schemaPackageName = "example") public interface CustomTypeAdapterInitializer extends GeneratedSchema { }
import java.math.BigInteger; public class CustomTypeEntry { @ProtoField(number = 1) final String description; @ProtoField(number = 2) final BigInteger bigInt; @ProtoFactory public CustomTypeEntry(String description, BigInteger bigInt) { this.description = description; this.bigInt = bigInt; } // public Getter, Setter, equals and HashCode methods and convenient constructors omitted for brevity }
import org.infinispan.protostream.GeneratedSchema; import org.infinispan.protostream.annotations.AutoProtoSchemaBuilder; import org.infinispan.protostream.types.java.math.BigIntegerAdapter; @AutoProtoSchemaBuilder(includeClasses = { CustomTypeEntry.class, BigIntegerAdapter.class }, schemaFileName = "customtype.proto", schemaFilePath = "proto", schemaPackageName = "example") public interface CustomTypeInitializer extends GeneratedSchema { }
syntax = "proto2"; package example; message BigInteger { optional bytes bytes = 1; } message CustomTypeEntry { optional string description = 1; optional BigInteger bigInt = 2; }