Package: javaobj-py3
javaobj-py3python-javaobjis a python library that provides functions for reading and
writing (writing is WIP currently) Java objects serialized or will be
deserialized byObjectOutputStream. This form of object representation is a
standard data interchange format in Java world.Thejavaobjmodule exposes an API familiar to users of the standard librarymarshal,pickleandjsonmodules.About this repositoryThis project is a fork ofpython-javaobjby Volodymyr Buell, originally fromGoogle Codeand now hosted onGitHub.This fork intends to work both on Python 2.7 and Python 3.4+.Compatibility WarningsNew implementation of the parserImplementationsVersionv1,v20.4.0+Since version 0.4.0, two implementations of the parser are available:v1: theclassicimplementation ofjavaobj, with a work in progress
implementation of a writer.v2: thenewimplementation, which is a port of the Java projectjdeserialize,
with support of the object transformer (with a new API) and of thenumpyarrays loading.You can use thev1parser to ensure that the behaviour of your scripts
doesn't change and to keep the ability to write down files.You can use thev2parser for new developmentswhich won't require marshallingand as afallbackif thev1fails to parse a file.Object transformers V1ImplementationsVersionv10.2.0+As of version 0.2.0, the notion ofobject transformerfrom the original
project as been replaced by anobject creator.Theobject creatoris called before the deserialization.
This allows to store the reference of the converted object before deserializing
it, and avoids a mismatch between the referenced object and the transformed one.Object transformers V2ImplementationsVersionv20.4.0+Thev2implementation provides a new API for the object transformers.
Please look at theUsage (V2)section in this file.Bytes arraysImplementationsVersionv10.2.3+As of version 0.2.3, bytes arrays are loaded as abytesobject instead of
an array of integers.Custom TransformerImplementationsVersionv20.4.2+A new transformer API has been proposed to handle objects written with a custom
Java writer.
You can find a sample usage in theCustom Transformersection in this file.FeaturesJava object instance un-marshallingJava classes un-marshallingPrimitive values un-marshallingAutomatic conversion of Java Collections to python ones
(HashMap=>dict,ArrayList=>list, etc.)Basic marshalling of simple Java objects (v1implementation only)Automatically uncompresses GZipped filesRequirementsPython >= 2.7 or Python >= 3.4enum34andtypingwhen using Python <= 3.4 (installable withpip)Maven 2+ (for building test data of serialized objects.
You can skip it if you do not plan to runtests.py)Usage (V1 implementation)Un-marshalling of Java serialised object:importjavaobjwithopen("obj5.ser","rb")asfd:jobj=fd.read()pobj=javaobj.loads(jobj)print(pobj)Or, you can useJavaObjectUnmarshallerobject directly:importjavaobjwithopen("objCollections.ser","rb")asfd:marshaller=javaobj.JavaObjectUnmarshaller(fd)pobj=marshaller.readObject()print(pobj.value,"should be",17)print(pobj.next,"should be",True)pobj=marshaller.readObject()Note:The objects and methods provided byjavaobjmodule are shortcuts
to thejavaobj.v1package, for Compatibility purpose.
It isrecommendedto explicitly import methods and classes from thev1(orv2) package when writing new code, in order to be sure that your code
won't need import updates in the future.Usage (V2 implementation)The following methods are provided by thejavaobj.v2package:load(fd, *transformers, use_numpy_arrays=False):
Parses the content of the given file descriptor, opened in binary mode (rb).
The method accepts a list of custom object transformers. The default object
transformer is always added to the list.Theuse_numpy_arraysflag indicates that the arrays of primitive type
elements must be loaded usingnumpy(if available) instead of using the
standard parsing technic.loads(bytes, *transformers, use_numpy_arrays=False):
This the a shortcut to theload()method, providing it the binary data
using aBytesIOobject.Note:The V2 parser doesn't have the marshalling capability.Sample usage:importjavaobj.v2asjavaobjwithopen("obj5.ser","rb")asfd:pobj=javaobj.load(fd)print(pobj.dump())Object TransformerAn object transformer can be called during the parsing of a Java object
instance or while loading an array.The Java object instance parsing works in two main steps:The transformer is called to create an instance of a bean that inheritsJavaInstance.The latter bean is then called:When the object is written with a custom block dataAfter the fields and annotations have been parsed, to update the content
of the Python bean.Here is an example for a JavaHashMapobject. You can look at the code of
thejavaobj.v2.transformermodule to see the whole implementation.classJavaMap(dict,javaobj.v2.beans.JavaInstance):"""Inherits from dict for Python usage, JavaInstance for parsing purpose"""def__init__(self):# Don't forget to call both constructorsdict.__init__(self)JavaInstance.__init__(self)defload_from_blockdata(self,parser,reader,indent=0):"""Reads content stored in a block data.This method is called only if the class description has both the`SC_EXTERNALIZABLE` and `SC_BLOCK_DATA` flags set.The stream parsing will stop and fail if this method returns False.:param parser: The JavaStreamParser in use:param reader: The underlying data stream reader:param indent: Indentation to use in logs:return: True on success, False on error"""# This kind of class is not supposed to have the SC_BLOCK_DATA flag setreturnFalsedefload_from_instance(self,indent=0):# type: (int) -> bool"""Load content from the parsed instance object.This method is called after the block data (if any), the fields andthe annotations have been loaded.:param indent: Indentation to use while logging:return: True on success (currently ignored)"""# Maps have their content in their annotationsforcd,annotationsinself.annotations.items():# Annotations are associated to their definition classifcd.name=="java.util.HashMap":# We are in the annotation created by the handled class# Group annotation elements 2 by 2# (storage is: key, value, key, value, ...)args=[iter(annotations[1:])]*2forkey,valueinzip(*args):self[key]=value# Job donereturnTrue# Couldn't load the datareturnFalseclassMapObjectTransformer(javaobj.v2.api.ObjectTransformer):"""Creates a JavaInstance object with custom loading methods for theclasses it can handle"""defcreate_instance(self,classdesc):# type: (JavaClassDesc) -> Optional[JavaInstance]"""Transforms a parsed Java object into a Python object:param classdesc: The description of a Java class:return: The Python form of the object, or the original JavaObject"""ifclassdesc.name=="java.util.HashMap":# We can handle this class descriptionreturnJavaMap()else:# Return None if the class is not handledreturnNoneCustom Object TransformerThe custom transformer is called when the class is not handled by the default
object transformer.
A custom object transformer still inherits from theObjectTransformerclass,
but it also implements theload_custom_writeObjectmethod.The sample given here is used in the unit tests.Java sampleOn the Java side, we create various classes and write them as we wish:classCustomClassimplementsSerializable{privatestaticfinallongserialVersionUID=1;publicvoidstart(ObjectOutputStreamout)throwsException{this.writeObject(out);}privatevoidwriteObject(ObjectOutputStreamout)throwsIOException{CustomWritercustom=newCustomWriter(42);out.writeObject(custom);out.flush();}}classRandomChildextendsRandom{privatestaticfinallongserialVersionUID=1;privateintnum=1;privatedoubledoub=4.5;RandomChild(intseed){super(seed);}}classCustomWriterimplementsSerializable{protectedRandomChildcustom_obj;CustomWriter(intseed){custom_obj=newRandomChild(seed);}privatestaticfinallongserialVersionUID=1;privatestaticfinalintCURRENT_SERIAL_VERSION=0;privatevoidwriteObject(ObjectOutputStreamout)throwsIOException{out.writeInt(CURRENT_SERIAL_VERSION);out.writeObject(custom_obj);}}An here is a sample writing of that kind of object:ObjectOutputStreamoos=newObjectOutputStream(newFileOutputStream("custom_objects.ser"));CustomClasswriter=newCustomClass();writer.start(oos);oos.flush();oos.close();Python sampleOn the Python side, the first step is to define the custom transformers.
They are children of thejavaobj.v2.transformers.ObjectTransformerclass.classBaseTransformer(javaobj.v2.transformers.ObjectTransformer):"""Creates a JavaInstance object with custom loading methods for theclasses it can handle"""def__init__(self,handled_classes=None):self.instance=Noneself.handled_classes=handled_classesor{}defcreate_instance(self,classdesc):"""Transforms a parsed Java object into a Python object:param classdesc: The description of a Java class:return: The Python form of the object, or the original JavaObject"""ifclassdesc.nameinself.handled_classes:self.instance=self.handled_classes[classdesc.name]()returnself.instancereturnNoneclassRandomChildTransformer(BaseTransformer):def__init__(self):super(RandomChildTransformer,self).__init__({"RandomChild":RandomChildInstance})classCustomWriterTransformer(BaseTransformer):def__init__(self):super(CustomWriterTransformer,self).__init__({"CustomWriter":CustomWriterInstance})classJavaRandomTransformer(BaseTransformer):def__init__(self):super(JavaRandomTransformer,self).__init__()self.name="java.util.Random"self.field_names=["haveNextNextGaussian","nextNextGaussian","seed"]self.field_types=[javaobj.v2.beans.FieldType.BOOLEAN,javaobj.v2.beans.FieldType.DOUBLE,javaobj.v2.beans.FieldType.LONG,]defload_custom_writeObject(self,parser,reader,name):ifname!=self.name:returnNonefields=[]values=[]forf_name,f_typeinzip(self.field_names,self.field_types):values.append(parser._read_field_value(f_type))fields.append(javaobj.beans.JavaField(f_type,f_name))class_desc=javaobj.beans.JavaClassDesc(javaobj.beans.ClassDescType.NORMALCLASS)class_desc.name=self.nameclass_desc.desc_flags=javaobj.beans.ClassDataType.EXTERNAL_CONTENTSclass_desc.fields=fieldsclass_desc.field_data=valuesreturnclass_descSecond step is defining the representation of the instances, where the real
object loading occurs. Those classes inherit fromjavaobj.v2.beans.JavaInstance.classCustomWriterInstance(javaobj.v2.beans.JavaInstance):def__init__(self):javaobj.v2.beans.JavaInstance.__init__(self)defload_from_instance(self):"""Updates the content of this instancefrom its parsed fields and annotations:return: True on success, False on error"""ifself.classdescandself.classdescinself.annotations:# Here, we known there is something written before the fields,# even if it's not declared in the class descriptionfields=["int_not_in_fields"]+self.classdesc.fields_namesraw_data=self.annotations[self.classdesc]int_not_in_fields=struct.unpack(">i",BytesIO(raw_data[0].data).read(4))[0]custom_obj=raw_data[1]values=[int_not_in_fields,custom_obj]self.field_data=dict(zip(fields,values))returnTruereturnFalseclassRandomChildInstance(javaobj.v2.beans.JavaInstance):defload_from_instance(self):"""Updates the content of this instancefrom its parsed fields and annotations:return: True on success, False on error"""ifself.classdescandself.classdescinself.field_data:fields=self.classdesc.fields_namesvalues=[self.field_data[self.classdesc][self.classdesc.fields[i]]foriinrange(len(fields))]self.field_data=dict(zip(fields,values))if(self.classdesc.super_classandself.classdesc.super_classinself.annotations):super_class=self.annotations[self.classdesc.super_class][0]self.annotations=dict(zip(super_class.fields_names,super_class.field_data))returnTruereturnFalseFinally we can use the transformers in the loading process.
Note that even if it is not explicitly given, theDefaultObjectTransformerwill be also be used, as it is added automatically byjavaobjif it is
missing from the given list.# Load the object using those transformerstransformers=[CustomWriterTransformer(),RandomChildTransformer(),JavaRandomTransformer()]pobj=javaobj.loads("custom_objects.ser",*transformers)# Here we show a field that isn't visible from the class description# The field belongs to the class but it's not serialized by default because# it's static. See: https://stackoverflow.com/a/16477421/12621168print(pobj.field_data["int_not_in_fields"])