Package: py4jps
What is py4jpsPy4jpsis a thin Python wrapper for theTheWorldAvatarproject. The code is heavily based on thepy4jpackage, which enables Python programs running in a Python interpreter to dynamically access Java objects in a Java Virtual Machine.RequirementsYou need Python >=3.5 to run the py4jps. You can install Python by going to the official Pythondownload pageYou also need to install aJava Runtime Environment version 11InstallationThese instructions will get you a copy of the project up and running on your local machine for development and testing purposes.Virtual environment setupIt is highly recommended to use avirtual environmentfor thepy4jpsinstallation. The virtual environment can be created as follows:(Windows)$ python -m venv py4jps_venv
$ py4jps_venv\Scripts\activate.bat
(py4jps_venv) $(Linux)$python3-mvenvpy4jps_venv
$sourcepy4jps_venv\bin\activate(py4jps_venv)$The above commands will create and activate the virtual environmentpy4jps_venvin the current directory.Installation via pipTo install thepy4jpssimply run the following command:(py4jps_venv)$pipinstallpy4jpsThe above command will install thepy4jpspackage including theJpsBaseLiblibrary that has been packaged together with the code.Installation from the version-controlled source (for developers)This type of installation is only for the developers. To installpy4jpsdirectly from its repository you need to first clone theTheWorldAvatarproject. Then simply navigate to theTheWorldAvatar\JPS_BASE_LIB\python_wrapperdirectory and execute the following commands:# build and install(py4jps_venv)$pipinstall.# or build for in-place development(py4jps_venv)$pipinstall-e.# or use the provided "install_wrapper.sh" convenience script,# that can create virtual environment and install the py4jps package in one go# build and install$install_wrapper.sh-v-i# or build for in-place development$install_wrapper.sh-v-i-eThe above commands will install thepy4jpspackage only. To include theJpsBaseLiblibrary, please see the next section.Installing additional java resourcesThepy4jpsproject can be easily extended to provide wrapping for otherTheWorldAvatarjava projects. To do that, all resource files and their dependencies must be collated into a single directory, with the main jar file located at the directory root. TheJpsBaseLib pom.xmlfile shows an example of how to do it with maven. If you wish to do it for otherTheWorldAvatarproject, simply copy themaven-jar-pluginandmaven-dependency-pluginplugins into the project pom file and add thenet.sf.py4jdependency. These changes will collate all the project dependencies into thetarget\libdirectory and include the requiredpy4jpackage in your project. Once that is done and the project is successfully built, thepy4jpsresource manager command-line utility,jpsrm, can be used to install and register the resource. Here are the steps:Copy the project main jar file and the entirelibfolder a temporary directory, e.g.,tmp_dir.Run the following command in the terminal (withpy4jps_venvvirtual environment activated):(py4jps_venv)$jpsrminstall<name><from>--jarJARFILEwherejpsrmis thepy4jpsresource manager,<name>is the name you wish to assign to your java resource. Note that the<name>MUST follow python's classes names requirements, otherwise it will be impossible to import it in python. The<from>argument is the absolute path to thetmp_dirwith all the java project files and the--jaroption is used to provide a name of the main jar file,JARFILE, to be used for communication.After the successful installation you can access the resource (classes and methods) inpy4jpsby simply importing it:frompy4jps.resourcesimportyourResourceNameRemove the no longer neededtmp_dirdirectory.To see alljpsrmcommands and options, runjpsrm -hin the terminal.Installing resources (for developers)These instructions explain how to install theJPS_BASE_LIBlibrary and are useful for developers working directly with theTheWorldAvatarrepository.Steps to install theJPS_BASE_LIBlibrary:Build theJPS_BASE_LIBproject.Go to theJPS_BASE_LIB/targetdirectory and copy the main project jar file,jps-base-lib.jar, and the entirelibfolder containing the project dependencies into a temporary directory, let us call ittmp_JpsBaseLib.Run the following command in the terminal (withpy4jps_venvvirtual environment activated):(py4jps_venv)$jpsrminstallJpsBaseLib<path_to_the_tmp_JpsBaseLib_directory>--jarjps-base-lib.jarAfter successful installation you can access theJpsBaseLibresource (classes and methods) inpy4jpsby simply importing it:frompy4jps.resourcesimportJpsBaseLibRemove the no longer neededtmp_JpsBaseLibdirectory.Note. Thejpsrmincludes a developer-only convenience commanddevinstallwhich will run all the above steps and install theJpsBaseLibresource. The command will only work if:theJPS_BASE_LIBproject is present and was successfully builtthepy4jpsproject was installed in a developer mode (-e option) inside theTheWorldAvatarrepositoryHere is how to execute thedevinstallcommand:# execute devinstall(py4jps_venv)$jpsrmdevinstallHow to useThis section explains how to effectively use thepy4jpspackage by following the best practices. Note thatpy4jpsis a very thin wrapper, thus it does not provide a high-level abstraction to the java classes. Therefore, the java project documentation must be consulted with to know which java objects to call to perform a desired task.Python-Java communication via Python's JPSGateway wrapper classThe Python-Java communication, handled by thepy4jpackage, happens via the local network sockets. The main interaction point between a Python and Java is provided by thepy4j JavaGatewayclass. This class is wrapped within thepy4jps JPSGatewayclass for convenience. TheJPSGatewayclass is the parent class for any installed java resources that one wishes to access.Although, it is not recommended to use it directly, theJPSGatewayclass can be useful in experimenting with different resources without installing them first, as it allows to change the resources at runtime. Please see theJPSGateway helpto learn more, whereas below, only the most important aspects are covered.Thepy4jps JPSGatewayclass can be imported to your project via the following commandfrom py4jps import JPSGateway. The class can be used in the following way:frompy4jpsimportJPSGatewayyourGateway=JPSGateway(resName=yourResName,jarPath=yourResJarPath,**JGkwargs)whereresNameis the name you wish to give to your resource,yourResJarPathis the absolute path to the resource main jar file and**JGkwargsis a dictionary of any optionalargument: valuepairs one wishes to pass to thepy4j JavaGatewayconstructor. Note that if you wish to access an already installed resource through theJPSGateway(not recommended), then thejarPathargument can be omitted as it can be looked up by the resource name in the resource registry. Also, note that some of thepy4j JavaGatewayconstructor arguments are not allowed or should be passed to thepy4j launch_gatewaymethod instead. If that is the case, the code will print a warning message which shows how to set the desired argument correctly. Please also note that according to thepy4jdocumentation, thegateway_parametersargument to thepy4j JavaGatewayconstructor must have the type of thepy4j GatewayParametersobject. However, to make it easy for thepy4jpsusers, this argument can be passed as a dictionary which then is automatically converted into thepy4j GatewayParametersobject. Here is an example call to theJPSGatewayconstructor with extrapy4j JavaGatewayarguments:frompy4jpsimportJPSGatewayyourGateway=JPSGateway(**{'eager_load':True,'gateway_parameters':{'auto_field':True}})Please refer to thepy4j documentationfor the description of all the possible arguments. The most important and useful settings are set by default in thepy4jps JPSGatewayconstructor so a user hardly ever need to pass any arguments in that call. If required, however, thepy4jps JPSGatewayconstructor defaults can be overwritten by simply passing their new values. Please also note that theJPSGatewayconstructor only instantiates theJPSGatewayobject, and it DOES NOT instantiate thepy4j JavaGateway, which only happens in thepy4jps launchGatewaymethod explained in detail below.JPSGateway.launchGateway wrapper methodTo start the communication with the Java side, please run the following method on your gateway object:yourGateway.launchGateway(**LGkwargs)where the**LGkwargsargument represents a dictionary of any optionalargument: valuepairs one wishes to pass to thepy4j launch_gatewaymethod. Please refer to the method documentation for the description of all the possible arguments. The most important and useful settings are set by default in thepy4jps launchGatewaymethod so a user hardly ever need to pass any arguments in that call. If required, however, thepy4jps launchGatewaymethod defaults can be overwritten by simply passing their new values.The code below shows an examplepy4jps launchGatewaymethod call with custom arguments.# The file paths are exemplary only, please change# according to your needsstdout_file_handle=open('D:\\stdout_file.out','w')sterr_file_handle=open('D:\\stderr_file.out','w')yourGateway.launchGateway(**{'redirect_stdout':stdout_file_handle,'redirect_stderr':sterr_file_handle})The example above shows how to set the optionalpy4j JavaGateway.launch_gatewayarguments:redirect_stdoutandredirect_stderrthat might be useful for debugging purposes.Accessing Java classes and methodsOnce your resource gateway is launched you can access anypublicclasses and methods. Here is an example on how to access theStoreRouterclass:StoreRouter=yourGateway.gateway.jvm.uk.ac.cam.cares.jps.base.query.StoreRouterAs can be seen, the gateway's jvm class serves as an entry point to any java classes and objects you wish to access. Simply provide the fully qualified name of the resource to access it. Theuk.ac.cam.cares.jps.base.query.StoreRouterpath is for example purposes only.Please note that though accessing objects via their fully qualified names is possible, it is not recommended. A much simpler way exists, through the java import statements explained in theJVM module viewssection.It is also important to understand the difference in accessing thejava static and non-static methods. The former can be done without the object instantiation, e.g.StoreRouter=yourGateway.gateway.jvm.uk.ac.cam.cares.jps.base.query.StoreRouterStoreRouter.getStoreClient('http://kb/ontokin',True,False)here thestatic getStoreClientmethod can be called without theStoreRouterinstantiation. The non-static methods can only be accessed once the object is instantiated, e.g# create a FileUtil instance to access its non static getDirectoryFiles methodFileUtil=jpsBaseLib_view.FileUtil()# call the getDirectoryFiles methodoutput=FileUtil.getDirectoryFiles(method_arguments)Resource gatewaysThe resource gateway class is automatically created upon the resource installation. The class name is the same as the name of the resource given during the installation. Furthermore,each resource gateway class inherits from thepy4jps JPSGatewayclass. Therefore,one should ideally only work with the resource classes rather than with the parent JPSGateway class. It is also recommended to have only one gateway object in your Python application per java resource you wish to access. This can be easily achieved by instantiating and starting the resource gateway objects in a single module and then to import these objects instances to any other module that requires it.To give a simple example, after installing theJpsBaseLibresource, one can import and use it in the following way:frompy4jps.resourcesimportJpsBaseLibjpsBaseLibGW=JpsBaseLib(**JGkwargs)jpsBaseLibGW.launchGateway(**LGkwargs)Do note that compared to theJPSGatewayclass theJpsBaseLibconstructor call neither accepts the resource name nor the resource jar path as arguments. This ensures that the resource is properly encapsulated.JVM module viewsPy4jallows importing Java packages so that you do not have to type the fully qualified names of the classes you want to access. However, any such import statement is global in Python, e.g., thejvminstance can be shared across multiple Python modules. Therefore, the recommended way is to use one Java Virtual Machine View (JVMView) per Python module (file). A convenience method has been added to thepy4jpsgateway class that allows to create the module views (seepy4jdocumentation for further details on module views):# create the module viewmoduleViewInstance=resGateway.createModuleView()where theresGatewayis the gateway of a resource we wish to create the module view for. Once the module view is created, it can be used to import the desired java classes using theimportPackagesmethod, e.g:# import classes into the module viewresGateway.importPackages(moduleViewInstance,"uk.ac.cam.cares.jps.base.query.*")# once classes are imported, they can be accesses as followsStoreRouter=moduleViewInstance.StoreRouterThe module view should be created for each Python module in your application that requires access to the java resource, followed by any desired import statements. The name of the module view is arbitrary, though it is recommended to name it after your parent resource to avoid confusion.Python-Java objects translationIt is important to understand that calling the Java methods from Python requires a bit of attention to the type of arguments on the Java side. The simplest case is when the Java method arguments are of any primitive type (e.g. int, float, String, etc..). Then on the Python side, it is enough to pass Python primitives (e.g 5 - int, 3.4 - float, 'a_string', etc..) without even declaring their types. Primitives are automatically converted between Python and Java. Another case is when the Java method arguments are of any collections type (e.g., Array, java.util.List, java.util.Set, etc..). If thepy4j JavaGatewayoptionauto_convertis set to True, then the Python - Java collections objects translation happens automatically. Please see thepy4j documentationfor a detailed explanation of what is converted into what. As explained in thePython-Java communication via Python's JPSGateway wrapper classsection, inpy4jpstheauto_convertoption is set to True by default so a user does not need to worry about it. Here are some code examples:# example of calling a Java aMethod1 with an int argumentmoduleViewInstance.aMethod1(5)# example of calling a Java aMethod2 with a float argumentmoduleViewInstance.aMethod2(3.4)# example of calling a Java aMethod3 with a string argumentmoduleViewInstance.aMethod3('a_string')# example of calling a Java aMethod4 with a list of strings argument# this only works because the auto_convert option is enabled by default in py4jpsmoduleViewInstance.aMethod4(['a_string'])However, there could be cases where a Java object of a specific type needs to be created for the method call. As an example, theFileUtilclass in theJpsBaseLibresource has a method calledgetDirectoryFiles. The method simply returns a list of files in a directory that matches the provided file extensions. Here is its Java interface:publicList<File>getDirectoryFiles(Filefolder,List<String>fileExtensions)As can be seen, the first argument is of typeFile, and the second is of typeList<String>. Additionally, the method is not static so theFileUtilobject needs to be instantiated first to be able to access this method. Since the JavaFileobject needs to be created on the Python side so that the method can be called with the correct arguments, the question arises on how to do it. Now it is important to understand that the launched resource gateway allows you to not only access the resource classes and methods, but it also allows you to access any default Java classes and methods. This can be done in two ways: either via theresGateway.gateway.jvmfield (not recommended) or via the created resourcemoduleViewInstance(recommended). Here is an example:# create and launch the jpsBaseLib gatewayjpsBaseLibGW=JpsBaseLib()jpsBaseLibGW.launchGateway()# create the separate JVM viewjpsBaseLibGW_view=jpsBaseLibGW.createModuleView()# import required Java packages into the created JVM viewjpsBaseLibGW.importPackages(jpsBaseLibGW_view,'uk.ac.cam.cares.jps.base.util.*')# create a FileUtil instance to access its non-static methods# compare it with the StoreRouter example, where there was no need to add# () brackets to the router, so we were only retrieving the StoreRouter classFileUtil=jpsBaseLibGW_view.FileUtil()# create the required `File folder` argument to be passed to the `getDirectoryFiles` method# option 1 - NOT RECOMMENDED# not recommended use of a gateway.jvm class to access the default java classes and methods# here we are instantiating the java.io.File classjavaFolder=jpsBaseLibGW.gateway.jvm.java.io.File('D:\\my_dir')# option 2 - RECOMMENDED# recommended use of JVM view to access default java classes and methodsjavaFolder=jpsBaseLibGW_view.java.io.File('D:\\my_dir')# call the getDirectoryFiles# Note that the `List<String> fileExtensions` argument can be passed# as a simple Python list of strings thanks to the enabled auto conversionfileListArray=FileUtil.getDirectoryFiles(javaFolder,[".txt"])# The returned object is of Java List<File> type. So, one needs to know a bit# of Java to extract information. Here is an example:files_list=[]foriinrange(fileListArray.size()):files_list.append(fileListArray.get(i).toString())# An extra example# If the Java method requires as an input a list of Java objects,# where the objects are not primitives e.g a list of file objects# that is returned as an output from the getDirectoryFiles call.# Then one can simply create such list as follows:javaFileObj=jpsBaseLibGW_view.java.io.File(path_to_a_folder)# this creates a Python List which stores a javaFileObj insidePythonList=[javaFileObj]# or# this creates a javaArray which stores the javaFileObj inside# this example, however, is not needed thanks to the auto conversion# so, one can simply pass the PythonList created above to the methodjavaArray=jpsBaseLibGW_view.java.util.ArrayList()javaArray.append(javaFileObj)Example code:Putting the above recommendations together leads to the following project layout:Example project source files layout.
├── ...                         # other project files (README, LICENSE, etc..)
└── projectname                 # project source files
    ├── kgoperations            # module that deals with the KG communication
    │    ├── __init__.py
    │    ├── query.py           # base query function
    │    ├── ...                # any other kg-related file modules
    │    └── gateways.py        # python-java communication gateways
    ├── app1                    # application module 1
    │   ├── __init__.py
    │   └── app.py
    ├── app2                    # application module 2, etc..
    │   ├── __init__.py
    │   └── app.py
    └── main.py                 # application main entry pointExample codeFile: projectname.kgoperations.gateways.py# The purpose of this module is to create and start resource# gateway objects to be used in all of your other modules#============================================================frompy4jps.resourcesimportJpsBaseLib# jpsBaseLib resource gateway# you may also wish to pass any **JGkwargsjpsBaseLibGW=JpsBaseLib()# you may also wish to pass any **LGkwargsjpsBaseLibGW.launchGateway()# gateways instances for any other resources...#============================================================File: projectname.kgoperations.query.py# The purpose of this module is to provide a standard api# interface for querying the KG. Note that the actual# implementation is dependent on the version of the jps-base-lib# installed. Always consult the jps-base-lib docs if in doubt.#============================================================fromprojectname.kgoperations.gatewaysimportjpsBaseLibGWimportjson# create a JVM module view and use it to import the required java classesjpsBaseLib_view=jpsBaseLibGW.createModuleView()jpsBaseLibGW.importPackages(jpsBaseLib_view,"uk.ac.cam.cares.jps.base.query.*")defquerykg(sparqlEndPoint=None,queryStr=None):# perform an example sparql query,# see the jps-base-lib docs for further detailsStoreRouter=jpsBaseLib_view.StoreRouterStoreClient=StoreRouter.getStoreClient(sparqlEndPoint,True,False)response=json.loads(str(StoreClient.executeQuery(queryStr)))returnresponse#============================================================File: main.py# The purpose of this file is to have a single entry point# to your application#============================================================fromprojectname.app1.appimportdoTask1fromprojectname.app2.appimportdoTask2response1=doTask1()response2=doTask2()File: app1.app.py# The purpose of this module is to group certain tasks you# wish to perform on the KG#============================================================fromprojectname.kgoperations.queryimportquerykg# create a JVM module view and use it to import the required java classesjpsBaseLib_view=jpsBaseLibGW.createModuleView()jpsBaseLibGW.importPackages(jpsBaseLib_view,"uk.ac.cam.cares.jps.base.query.*")# this function shows how to do a simple KG querydefdoTask1():queryStr="""PREFIX ontokin: <http://www.theworldavatar.com/ontology/ontokin/OntoKin.owl#>PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>	SELECT ?mechanismIRIWHERE	{ ?mechanismIRI rdf:type ontokin:ReactionMechanism .} LIMIT 10"""returnquerykg(sparqlEndPoint='http://kb/ontokin',queryStr=queryStr)File: app2.app.py# The purpose of this module is to group certain tasks you# wish to perform on the KG#============================================================fromprojectname.kgoperations.gatewaysimportjpsBaseLibGW# create a JVM module view and use it to import the required java classesjpsBaseLib_view=jpsBaseLibGW.createModuleView()jpsBaseLibGW.importPackages(jpsBaseLib_view,'uk.ac.cam.cares.jps.base.util.*')# this function shows how to use jps-base-lib file util object# to read a local file.defdoTask2():FileUtil=jpsBaseLib_view.FileUtil# any paths passed to the jps-base-lib FileUtil object should be# absolute paths. Here the FileUtil object does not need to be# instantiated as the readFileLocally method is staticreturnFileUtil.readFileLocally('<your_absolute_file_path>')#============================================================Packageagentlogging:As ofpy4jps==1.0.29,agentlogging, which originally placedhere, is also packaged and released as part of this python wrapper. One can import and use as below:frompy4jpsimportagentloggingdev_logger=agentlogging.get_logger("dev")dev_logger.debug("This is a DEBUG statement")dev_logger.info("This is an INFO statement")prod_logger=agentlogging.get_logger("prod")prod_logger.debug("This is a DEBUG statement")prod_logger.info("This is an INFO statement")For more details, see theLoggingpage on the Wiki.Note to developersThe py4jps aim is to provide Python access to theTheWorldAvatarproject classes and methods. However, it is important to understand that not allTheWorldAvatarclasses can be accessed. Namely,any servlet depending classes can not be instantiated in Python without running the TomCat server first. Since this has not been tested, it is not guaranteed that running the TomCat server would fix the problem. However, this should not be an issue for thepy4jpsusers, given that the main purpose of the wrapper is to use the client-sideTheWorldAvatarcode to perform KG queries or updates. In other words, it is not thepy4jpspurpose to develop the server-side code, which should happen in Java.Package releaseNOTE: Before making the package release, please remove all sub-folders andresources_registry.jsonfile inpython_wrapper/py4jps/resourcesfolder to prevent incorrect packing of Java resources. For more information, please refer to thisissue.Maintainers who package and publish the most up-to-date codes from thedevelopbranch handle the distribution of package py4jps on PyPI and Test-PyPI. If you want to release the package independently, i.e. become a maintainer, please contact the repository's administrator to indicate your interest.The release procedure is currently semi-automated and requires a few items:Your Test-PyPI and PyPI account and passwordThe version number x.x.x for the releaseClone ofTheWorldAvatarrepository on your local machinePlease create and checkout to a new branch from the newestdevelopbranch once these details are ready. The release process can then be started by using the commands below, depending on the operating system you're using. (REMEMBER TO CHANGE THE CORRECT VALUES IN THE COMMANDS BELOW!)(Windows)$ cd \absolute_path_to\TheWorldAvatar\JPS_BASE_LIB\python_wrapper
$ release_py4jps_to_pypi.sh -v x.x.x(Linux)$cd/absolute_path_to/TheWorldAvatar/JPS_BASE_LIB/python_wrapper
$./release_py4jps_to_pypi.sh-vx.x.xPlease follow the instructions presented in the console once the process has begun. If everything goes well, change the version number inJPS_BASE_LIB/python_wrapper/release_py4jps_to_pypi.shto the one you used for the script release.echo"./release_py4jps_to_pypi.sh -v 1.0.15   - release version 1.0.15"The changes mentioned above should then be committed with the changes performed automatically during the release process, specifically in python scriptJPS_BASE_LIB/python_wrapper/py4jps/__init__.py__version__ = "1.0.15"andJPS_BASE_LIB/python_wrapper/setup.pyversion='1.0.15',Finally, make a Pull Request for the branch and merge it back into thedevelopbranch.AuthorsDaniel Nurkowski
Jiaru Bai