Package: ghidra-pipe
Ghidra Pipe: Teleport Python code from CPython to JythonWhat is Ghidra PipeInstallationStart/Stop the Pipe ServerSetting Custom port and hostname for the PipeServer SideClient SideTeleport Python Code from CPython 3 to JythonCodeFunctionClassStandard Output and Error RedirectionRemote ExceptionFull Usage ExampleCustom Pipe Communication RoutinesCustom Binary Communication ExampleCustom JSON Communication ExampleReach Existing Remote Object from Everywhere Through ProxyProxy Remote Object Tracking InformationFile Copy Through PipePipe Server JSON RPC InterfaceDevelopmentFAQWhy an Another ToolWhy the Pipe Server use Java SocketWhy Ghidra-Pipe no Proxify Ghidra API in the Client Side Global NamespaceWhat is Ghidra PipeGhidra-Pipe provides various ways to interface custom reverse engineering tools with Ghidra environment and its Ghidra Jython API. On one side, a Ghidra Python script run a pipe server which exposes several services though RPC methods. On the other hand, Ghidra-Pipe provides a pipe client API to access these services. The pipe client and the server communicate through classic TCP socket with the JSON RPC V2 protocol. So it is possible to implement a custom pipe client in any language to use the pipe server services. The network traffic is non encrypted (on local by default), since this tool is for research purpose no effort have been done for encrypted communication.Summary of features:Teleport Python code from CPython 3 to Jython:Remote Python code execution;Remote Python functions declaration;Remote Python class declaration;Function call, class usage and object creation/usage through proxy.Custom pipe communication:Remote custom communication routine declaration;Custom communication channel opening through proxy;Helpers for binary/JSON communication.File copy from local to remote or from remote to local.JSON RPC V2 interface:Python code execution;Python function execution;Python object creation;Python object attribute getter and setter;Register/call custom Python communicator;File copy from local to remote or from remote to local.Lightweight code (< 1000 loc), zero dependency and easy to modify to feet your need.InstallationInstall the Ghidra-Pipe Python package from the Python Package Index (PyPI).$ pip install ghidra-pipeCopy the pipe server plugins to a custom Ghidra plugins directory.$ ghidra-pipe --plugin-install /path/to/ghidra_pluginsThe plugin installer copy the following Python script in the ghidra-pipe directory.$ tree ghidra-pipe/
ghidra-pipe/
├── pipe_default_conf.py
├── pipe_server.py
├── plugin_ghidra_pipe_server_start.py
└── plugin_ghidra_pipe_server_stop.py

0 directories, 4 filesThis tool has been tested on these platforms and configuration, but can probably work on variant.OSPythonJythonJava RuntimeGNU/linux3.10.52.7.2OpenJDK 11.0.15+10GNU/linux3.8.02.7.2OpenJDK 11.0.15+10GNU/linux3.7.02.7.2OpenJDK 11.0.15+10GNU/linux3.6.02.7.2OpenJDK 11.0.15+10GNU/linux3.5.42.7.2OpenJDK 11.0.15+10Windows x643.10.52.7.2adoptopenjdk jdk-17.0.3.7-hotspotWindows x643.8.12.7.2adoptopenjdk jdk-17.0.3.7-hotspotWindows x643.7.12.7.2adoptopenjdk jdk-17.0.3.7-hotspotWindows x643.6.02.7.2adoptopenjdk jdk-17.0.3.7-hotspotWindows x643.5.32.7.2adoptopenjdk jdk-17.0.3.7-hotspotStart/Stop the Pipe ServerStart the pipe server via the Ghidra GUI, open the script manager window inWindow > Script Manager, and  run  the  scriptplugin_ghidra_pipe_server_start.pylocalised in theghidra_pipedirectory.Stop the pipe server via the Ghidra GUI, open the script manager window inWindow > Script Manager, and  run  the  scriptplugin_ghidra_pipe_server_stop.pylocalised in theghidra_pipedirectory.It is also possible to stop the pipe server via the pipe client API.>>> from ghidra_pipe import PipeClient
>>> PipeClient().server_remote_shutdown()Setting Custom port and hostname for the PipeServer SideBy default, the pipe server listen for incoming connection on localhost and TCP port 5098. These parameters are configurable through the configuration file localised in the Ghidra-Pipe plugins directory inghidra_pipe/pipe_default_conf.pywith the variablesPIPE_IPandPIPE_PORT(before Python module import).Client SideBy default, all pipe client methods initiate connection on localhost and TCP port 5098. These parameters are configurable globally via the environment variablesPIPE_IPandPIPE_PORT(before Python module import). Otherwise, thePipeClientclass accept the optional keyword argumentsip_addressandport.>>> from ghidra_pipe import PipeClient
>>> pipe_client = PipeClient(ip_address='192.168.1.35', port=5090)Teleport Python Code from CPython 3 to JythonTeleport Python code from CPython 3 to Jython requires that the teleported code is compatible with CPython 3 and the remote version of Jython (2/3).CodeThePipeClient.execmethod allows for remote Python code execution. The method take Python source code as argument which will be executed on the remote global namespace of the pipe server via a classic exec. Note that this Python source code does not need to be compatible Python 3 since it will be never evaluate locally. Stdout and stderr of the code executed remotely is forwarded locally and can be captured and returned by the method if thestd_capoption is set.>>> from ghidra_pipe import PipeClient
>>> pipe_client = PipeClient()
>>> pipe_client.exec("""
... import sys
... print(sys.version)
... """)
2.7.2 (v2.7.2:925a3cc3b49d, Mar 21 2020, 10:03:58)
[OpenJDK 64-Bit Server VM (Oracle Corporation)]Since the code is executed in the remote global namespace of the pipe server all your import and object created at runtime are available betweenexeccall.>>> pipe_client.exec('a = 78')
>>> output = pipe_client.exec('print(a)', std_cap=True)
78
>>> output
'78\n'This feature can be useful to execute a third party script in Jython interpreter.>>> with open('/tmp/test_jython_script.py', 'r') as f:
...     output = PipeClient().exec(f.read(), std_cap=True)
...FunctionThePipeClient.register_funcmethod allow remote function declaration. It retrieves the source code of the function pass as argument and execute it on the remote global namespace of the pipe server. Note that this feature is supported natively in IPython REPL but not in classic REPL due to source code retrieving issues.fromghidra_pipeimportPipeClientdefremote_func(a,b=True):returna,bremote_func=PipeClient().register_func(remote_func)The method return a function proxy which can be used to invoke the remote Python function transparently. Function arguments and return values are limited to the following Python basic types : None, int, float, bool, str, dict, list, tuple, bytearray. Stdout and stderr of the function invoked remotely is forwarded locally.print(remote_func)a,b=remote_func(4,b=False)print(a)print(b)Output.<function PipeClient.func_proxy_factory.<locals>.func_proxy at 0x7f1b249abac0>
4
FalseClassThePipeClient.register_classmethod allow remote class declaration. It retrieves the source code of the class pass as argument and execute it on the remote global namespace of the pipe server. Note that this feature is not supported in Python/IPython REPL due to source code retrieving issues.fromghidra_pipeimportPipeClientclassFoo:CLASS_ATTR=78@staticmethoddefstatic_method(a):returna@classmethoddefclass_method(cls):returncls.CLASS_ATTR+2definstance_method(self):print(self)Foo=PipeClient().register_class(Foo)The decorator return a class proxy which can be used transparently as a standalone class or to create new object.print(Foo)print(Foo.CLASS_ATTR)print(Foo.static_method(5))print(Foo.class_method())Output.<class 'ghidra_pipe.pipe_client._class_proxy_factory.<locals>.ClassProxy'>
78
5
80When the class proxy is called for new object creation, a new object is created in the remote global namespace of the pipe server and the class proxy return an object proxy which can be used transparently.foo_obj=Foo()print(foo_obj)foo_obj.instance_method()Output.<ghidra_pipe.pipe_client.ObjProxy object at 0x7febb567b070>
<pipe_server.Foo instance at 0x80>Note that attributes access, return values, class and object method arguments are limited to the following Python basic types : None, int, float, bool, str, dict, list, bytearray. Stdout and stderr of the class/object methods executed remotely is forwarded locally.Standard Output and Error RedirectionBy default, the standard output and the standard error of the code executed remotely are forwarded to the standard output and error of the client. This behaviour can be change with thestd_forwardflag of thePipeClient.>>> from ghidra_pipe import PipeClient
>>> PipeClient().exec('print("debug")')
debug
>>> PipeClient(std_forward=False).exec('print("debug")')Remote ExceptionIf Python code executed remotely raise an exception anPipeServerRemoteCodeExecErrexception is raised locally. This exception contains various debug information as the code which raise the exception, the remote stacktrace, the port and the ip of the pipe server.fromghidra_pipeimportPipeClient,PipeServerRemoteCodeExecErrdefthis_func_raise_an_exception():v=1+not_existthis_func_raise_an_exception=PipeClient().register_func(this_func_raise_an_exception)try:this_func_raise_an_exception()exceptPipeServerRemoteCodeExecErrasex:print(ex.code)print('-'*10)print(ex.stacktrace)print('-'*10)print(ex.ip)print(ex.port)Output.__ret__=this_func_raise_an_exception()
----------
Traceback (most recent call last):
  File "/home/pink/ghidra-pipe/src/ghidra_pipe/pipe_server.py", line 263, in py_code_exec
    exec("""exec py_code in globals()""")
  File "<string>", line 1, in <module>
  File "<string>", line 1, in <module>
  File "<string>", line 2, in this_func_raise_an_exception
NameError: global name 'not_exist' is not defined

----------
localhost
5098Full Usage ExampleThis demonstration script shows how the pipe client interface can be used in a complementary way. First, thePipeClient.execis used to perform Python module import in the remote global namespace of the pipe server. Next, The classGhidraColorand the functionsset_memory_color,get_current_addrare declared in the remote global namespace though thePipeClient.register_classand thePipeClient.register_funcmethods. Then these class and functions are used locally to color 8 bytes of memory in blue and the next 8 bytes of memory in white. Next, thePipeClient.execsend code to the pipe server which use these same class and functions remotely to color the next 8 bytes of memory in blue.fromghidra_pipeimportPipeClientpipe_client=PipeClient()# Python modules importpipe_client.exec("""from ghidra.program.model.address import AddressSetfrom ghidra.app.plugin.core.colorizer import ColorizingServicefrom java.awt import Color""")classGhidraColor:def__init__(self):self.colorizing_service=state.getTool().getService(ColorizingService)defset_color(self,addr,rgb1,rgb2,rgb3):self.colorizing_service.setBackgroundColor(toAddr(addr),toAddr(addr),Color(rgb1,rgb2,rgb3))defset_memory_color(addr,rgb1,rgb2,rgb3):# Usage of class GhidraColor previously declaredg_color=GhidraColor()g_color.set_color(addr,rgb1,rgb2,rgb3)defget_current_addr():returnint(currentAddress.toString(),16)# Remote object declarationGhidraColor=pipe_client.register_class(GhidraColor)set_memory_color=pipe_client.register_func(set_memory_color)get_current_addr=pipe_client.register_func(get_current_addr)# Local usage of remote class and function declaredcurrent_addr=get_current_addr()ghidra_color=GhidraColor()ghidra_color.set_color(current_addr,0,0,255)# blueghidra_color.set_color(current_addr+4,0,0,255)# blueset_memory_color(current_addr+8,255,255,255)# whiteset_memory_color(current_addr+12,255,255,255)# white# Remote usage of class and function declared remotlypipe_client.exec("""current_addr = get_current_addr()remote_ghidra_color = GhidraColor()remote_ghidra_color.set_color(current_addr + 16, 255, 0, 0)  # redremote_ghidra_color.set_color(current_addr + 20, 255, 0, 0)  # red""")Custom Pipe Communication RoutinesThePipeClient.register_custom_communicatormethod allows to create a custom communication channels between an external tools and the remote routine. It retrieves the source code of the function pass as argument and executes it on the remote global namespace of the pipe server. The remote function is registered as a custom communication routine and become available. A communicator proxy is returned which can be used to open a custom communication channel with the remote routine. The code of the routine must compatible with CPython 3 and the remote version of Jython (2/3)Python.Custom Binary Communication ExampleThe following example registers thecoffee_communicatorcommunication routine. The routine send the value0xc0dec0feto the client and enter an infinite receive loop which except the value0xc0febab1to close the communication.fromghidra_pipeimportPipeClientdefcoffee_communicator(tcp_net_io):msg_out=jarray.zeros(0,'b')msg_out.fromstring(b'\xC0\xDE\xC0\xFE')tcp_net_io.sendall(msg_out)whileTrue:msg_in=tcp_net_io.recvall(4)msg_out=jarray.zeros(0,'b')ifmsg_in.tostring()==b'\xC0\xFE\xBA\xB1':msg_out.fromstring(b'\xDE\xAD\xBE\xEF')tcp_net_io.sendall(msg_out)tcp_net_io.sock.close()returnelse:msg_out.fromstring(b'\xFF\xFF\xFF\xFF')tcp_net_io.sendall(msg_out)coffee_communicator=PipeClient().register_custom_communicator(coffee_communicator)print(coffee_communicator)Output.<function PipeClient.communicator_proxy_factory.<locals>.communicator_proxy at 0x7fc858293880>When the communicator proxy is invoked a communication channel is open with the pipe server and the remote communication routine on the server side is called with apipe_server.JavaTcpNetIoinstance as argument. The communicator proxy return apipe_client.TcpNetIoinstance. These two objects in each side wrap a client socket connected to each other and provide communication helper methods (sendall,recvall,recvall_to_file,sendall_from_file)client_tcp_net_io=coffee_communicator()print(client_tcp_net_io)print(client_tcp_net_io.sock)Output.<ghidra_pipe.pipe_client.TcpNetIo object at 0x7fc858392320>
<socket.socket fd=3, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 60350), raddr=('127.0.0.1', 5098)>Next the communication with the remote routine is easy.print(client_tcp_net_io.recvall(4))client_tcp_net_io.sendall(b'\xC0\xFE\xBA\xB1')print(client_tcp_net_io.recvall(4))Output.bytearray(b'\xc0\xde\xc0\xfe')
bytearray(b'\xde\xad\xbe\xef')Use the underlying socket to close the communication.client_tcp_net_io.sock.close()Custom JSON Communication ExampleThe following example registers thejson_coffe_communicatorcommunication routine. The routine send a JSON message with the string 'C0DEC0FE' to the client and enter an infinite receive loop which except a JSON message with the string 'C0FEBAB1' to close the communication.fromghidra_pipeimportPipeClientdefjson_coffee_communicator(tcp_json_com):tcp_json_com.send({'data':'C0DEC0FE'})whileTrue:msg_in=tcp_json_com.recv()ifmsg_in['data']=='C0FEBAB1':tcp_json_com.send({'data':'DEADBEEF'})tcp_json_com.io.sock.close()else:tcp_json_com.send({'data':'retry'})json_coffee_communicator=PipeClient().register_custom_communicator(json_coffee_communicator,com_type='json')print(json_coffee_communicator)Output.<function PipeClient.communicator_proxy_factory.<locals>.communicator_proxy at 0x7f5308aa3ac0>When the communicator proxy is invoked a communication channel is open with the pipe server and the remote communication routine on the server side is called with apipe_server.JavaTcpJsonCominstance as argument and the function proxy return aTcpJsonCominstance. These two objects in each side wrap a client socket connected to each other and provide JSON communication helper methods (send,recv).tcp_json_com=json_coffee_communicator()print(tcp_json_com)print(tcp_json_com.io)print(tcp_json_com.io.sock)Output.<ghidra_pipe.pipe_client.TcpJsonCom object at 0x7f5308977fa0>
<ghidra_pipe.pipe_client.TcpNetIo object at 0x7f5308977c40>
<socket.socket fd=3, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 46422), raddr=('127.0.0.1', 5098)>Next the communication with the remote routine is easy.print(tcp_json_com.recv())tcp_json_com.send({'data':'C0FEBAB1'})print(tcp_json_com.recv())Output.{'data': 'C0DEC0FE'}
{'data': 'DEADBEEF'}Use the underlying socket to close the communication.tcp_json_com.io.sock.close()Reach Existing Remote Object from Everywhere Through ProxyRemote existing object declared in the global namespace of the pipe server as class, object and communicator can be reach from anywhere via the following pipe client proxy interface:PipeClient.func_proxy_factoryPipeClient.class_proxy_factoryPipeClient.obj_proxy_factoryPipeClient.communicator_proxy_factory.Example to reach a function previously declared.fromghidra_pipeimportPipeClientdeffoo():print(sys.version)foo=PipeClient().register_func(foo)foo()Output.2.7.2 (v2.7.2:925a3cc3b49d, Mar 21 2020, 10:03:58)
[OpenJDK 64-Bit Server VM (Oracle Corporation)]Reach this object from another place.>>> from ghidra_pipe import PipeClient
>>> foo = PipeClient().func_proxy_factory('foo')
>>> foo()
2.7.2 (v2.7.2:925a3cc3b49d, Mar 21 2020, 10:03:58)
[OpenJDK 64-Bit Server VM (Oracle Corporation)]Proxy Remote Object Tracking InformationFor each remote function, class, object or custom communicator declared in the remote global namespace of the pipe server a proxy is returned by the pipe client interface. Each proxy keep track of basic information about the target pipe server and the remote object via the following attributes:__PROXY_IP__,__PROXY_PORT__,__PROXY_OBJECT_NAME__,__PROXY_SRC__. Example with a remote function.fromghidra_pipeimportPipeClientdeffoo():passfoo=PipeClient().register_func(foo)print(foo.__PROXY_IP__)print(foo.__PROXY_PORT__)print(foo.__PROXY_OBJECT_NAME__)print("-"*80)print(foo.__PROXY_SRC__)Output.localhost
5098
foo
--------------------------------------------------------------------------------
def foo():
    passKeep in mind that if an object is redefined on a target remote pipe server, all object proxy which bind this object have the__PROXY_SRC__attribute de-synchronised with the remote object, because this information was not updated at runtime.File Copy Through PipeThe pipe client interface provides a way to copy file from local to remote, from remote to local, from local bytes buffer to remote file and remote file to local bytes buffer.Copy a local file on the remote pipe server filesystem.>>> from ghidra_pipe import PipeClient
>>>
>>> with open('/tmp/local_file.bin', 'wb') as f:
...     f.write(b'\xDE\xAD\xC0\xDE')
... 
4
>>> PipeClient().file_transfer_to_server('/tmp/local_file.bin', '/tmp/remote_file.bin')Copy a remote file from remote pipe server filesystem to local.>>> PipeClient().file_transfer_to_client('/tmp/remote_file.bin', '/tmp/local_file_comeback.bin')
4
>>> with open('/tmp/local_file_comeback.bin', 'rb') as f:
...      f.read()
... 
b'\xde\xad\xc0\xde'Copy a local bytes buffer to a file on the remote pipe server filesystem.>>> PipeClient().file_bytes_transfer_to_server(b'\xC0\xFE\xBA\xB1', '/tmp/bytes_remote_file.bin')Copy a file on the remote pipe server filesystem to local bytes buffer.>>> PipeClient().file_bytes_transfer_to_client('/tmp/bytes_remote_file.bin')
bytearray(b'\xc0\xfe\xba\xb1')Pipe Server JSON RPC InterfaceThe pipe server expose aJSON RPC V2Interface. The batch mode is not implemented. The pipe server is mono thread and process only one client at time. One RPC method is processed by connection.  The JSON frames exchanged by the client and the server are length prefixed as following. This frame encoding scheme is very simply and can be implemented in any language.4 bytes
+----------------+-------------------- // --------------------+
|  JSON LENGTH   |                JSON MESSAGE                |
+----------------+-------------------- // --------------------+As described in the JSON RPC V2 documentation the RPC requests take the following forms:{'jsonrpc': '2.0', 'id': <unique_request_identifier>, 'method': <method_name>, 'params': {}}And the RPC notification take the following forms:{'jsonrpc': '2.0', 'method': <method_name>, 'params': {}}The following RPC methods are available via RPC request:get_server_bannercode_execfunc_execobject_proxy_newobject_proxy_getattrobject_proxy_setattrremote_shutdownregister_custom_communicatorThe following RPC methods are available via RPC notification:execute_custom_communicatorfile_transfer_to_clientfile_transfer_to_serverAll the pipe server RPC methods are described in the following documentjson_rpc_api_pipe_server.mdDevelopmentInstall the package in develop mode with theDEVidentifier.$ git clone https://github.com/vincentdary/ghidra-pipe
$ cd ghidra_pipe
$ pip install -e .[DEV]For coverage information for both pipe client and server side install coverage in Jython and in Python2 (Required because Jython coverage do not support report generation).$ jython -m pip install coverage==5.6b1
$ python2 -m pip install coverage==4.3.4Run the tests.$ cd ghidra_pipe/test/  && ./run_tests.shFor coverage information run the test with the coverage flag.$ cd ghidra_pipe/test/ && ./run_tests.sh --coverageSee the coverage of the pipe client code.$ firefox  pipe_client_coverage/htmlcov/index.html &See the coverage of the pipe server code.$ cd pipe_server_coverage/ && coverage2 html
$ firefox pipe_server_coverage/htmlcov/index.html &FAQWhy an Another ToolThe author was charmed byGhidra Bridge, but the tool was not working as expected (very slow) and was not expose the desired interface. That's why this new tool was created, with fewer functionalities but with different technical choices and much less code.Why the Pipe Server use Java SocketThe pipe server use Java socket based onjava.netinstead of the socket library of Jython. The reason of this choice is caused by the slowness of the Jython socket interface (based onio.netty) due to the conversion of Java bytes to Python bytes. In any case the conversion Java/Python bytes with tostring /fromstring must be avoided for large data length when it is possible to avoid bottlenecks. The pipe server gets around this problem for exceptional cases when it is necessary by dropping and loading the content of Java/Python byte array in temporary file. It is a bit dirty, but it allows avoiding bottlenecks.Why Ghidra-Pipe no Proxify Ghidra API in the Client Side Global NamespaceBind the Ghidra API in the global namespace of the pipe client can be more convenient for REPL purpose and can allow less boilerplate code to access Ghidra Jython API.Ghidra Bridgeprovides this features. However, this choice has disastrous performance because for each method call or attribute access on remote object an underlying request must be sent to the server proxy which includes at least request/response serialization/deserialization and request processing. For example, to perform comparison between two remote object of type GenericAdresse this will involve several network exchanges and server/client processing before to obtain the result. This proxy mechanism slow by design is not suitable or unusable for large Ghidra script. Moreover, provide this feature correctly requires implementing a lot of mechanics. This is why Ghidra-Pipe has chosen to not implement this feature.