Package: python2
A library for running Python 2 code from a Python 3 application.Effortlessly harness the power and convenience of Python 2… in Python 3!Why?Why not??This library was created for more whimsical than practical reasons.  In theory,
it could be used to interface with legacy Python 2 code which for one reason or
another cannot be ported to Python 3.Installationpython2requires a working install of both Python 2 and Python 3.
Currently the library has only been tested with Python 2.7 and Python 3.4, 3.5,
and 3.6.To install the package:pip install -U python2If using virtualenvs, you will need to create separate Python 2 and 3
virtualenvs, and install the package into both.UsageTo begin working with Python 2, import the package in Python 3 and create a newPython2object:>>> from python2.client import Python2
>>> py2 = Python2('/path/to/python2/executable')This object is our gateway to the Python 2 world.  Python 2 builtins can be
accessed as attributes of thePython2object. Let’s use Python 2’s__import__()function to import the deprecatedshamodule, which was
removed in Python 3:>>> py2_sha = py2.__import__('sha')
>>> py2_sha.sha('abc')
<Py2Object <sha1 HASH object @ 0x107463c30>>Ahh, just like the good ol’ days.  You can deprecate a module but you can’t
deprecate the human spirit!We can use thePython2.project()method to convert Python 3 objects to
Python 2:>>> py2.project(1)
<Py2Object 1>
>>> py2.project('foo')
<Py2Object u'foo'>You can usePython2.lift()to lift Python 2 objects back to Python 3.  For
container types, usePython2.deeplift()to recursively perform the lifting.Py2Objectinstances have special properties_and__to perform the
equivalent operations:>>> o = py2.project([1, 2, 3])
>>> o
<Py2Object [1, 2, 3]>
>>> o._
[<Py2Object 1>, <Py2Object 2>, <Py2Object 3>]
>>> o.__
[1, 2, 3]Python 2 objects can be used pretty much like regular Python 3 objects.  You
can also freely mix and match with Python 3 builtin types:>>> x = py2.project(1)
>>> x
<Py2Object 1>
>>> str(x)
'1'
>>> x + 1
<Py2Object 2>
>>> d = py2.dict(foo=x, bar=None)
>>> d['foo'] is x
True
>>> del d['foo']
>>> d
<Py2Object {u'bar': None}>
>>> d.__
{'bar': None}If you just want to execute some Python 2 code directly, you can use thePython2.exec()method.  This method accepts a string containing Python 2
code and an optional dict representing the scope to execute the code in, and
returns the resulting scope after executing the code.  This can be used to
define new Python 2 classes and functions:>>> scope = py2.exec("""
... def foo(x):
...     return x + 1
... """)
>>> foo = scope['foo']
>>> foo(2)
<Py2Object 3>If an exception occurs in Python 2, aPy2Errorwill be thrown by the
client.  The Python 2 exception is stored as theexceptionattribute of thePy2Errorobject.  The underlying traceback is attached to the Python 2
exception as the__traceback__attribute.>>> py2.int('asdf')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  ...
python2.client.exceptions.Py2Error: ValueError: invalid literal for int() with base 10: 'asdf'When you’re done using Python 2, you can end the session by calling thePython2.shutdown()method.  You can also use thePython2object as a
context manager to automatically do the same thing when exiting the context.>>> py2.shutdown()TestingThis package uses Tox for testing.  Tests are not included in the Python dist,
so you will need to clone the repo to run them.  To run the unit tests, install
Tox and run the following command from the project’s base directory:toxAfter running tox, you can run the client-server integration tests with the
‘integration_tests.sh’ script.  This script takes two arguments specifying the
Tox virtualenvs to use for Python 2 and 3, respectively:./integration_tests.sh py27 py36To modify the behavior of Tox, you can set thePYTEST_ADDOPTSvariable.
For example, you can set the-xflag to abort after the first test
failure:export PYTEST_ADDOPTS=-xYou can use the-nNUMflag to parallelize the tests using thepytest-xdist pluginThis adds some overhead to the test setup, so this
option is primarily useful for speeding up the integration tests.export PYTEST_ADDOPTS=’-n 4’CaveatsSupported typesProjection is only supported for basic builtin types.  Other objects cannot be
projected to Python 2.  The supported types are:bool,int,float,complex,bytes,unicode,bytearray,range,slice,list,tuple,set,frozenset, anddict.  TheNone,NotImplemented, andEllipsissingletons are also supported.In particular, Python 3 functions, types, and instances of user-defined classes
cannot currently be projected into Python 2.Type introspectionThePy2Objectclass implements many “magic methods” from the Python 3 data
model.  As a result, aPy2Objectappears to be callable, iterable, etc.,
even if the underlying object is not.  Attempting to perform such operations may
result in aPy2Error.If you need to introspect a Python 2 object, use the correspondingPython 2builtin functions.  For example:>>> i = py2.project(1)
>>> py2.callable(i)
<Py2Object False>
>>> py2.isinstance(i, py2.int)
<Py2Object True>String typesIn Python 2,strobjects are raw byte strings, while in Python 3 they are
Unicode strings.  This can lead to some confusion, as projecting a Python 3strwill result in a Python 2unicodeobject, while lifting a Python 2strwill return a Python 3bytesobject.>>> py2.project('foo')
<Py2Object u'foo'>
>>> py2.lift(py2.str(123))
b'123'DivisionThe behavior of the division operator changed withPEP 238.  This created
two alternate division operations, “true division” and “classic division”.
Classic division was removed in Python 3.To respect this change, when twoPy2Objects are divided, classic division
is used.  When aPy2Objectdivides or is divided by a Python 3 value, true division is used.>>> i = py2.project(1)
>>> j = py2.project(2)
>>> i / j  # classic division
<Py2Object 0>
>>> i / 2  # true division
<Py2Object 0.5>
>>> 1 / j  # true division
<Py2Object 0.5>Further discussionHow it worksWhen you launch a Python 2 session, the library spawns a child process running
Python 2.  This child process runs aserverthat listens for commands from
the Python 3client.  For each command, the server performs an operation in
Python 2 and returns the result either as an encodedvaluemade up of
supported types, or areferenceto a Python 2 object stored on the server.On the client side, the library wraps Python 2 references with thePy2Objectclass.  This class implements many of the “magic methods” of thePython 3 data modelby sending commands to the Python 2 server to perform
the appropriate operation on the underlying Python 2 object.Call-by-value semanticsWhen projecting a value or calling a Python 2 function with Python 3 arguments,
the arguments will be passed to Python 2 “by value”, that is, by encoding the
value of the argument to be decoded by the server.  When using a Python 2
object, the object is stored in the Python 2 session and is passed “by
reference”.This has some implications for the semantics of Python 2 functions.  Suppose we
have a Python 2 function that mutates a list.  If we pass this function a
Python 3 list, the list will be copied into Python 2 and the copy will be
mutated, but the original will not be modified:>>> f = py2.eval("lambda l: l.append(1)")
>>> l = []
>>> f(l)
<Py2Object None>
>>> l
[]However, if we project the list into Python 2 before passing it to the
function, then we can observe the modifications on the projected list:>>> py2_l = py2.project(l)
>>> f(py2_l)
<Py2Object None>
>>> py2_l
<Py2Object [1]>Return semanticsReturning generally occurs by reference except for operations that require a
specific return type (str(),int(), etc.).  The main reason for this is
that returning by value may lose information about object identity that needs
to be preserved.  Return values can be easily lifted to Python 2 if desired.Object identity and lifespanEach Python 2 object returned by the server is represented by a uniquePy2Object.  This means that theisoperator can be used to determine if
twoPy2Objects refer to the same underlying object.The Python 2 server stores all objects it returns, to prevent them from being
deallocated.  When the correspondingPy2Objectis deallocated in the Python
3 process, the underlying Python 2 object will be removed from the server cache
to allow it to be deallocated as appropriate.Encoding algorithmThis library uses a simple JSON encoding for supported types.  For a given
function call, each unique object will only be encoded once.  This means that
data structures with circular references are supported.  For a detailed
description of the algorithm, see thepython2.shared.codecmodule.Possible improvementsPython 2 typesCurrently there is a single type for Python 2 objects in Python 3,Py2Object. An alternate strategy would be to dynamically create Python 3
classes for each Python 2 type encountered, and create proxy objects as
instances of these classes.The main benefit of this change would be better type introspection for Python 2
objects (see the discussion atType introspection).  However, it would be
more cumbersome and incur a performance cost, since the client would need to
know the type of each object and the methods supported by that type.
Additionally, this approach would not fully support the dynamic nature of the
Python type system, since the proxied type would not reflect changes to the
underlying type such as adding or removing methods.This would require the server to return the object type for references, and
some mechanism for the client to introspect Python 2 types.  The client would
cache types for the lifetime of the Python 2 session, with a mechanism to
explicitly refresh a type to pick up any changes that had occurred in Python 2.Bootstrapping the type system might be a little tricky.  We would want to
create a typePy2typesuch all proxy types are instances of,including
``Py2type`` itself.We would also probably want a base type for all proxy
objects, including types.Python 3 proxy objects in Python 2Currently the relationship between client and server is asymmetrical.  The
client has a representation of Python 2 objects, but the server does not have
a way to represent Python 3 objects.  We might like to add such a mechanism.
This would mean that instead of the simple request-response pattern from client
to server we have now, there would be the possibility of callbacks.  In effect,
the two processes would act more like coroutines with the flow of control
passing back and forth between them.Better Python version supportWe could extend support to more Python 2 and 3 versions.Similar projectsAfter writing this library, I discovered that I’m not the only one to have had
this idea.Suxis a library that provides similar functionality, with some
notable differences:The library is much smaller and more lightweight, and only needs to be
installed in the Python 3 environment to work.The main emphasis is on imports and function calls, which makes sense since
these are the most important operations for the using legacy packages.  Most
other operators (e.g. arithmetic operators) are not supported.The library uses Pickle to communicate between the Python 2 and 3 processes.
This is a good idea and I should probably have done the same, although I had
fun implementing the current encoding algorithm.