Package: grumpy-tools
# Grumpy: Go running Python[![Build Status](https://travis-ci.org/alanjds/grumpy.svg?branch=master)](https://travis-ci.org/alanjds/grumpy)[![Join the chat at https://gitter.im/grumpy-devel/Lobby](https://badges.gitter.im/grumpy-devel/Lobby.svg)](https://gitter.im/grumpy-devel/Lobby?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)## OverviewGrumpy is a Python to Go source code transcompiler and runtime that is intendedto be a near drop-in replacement for CPython 2.7. The key difference is that itcompiles Python source code to Go source code which is then compiled to nativecode, rather than to bytecode. This means that Grumpy has no VM. The compiled Gosource code is a series of calls to the Grumpy runtime, a Go library serving asimilar purpose to the Python C API (although the API is incompatible withCPython's).## Limitations### Things that will probably never be supported by Grumpy1. `exec`, `eval` and `compile`: These dynamic features of CPython are notsupported by Grumpy because Grumpy modules consist of statically-compiled Gocode. Supporting dynamic execution would require bundling Grumpy programswith the compilation toolchain, which would be unwieldy and impracticallyslow.2. C extension modules: Grumpy has a different API and object layout thanCPython and so supporting C extensions would be difficult. In principle it'spossible to support them via an API bridge layer like the one that[JyNI](http://jyni.org) provides for Jython, but it would be hard to maintain andwould add significant overhead when calling into and out of extensionmodules.### Things that Grumpy will support but doesn't yetThere are three basic categories of incomplete functionality:1. [Language features](https://github.com/google/grumpy/wiki/Missing-features#language-features):Most language features are implemented with the notable exception of[old-style classes](http://stackoverflow.com/questions/54867/what-is-the-difference-between-old-style-and-new-style-classes-in-python).There are also a handful of operators that aren't yet supported.2. [Builtin functions and types](https://github.com/google/grumpy/wiki/Missing-features#builtins):There are a number of missing functions and types in `__builtins__` that havenot yet been implemented. There are also a lot of methods on builtin typesthat are missing.3. [Standard library](https://github.com/google/grumpy/wiki/Missing-features#standard-libraries):The Python standard library is very large and much of it is pure Python, soas the language features and builtins get filled out, many modules willjust work. But there are also a number of libraries in CPython that are Cextension modules which will need to be rewritten.4. C locale support: Go doesn't support locales in the same way that C does. As such,some functionality that is locale-dependent may not currently work the same as inCPython.## Running Grumpy Programs### Pre-requisitesThe commands ahead assumes that you have Golang installed and a recentversion of Python 2, `setuptools` and `pip`.### Method 1: binary packageFor convenience, a Python package is provided from the PyPI. During install,many Grumpy will be compiled and stored inside your Python installation.You need Golang preinstalled anyway for the installation to be successful.```pip2 install -U grumpy-runtime -I --no-cache(wait about 5 minutes)echo "print 'hello, world'" | grumpy run```### Method 1: make run:The simplest way to execute a Grumpy program is to use `make run`, which wraps ashell script called grumprun that takes Python code on stdin and builds and runsthe code under Grumpy:```cd grumpy-tools-srcpython2 setup.py developcd ../grumpy-runtime-srcecho "print 'hello, world'" | make run```### Method 2: grumpc and grumprun:For more complicated programs, you'll want to compile your Python source code toGo using grumpc (the Grumpy compiler) and then build the Go code using `gobuild`. Since Grumpy programs are statically linked, all the modules in aprogram must be findable by the Grumpy toolchain on the GOPATH. Grumpy looks forGo packages corresponding to Python modules in the \_\_python\_\_ subdirectoryof the GOPATH. By convention, this subdirectory is also used for staging Pythonsource code, making it similar to the PYTHONPATH.The first step is to set up the shell so that the Grumpy toolchain and librariescan be found. From the root directory of the Grumpy source distribution run:```cd grumpy-tools-srcpython2 setup.py developcd ../grumpy-runtime-srcmakeexport PATH=$PWD/build/bin:$PATHexport GOPATH=$PWD/buildexport PYTHONPATH=$PWD/build/lib/python2.7/site-packages```You will know things are working if you see the expected output from thiscommand:```cd grumpy-runtime-srcecho 'import sys; print sys.version' | grumprun```Next, we will write our simple Python module into the \_\_python\_\_ directory:```cd grumpy-runtime-srcecho 'def hello(): print "hello, world"' > $GOPATH/src/__python__/hello.py```To build a Go package from our Python script, run the following:```cd grumpy-runtime-srcmkdir -p $GOPATH/src/__python__/hellogrumpc -modname=hello $GOPATH/src/__python__/hello.py > \$GOPATH/src/__python__/hello/module.go```You should now be able to build a Go program that imports the package"\_\_python\_\_/hello". We can also import this module into Python programsthat are built using grumprun:```cd grumpy-runtime-srcecho 'from hello import hello; hello()' | grumprun```grumprun is doing a few things under the hood here:1. Compiles the given Python code to a dummy Go package, the same way weproduced \_\_python\_\_/hello/module.go above2. Produces a main Go package that imports the Go package from step 1. andexecutes it as our \_\_main\_\_ Python package3. Executes `go run` on the main package generated in step 2.## Developing GrumpyThere are three main components and depending on what kind of feature you'rewriting, you may need to change one or more of these.### Grumpy ToolsGrumpy converts Python programs into Go programs and`grumpy transpile` is the CLI tool responsible for parsing Python codeand generating Go code from it. `grumpy transpile` is written in Pythonand uses the [`pythonparser`](https://github.com/m-labs/pythonparser)module to accomplish parsing.The CLI main entrypoint lives at `grumpy-tools-src/grumpy_tools/cli.py`.It is supported by a number of Python modules in the`grumpy-tools-src/grumpy_tools/compiler` subdir.### Grumpy RuntimeThe Go code generated by `grumpy transpile` performs operationson data structures that represent Python objects in running Grumpy programs.These data structures and operations are defined in the `grumpy` Go library(source is in the `grumpy-runtime-src/runtime` subdir of the sourcedistribution). This runtime is analogous to the Python C API and many of thestructures and operations defined by `grumpy` have counterparts in CPython.### Grumpy Standard LibraryMuch of the Python standard library is written in Python and thus "just works"in Grumpy. These parts of the standard library are copied from CPython 2.7(possibly with light modifications). For licensing reasons, these files are keptin the `grumpy-runtime-src/third_party` subdir.The parts of the standard library that cannot be written in pure Python, e.g.file and directory operations, are kept in the `grumpy-runtime-src/lib` subdir.In CPython these kinds of modules are written as C extensions. In Grumpy theyare written in Python but they use native Go extensions to access facilities nototherwise available in Python.### Source Code Overview- `grumpy-tools-src/grumpy_tools/compiler`: Python package implementating Python -> Go transcompilation logic.- `grumpy-runtime-src/lib`: Grumpy-specific Python standard library implementation.- `grumpy-runtime-src/runtime`: Go source code for the Grumpy runtime library.- `grumpy-runtime-src/third_party/ouroboros`: Pure Python standard libraries copied from the[Ouroboros project](https://github.com/pybee/ouroboros).- `grumpy-runtime-src/third_party/pypy`: Pure Python standard libraries copied from PyPy.- `grumpy-runtime-src/third_party/stdlib`: Pure Python standard libraries copied from CPython.- `grumpy-tools-src/grumpy_tools/`: Transcompilation and utility CLI.## ContactQuestions? Comments? Drop us a line at [grumpy-users@googlegroups.com](https://groups.google.com/forum/#!forum/grumpy-users)or join our [Gitter channel](https://gitter.im/grumpy-devel/Lobby)