Package: transpyler
.. image:: https://travis-ci.org/Transpyler/transpyler.svg?branch=master:target: https://travis-ci.org/Transpyler/transpyler.. image:: https://codecov.io/gh/Transpyler/transpyler/branch/master/graph/badge.svg:target: https://codecov.io/gh/Transpyler/transpyler.. image:: https://ci.appveyor.com/api/projects/status/it45pshyqi76irbx?svg=true:target: https://ci.appveyor.com/project/fabiommendes/transpylerTranspyler is an infrastructure to create simple localized versions ofPython. It was originally created as part of the Pytuguês language (Pythonin Portuguese), but it is now abstracted to handle any translation. Thegoal of such specialized languages is to provide a more friendly environment tonewcomers that are not fluent in English, children and adults alike.Creating support for a new language is very simple. Say we want to create aPy-Klingon to help us dominate the galaxy. Fortunately, Klingon structure isnot very different from English and we can go a long way just by translatingtokens. In transpyler it looks like this:.. code-block:: pythonfrom transpyler import Transpyler# Let us define Py-Klingonklingon = Transpyler(name='Py-Klingon',translations={# Warning: computer-based translation!'meH': 'for','chugh': 'if','latlh': 'else','Qap': 'def','nobHa': 'return',# ... you get the idea ;-)})# Now execute some Klingon number crunchingklingon.exec('''Qap fib(x):chugh x < 0:nobHa 1latlh:nobHa fib(x - 1) + fib(x - 2)''', globals())print(fib(10))  # It creates a Python function!Since the main goal is educational, transpyler-enabled languages automaticallysupports a series of nice educational tools inherited from the original Pytuguêsruntume:* Support for Jupyter/IPython: we can easily create a Jupyter kernel from aTranspyler instance. This enables nice consoles and notebooks whichcan be really handy in teaching a new programming language.* QCode support and syntax highlight: QCode is a Qt-based widget for codingediting.* Pygments plugin: Pygments is a source code highlighter commonly used to generatenice coloured HTML for source code.* QTurtle application: QTurtle is similar to Python's own turtle module rewrittenin Qt. This is great for teaching kids, and all transpyled languages supportit for free :)How does it work?-----------------Transpilation is the act of translating code from one programming language toanother (as opposed to compilation, which translate source code to machinecode). A **transpylation** is even simpler: it translate a Python-like languageback to Python (and sometimes execute it directly). This task is not verydifficult since both languages are similar and we can reuse lots of Python'sown machinery to analyze and modify its own source code.Transpyler acts exclusively at the token level: it modifies the stream of tokensfrom the original translated language to a new stream of tokens that must thengenerate a grammatically valid python program. Indeed, we do not even implementour own tokenizer and use Python's own ``tokenize`` module to handle this partfor us.Transpyler works by making several passes over the list of tokens. The firstpass simply performs direct translations such as those shown in the Py-Klingonexample above. A second pass makes maps groups of tokens into a single Pythontoken (eg.: we could make a "for each" command that is translated to "for").Subsequent passes may look for specific patterns of tokens and perform morecomplex translations. Pytuguês, for instance, implements a "repeat" command.In English it would be something like this:.. code-block:: transpylerrepeat 4 times:forward(100)left(90)This is transpiled to Python as:.. ignore-next-block.. code-block:: pythonfor ___ in range(4):forward(100)left(90)It is possible to make arbitrary modifications to the list of tokens which inprinciple could allow arbitrary syntactic constructs. Notice that tokens stillcome from the Python tokenizer and hence there are certain hard lexicalconstraints on (including indentation as block delimiter semantics which inPython is implemented at the tokenizer level rather than in the grammar).Transpyler do not implement source code maps because they are not needed byPytuguês and we don't think that simple localized Pythons would requireit. That said, new grammatical constructs should keep line numbers unaltered.Remember: transpyler is using your regular vanilla Python interpreter and puttinga small layer of token translation on top of it. We make a few dirty hacks in thePython runtime, but it does not requires any specially flavored interpreterand additional compilation is unnecessary.And the standard lib?---------------------I'm glad you asked :). This is by far the most laborious part of doing a decentPython translation. Transpyler offers a few helpful tools, but most of the workis the inevitable task of translating the names and docstrings of each functionyou want to support into a their localized counterparts.As a convenience tool, you can list the functions you want to translate andwe offer a tool that uses Google Translate to create a boilerplate for yourstandard lib functions. Google Translate is a wonderful tool, but we all knowhow bad it can get for serious translations. Always keep that in mind.Here is a list of all projects using Transpyler.* Pytuguês: the original Python to portuguese.* Pykor: Korean version.* Pytuñol: non-serious translation to Portuñol. It is a minimal project and canbe used as a template for new translations.How about the builtin types?----------------------------Python builtins poses a challenge. They cannot be monkey-patched at Python level,but we need to modify them. In Py-Klingon, we want the "teH" constant (True) tobe represented as "teH" rather than "True". We also want method names forlists, strings, etc to be fully translated.Transpyler messes with these types at C level using ctypes. The techniques arevery similar to those implemented in a module called ``forbiddenfruit``,which recommends never to use itself unless you want to do something dangerous orsilly :)In the language introduced by ``forbiddenfruit``, modifying a method to a builtintype is called making a curse. We provide a tools to curse Python's builtins easilyand effectively.A superset of Python?---------------------Transpyler languages are usually supersets of Python itself. In Pytuguês, forinstance, most Python code is also a valid Pytuguês code. This makes languageseasier to implement since we don't have to blacklist the original Pythonkeywords, but it also creates a path for going from a educational Pythonesquelanguages to vanilla Python used in real world applications.The fact that we don't make any effort to hide the Python internals and Pythonoriginal syntax is not a bug, its a feature!