Afaik it is only possible to install PHP modules when using PHP workers. It would be no problem if I could install anything I want but this is not the case in the public Travis version (in the private version you can provision the machines like you want).

A PHP worker has Ruby installed however the .gemfile of a repository will be ignored there then. I tried it here: https://travis-ci.org/frastel/Puphpet/builds/8999139 and the worker failed because the needed gems are not installed. Googling also confirmed that using both languages in this way is not supported.