Finally it seems to work:
1) hook rvm:trust_rvmrc to the proper capistrano task (before start), to make sure it gets executed when using deploy:cold as well.
2) modify rvm:trust_rvmrc to use the current release, so that it can be used independently.
3) pay attention to the passenger installation process because specifying the gemset when configuring PassengerRuby is important.