Hi guys! Let's go through this:

1. Yes it's true, this solution will never be perfect. However, whe we have a big project like Bukkit, I think it comes down to people just need to submit things right for them to be reviewed to be pulled. If a dev can't figure out basic procedures that would be outlined in a contribution guide, chances are that the pull request isn't any good. 
2. It would be possible in some ways to use the Gthub API. Travis returns a pull request number, so using Github's API we could get the commit writer and see if they have a Bukkit fork, etc. However, this is a lot of extra work, and I personally won't do it, as I unfortunately can't sit here all day tweaking a pull request to fit every single possibility. 
3. We unfortunately can't have Travis retry builds over and over again. Their API is pretty read only, so not much we could do there. 
4. @ryantheleach Travis can't read pull requests directly, but with the Github API we could. Again though, I'll let someone else do that. 

I think the point I'm trying to make is thus solution works if there was a contribution guide that outlined these steps. Any good developer reads these, and if they don't, then their change doesn't get merged in. If someone wants to take this and modify it so every possibility works, then they may go ahead.