This is perhaps too harsh, so I don't recommend it. Players should have a reason to stay interested, after all. Keep the overall net score for the pink ball separately, and give a prize to the team with the best pink ball score. If a team loses the pink ball, it's out. Trouble is a point game in which your actual score isn't relevant, at least not directly.
Players shoot for a set amount per point, often a dollar.
Thus, a player accumulating three trouble points owes each of his opponents three dollars. A player can erase all the points accumulated on a given hole by making par. At the end of the round, simply net all the points against each other and settle up. Trouble is an excellent game for the intermediate player. Often, such players are feeling pretty smug as their handicaps drop, and they need to be taken down a notch or two.