The only way to deal with it is to create a Watchdog rule to terminate or restart the offending process when its memory grows beyond a certain threshold.