Optimize patch drawing

This both makes patch drawing way faster and fixes #86.