I like to cut the kumquat back hard, then wait until it stops producing new growth. This usually takes a couple of weeks, but if you don't prune your kumquat you'll probably lose most of the fruit before then. You can then cut off all the side branches and the top of the tree.
The best time to pick kumquats is in late summer or early fall. You can store them for up to two months in the refrigerator. They keep really well, so you can make some delicious fresh-tasting jelly with them.