Fix up line number handling to be unsigned int safe
Clean up some more TODOs
Fix issue when writing parameter annotations