Remove annotations which can now be inferred