Clean up object::symbol::SymbolMap, making it a normal struct