Reference counted memory management using actual pointers