Add ability to add and remove delegates

This operation are not thread-safe by design and it will be responsibility of the caller to access this operation in thread-safe manner.