Fixed up annotation server handling

Signed-off-by: Brandyn A. White <bwhite@dappervision.com>