When I wrote the state machine, it was clear that the message processing was dependent on whether the queue was empty or not. So that would be the first handle_cast. Its just that queue.is_empty() isn't a valid guard :( I would love to get rid of it if there is a way around it.

empty is same as no reconnect timer. Changing the variable to make it waiting_on_reconnect instead of empty would make the following unclear.
handle_cast(Msg, State = #state{empty = false}) ->
  {noreply, queue_msg(Msg, State)};