Keep date when remove time suggestion

* Only remove date when click to remove time suggestion
* Do not show option to remove time suggestion when there is only one

https://trello.com/c/TxsRuEsd