Remove permissions from the todo list