Actually add something to the silence file