The toffee should cool for at least 2 hours before breaking it into pieces. This allows it to harden and set properly.