After shelling into the container and inspecting it appears that we are creating a dup entry of:
"deb http://archive.ubuntu.com/ubuntu precise universe"