[#50639317] Update robots file
[#50639317] Update 404 behaviour