Jesus indicates that the food is more than food. It is a sign. The crowd then asks for another sign as a demonstration so they can believe in Jesus. Jesus, refusing to make more bread, declares that he is the bread of life. Now notice that at no time does the Gospel of John criticize the people for thinking in concrete terms about bread.
Bread is good.
God gave real bread in the wilderness. God gives a very real Son whose very bodily presence in the world is called the bread of life. The people had lost contact with the basics. When they ate bread, they saw nothing more than food. They could not eat bread and see more than something to fill their stomachs.