Talk of the necessity of an 'external standard' can be misleading. Any standard I can use or appeal to is necessarily internal. I like your example of the taste of coffee. Say, I used Columbian beans instead of Brazilian, or I carelessly allowed the coffee to boil, or the jug was not washed out properly. However, suppose now that in parallel with my gradually 'forgetting' which taste I mean to refer to by the term 'coffee' there are other, compensating errors in my 'theory of an external world'.
Absolute idealism is one version of anti- solipsism.
Other versions are Berkeleian immaterialism, materialism etc. If I agree to work for you for a low wage, I have no right to demand extra financial support for myself and my family financed through extra taxes paid by you and others who are better off than me. That is theft. If I want more, I should work harder, or drive a harder bargain. Rawls' intuition is that a better overall 'average' is not necessarily preferable to a society where effort is put towards reducing sharp inequalities, at a cost of reducing the overall sum.