If you wanted the black dot to look black on a red field, you would most likely have to make it a very dark red to compensate for the ghostly aura of green. We call this effect in adjacent colors simultaneous contrast. The principle applies to all such interactions. The effect, furthermore, is modified or enhanced by the particular colors chosen. Pure, intense, primary hues are not much affected by context, but lower-intensity colors and colors that are not primary or spectral can change appearance radically.
Size can also influence the effect of simultaneous contrast.
Generally, larger color areas have more influence on smaller color areas. The small green square is changed by the color of the yellow or blue field, not the reverse. This is why colored type seems to be affected by the color of the ground and not the reverse, or why a red thread running through a green cloth looks so brilliantly red. We do not see it happen unless it is presented in diagramatic form as in the figures here. The effect of simultaneous contrast is only interesting to the viewer of a work of art or design, but absolutely critical to its maker.