A method of operating a three-dimensional object printer to produce a three-dimensionally printed object with a surface region that has different color arrangements visible to an observer viewing the surface region from different view directions, different view angles, and with illumination lighting the surface region at different angles, the method comprising: providing three-dimensional data that defines a geometry of a surface region of the three-dimensionally printed object, at least two different color arrangements for the surface region, and at least one of a view direction, a view angle, and an illumination angle assigned to each different color arrangement; generating a model of the three-dimensionally printed object that includes data corresponding to a plurality of different material regions that together define a model surface region of the model and that overlap by different amounts viewed from different directions to enable different proportions of light from the plurality of different material regions to be visible to an observer viewing the model surface region from at least one of different view directions, different view angles, and with illumination lighting the surface region at different angles such that a coloration of the model surface region viewed from the view directions, view angles, and illumination angles defined by the three-dimensional data corresponds to the at least two color arrangement to which the view directions, view angles, and illumination angles are assigned; generating a color change map for the surface region with reference to the geometry of the surface region, the at least two different color arrangements for the surface region, and the at least one view direction, view angle, and illumination angle assigned to each different color arrangement; generating a color change map of the model surface region with reference to the data corresponding to a plurality of different material regions that together define the model surface region and the at least one view direction, view angle, and illumination angle assigned to each different color arrangement; comparing the color change map of the surface region to the color change map of the model surface region to measure a similarity of at least one of color hue and color location between the color change map of the surface region to the color change map of the model surface region; continuing the generation of the model until the measured similarity between the color change map of the surface region to the color change map of the model surface region is greater than a predetermined threshold at the at least one view direction, view angle, and illumination angle assigned to each different color arrangement; modifying the geometry of the surface region of the three-dimensionally printed object with reference to the generated model; and operating a three-dimensional object printer with reference to the modified geometry to produce the three-dimensional object. providing three-dimensional data that defines a geometry of a surface region of the three-dimensionally printed object, at least two different color arrangements for the surface region, and at least one of a view direction, a view angle, and an illumination angle assigned to each different color arrangement; generating a model of the three-dimensionally printed object that includes data corresponding to a plurality of different material regions that together define a model surface region of the model and that overlap by different amounts viewed from different directions to enable different proportions of light from the plurality of different material regions to be visible to an observer viewing the model surface region from at least one of different view directions, different view angles, and with illumination lighting the surface region at different angles such that a coloration of the model surface region viewed from the view directions, view angles, and illumination angles defined by the three-dimensional data corresponds to the at least two color arrangement to which the view directions, view angles, and illumination angles are assigned; generating a color change map for the surface region with reference to the geometry of the surface region, the at least two different color arrangements for the surface region, and the at least one view direction, view angle, and illumination angle assigned to each different color arrangement; generating a color change map of the model surface region with reference to the data corresponding to a plurality of different material regions that together define the model surface region and the at least one view direction, view angle, and illumination angle assigned to each different color arrangement; comparing the color change map of the surface region to the color change map of the model surface region to measure a similarity of at least one of color hue and color location between the color change map of the surface region to the color change map of the model surface region; continuing the generation of the model until the measured similarity between the color change map of the surface region to the color change map of the model surface region is greater than a predetermined threshold at the at least one view direction, view angle, and illumination angle assigned to each different color arrangement; modifying the geometry of the surface region of the three-dimensionally printed object with reference to the generated model; and operating a three-dimensional object printer with reference to the modified geometry to produce the three-dimensional object.