Apparatus for converting a first parametric spatial audio signal (112, 114) representing a first listening position or a first listening orientation in a spatial audio scene to a second parametric spatial audio signal (212, 214) representing a second listening position or a second listening orientation, the apparatus comprising: a spatial audio signal modification unit (301, 302) adapted to modify at least one out of the downmix signal of the first parametric spatial audio signal (112, 114), the direction-of-arrival parameter of the first parametric spatial audio signal and the diffuseness parameter of the first parametric spatial audio signal dependent on a change of the first listening position or the first listening orientation so as to obtain the second parametric spatial audio signal (212, 214), wherein the second listening position or the second listening orientation corresponds to the first listening position or the first listening orientation changed by the change; wherein the spatial audio signal modification unit (301, 302) comprises: a parameter modification unit (301, 301a) adapted to modify a first directional parameter of the first parametric spatial audio signal (112, 114) so as to obtain a second directional parameter of the second parametric spatial audio signal (212, 214) depending on a control signal providing information corresponding to the change; and wherein the parameter modification unit (301, 301b) is adapted to modify a first diffuseness parameter of the first parametric spatial audio signal so as to obtain a second diffuseness parameter of the second parametric spatial audio signal depending on the first directional parameter or depending on the second directional parameter. a spatial audio signal modification unit (301, 302) adapted to modify at least one out of the downmix signal of the first parametric spatial audio signal (112, 114), the direction-of-arrival parameter of the first parametric spatial audio signal and the diffuseness parameter of the first parametric spatial audio signal dependent on a change of the first listening position or the first listening orientation so as to obtain the second parametric spatial audio signal (212, 214), wherein the second listening position or the second listening orientation corresponds to the first listening position or the first listening orientation changed by the change; wherein the spatial audio signal modification unit (301, 302) comprises: a parameter modification unit (301, 301a) adapted to modify a first directional parameter of the first parametric spatial audio signal (112, 114) so as to obtain a second directional parameter of the second parametric spatial audio signal (212, 214) depending on a control signal providing information corresponding to the change; and wherein the parameter modification unit (301, 301b) is adapted to modify a first diffuseness parameter of the first parametric spatial audio signal so as to obtain a second diffuseness parameter of the second parametric spatial audio signal depending on the first directional parameter or depending on the second directional parameter. a parameter modification unit (301, 301a) adapted to modify a first directional parameter of the first parametric spatial audio signal (112, 114) so as to obtain a second directional parameter of the second parametric spatial audio signal (212, 214) depending on a control signal providing information corresponding to the change; and wherein the parameter modification unit (301, 301b) is adapted to modify a first diffuseness parameter of the first parametric spatial audio signal so as to obtain a second diffuseness parameter of the second parametric spatial audio signal depending on the first directional parameter or depending on the second directional parameter.