A method implemented by a hearing assistance system (100, 200) comprising a left ear device and a right ear device respectively configured to be worn by a wearer, the method comprising: determining, using one or more positional sensors of the hearing assistance system (100, 200), a three-dimensional position of the hearing assistance system (100, 200) in response to the wearer looking at a sound source in space; tracking changes in the three-dimensional position of one or both of the hearing assistance system (100, 200) and the sound source, wherein tracking the changes comprises: determining changes in an interaural time difference between the left ear device and the right ear device; and determining changes in an interaural level difference between the left ear device and the right ear device; receiving, when the sound source in space is a target sound source of interest, a first input to the hearing assistance system (100, 200) directly from the wearer when the wearer is looking at the target sound source of interest; storing, in the hearing assistance system (100, 200), a first three-dimensional position of the hearing assistance system (100, 200) in response to the first input; receiving, when the sound source in space is a noise source, a second input to the hearing assistance system (100, 200) directly from the wearer when the wearer is looking at the noise source; storing, in the hearing assistance system (100, 200), a second three-dimensional position of the hearing assistance system (100, 200) in response to the second received input; and adjusting a directional polar pattern of one or more microphones of the hearing assistance system (100, 200) in response to the stored first and second three-dimensional positions and in response to the tracked changes, wherein the directional polar pattern of the one or more microphones is adjusted in response to changes in the ITD and ILD to enhance reception of the target sound source of interest and attenuate reception of the noise source by the one or more microphones. determining, using one or more positional sensors of the hearing assistance system (100, 200), a three-dimensional position of the hearing assistance system (100, 200) in response to the wearer looking at a sound source in space; tracking changes in the three-dimensional position of one or both of the hearing assistance system (100, 200) and the sound source, wherein tracking the changes comprises: determining changes in an interaural time difference between the left ear device and the right ear device; and determining changes in an interaural level difference between the left ear device and the right ear device; receiving, when the sound source in space is a target sound source of interest, a first input to the hearing assistance system (100, 200) directly from the wearer when the wearer is looking at the target sound source of interest; storing, in the hearing assistance system (100, 200), a first three-dimensional position of the hearing assistance system (100, 200) in response to the first input; receiving, when the sound source in space is a noise source, a second input to the hearing assistance system (100, 200) directly from the wearer when the wearer is looking at the noise source; storing, in the hearing assistance system (100, 200), a second three-dimensional position of the hearing assistance system (100, 200) in response to the second received input; and adjusting a directional polar pattern of one or more microphones of the hearing assistance system (100, 200) in response to the stored first and second three-dimensional positions and in response to the tracked changes, wherein the directional polar pattern of the one or more microphones is adjusted in response to changes in the ITD and ILD to enhance reception of the target sound source of interest and attenuate reception of the noise source by the one or more microphones. determining changes in an interaural time difference between the left ear device and the right ear device; and determining changes in an interaural level difference between the left ear device and the right ear device; receiving, when the sound source in space is a target sound source of interest, a first input to the hearing assistance system (100, 200) directly from the wearer when the wearer is looking at the target sound source of interest; storing, in the hearing assistance system (100, 200), a first three-dimensional position of the hearing assistance system (100, 200) in response to the first input; receiving, when the sound source in space is a noise source, a second input to the hearing assistance system (100, 200) directly from the wearer when the wearer is looking at the noise source; storing, in the hearing assistance system (100, 200), a second three-dimensional position of the hearing assistance system (100, 200) in response to the second received input; and adjusting a directional polar pattern of one or more microphones of the hearing assistance system (100, 200) in response to the stored first and second three-dimensional positions and in response to the tracked changes, wherein the directional polar pattern of the one or more microphones is adjusted in response to changes in the ITD and ILD to enhance reception of the target sound source of interest and attenuate reception of the noise source by the one or more microphones.