Repository: numpy/numpy

Issue:
Support `norm="unnormalized"` for FFT functions?
I tried searching in the issue tracker here but didn't find any, so let me know if this is a duplicate.

This issue was first brought up in CuPy (cupy/cupy#944) and has regained some interest, and I believe it's better to be implemented/considered first in upstream, which downstream libraries like SciPy and CuPy can follow.

The desired effect is to do the normalization opposite to `norm=None`; that is, the backward/inverse transform is unscaled, and the forward transform is scaled by 1/n. For a perfectionist's point of view adding this option makes a lot of sense, as it would cover every possible normalization scheme.

In particular, for GPU libraries like CuPy this could be performance critical for certain applications, as kernel launching is more costly in the GPU world. People usually don't do extra work whenever possible, especially if the normalization is of no interest.

The name `"unnormalized"` can be changed to a more informative one. I don't have any preference (in fact I'm just a messenger bringing this to NumPy...)


Discussion:
Related PR and Issue: https://github.com/numpy/numpy/pull/3883 and https://github.com/numpy/numpy/issues/2142 , adding a new kwarg value for norm seems to be doable if there is a usecase for it.
cc @charris 
Checked the issue and I'm about to start working on it; is somebody else working on that at the moment? I understand that we work without assigning issues, so I thought I'd check with you first. 

I've been using the `fft` and `ifft` functions and am also interested in the normalization you're discussing; to do so up to now, I was using `norm=None` and then dividing/multiplying the returned vector manually by `n`.

I'm going to introduce an additional `norm` option for the new scaling; I was thinking of calling it `norm=inverse`, since it's the inverse scaling compared to `norm=None` (ie the forward transform is scaled with `1/n` and the backward with 1). But that's just a recommendation, let me know if you think something else would be better suited.
Hi @cvav-git I am not aware of anyone working on this. `norm=inverse` sounds fine to me (previously I was thinking `norm=full`, but yours seems more informative), but perhaps you should discuss the work plan with @anirudh2290 @charris?

cc: @mattvend who was working on the CuPy counterpart.
Thanks for your quick reply @leofang!

Hi @anirudh2290 @charris my current plan is to modify the `_unitary(norm)` flag to be `True` if `norm=ortho` and `False` if `norm=None` or `norm=inverse` (name still under discussion); this seems to make the smallest modifications to the current code, then I'll have to add an additional `if` clause in the various transform functions to distinguish between `norm=None` and `inverse` so that the scaling is done accordingly. 

Let me know if you have any thoughts or comments so that we try to find the best solution possible; and of course the new norm name is still open to discussion, I'm just using the tentative name `inverse` to proceed with the modifications. 

cc @mattvend

Resolving patch:
diff --git a/numpy/fft/__init__.py b/numpy/fft/__init__.py
--- a/numpy/fft/__init__.py
+++ b/numpy/fft/__init__.py
@@ -128,11 +128,17 @@
 Normalization
 -------------
 
-The default normalization has the direct transforms unscaled and the inverse
-transforms are scaled by :math:`1/n`. It is possible to obtain unitary
-transforms by setting the keyword argument ``norm`` to ``"ortho"`` (default is
-`None`) so that both direct and inverse transforms will be scaled by
-:math:`1/\\sqrt{n}`.
+The argument ``norm`` indicates which direction of the pair of direct/inverse
+transforms is scaled and with what normalization factor.
+The default normalization (``"backward"``) has the direct (forward) transforms
+unscaled and the inverse (backward) transforms scaled by :math:`1/n`. It is
+possible to obtain unitary transforms by setting the keyword argument ``norm``
+to ``"ortho"`` so that both direct and inverse transforms are scaled by
+:math:`1/\\sqrt{n}`. Finally, setting the keyword argument ``norm`` to
+``"forward"`` has the direct transforms scaled by :math:`1/n` and the inverse
+transforms unscaled (i.e. exactly opposite to the default ``"backward"``).
+`None` is an alias of the default option ``"backward"`` for backward
+compatibility.
 
 Real and Hermitian transforms
 -----------------------------
diff --git a/numpy/fft/_pocketfft.py b/numpy/fft/_pocketfft.py
--- a/numpy/fft/_pocketfft.py
+++ b/numpy/fft/_pocketfft.py
@@ -3,20 +3,20 @@
 
 Routines in this module:
 
-fft(a, n=None, axis=-1)
-ifft(a, n=None, axis=-1)
-rfft(a, n=None, axis=-1)
-irfft(a, n=None, axis=-1)
-hfft(a, n=None, axis=-1)
-ihfft(a, n=None, axis=-1)
-fftn(a, s=None, axes=None)
-ifftn(a, s=None, axes=None)
-rfftn(a, s=None, axes=None)
-irfftn(a, s=None, axes=None)
-fft2(a, s=None, axes=(-2,-1))
-ifft2(a, s=None, axes=(-2, -1))
-rfft2(a, s=None, axes=(-2,-1))
-irfft2(a, s=None, axes=(-2, -1))
+fft(a, n=None, axis=-1, norm="backward")
+ifft(a, n=None, axis=-1, norm="backward")
+rfft(a, n=None, axis=-1, norm="backward")
+irfft(a, n=None, axis=-1, norm="backward")
+hfft(a, n=None, axis=-1, norm="backward")
+ihfft(a, n=None, axis=-1, norm="backward")
+fftn(a, s=None, axes=None, norm="backward")
+ifftn(a, s=None, axes=None, norm="backward")
+rfftn(a, s=None, axes=None, norm="backward")
+irfftn(a, s=None, axes=None, norm="backward")
+fft2(a, s=None, axes=(-2,-1), norm="backward")
+ifft2(a, s=None, axes=(-2, -1), norm="backward")
+rfft2(a, s=None, axes=(-2,-1), norm="backward")
+irfft2(a, s=None, axes=(-2, -1), norm="backward")
 
 i = inverse transform
 r = transform of purely real data
@@ -51,10 +51,6 @@ def _raw_fft(a, n, axis, is_real, is_forward, inv_norm):
     if n is None:
         n = a.shape[axis]
 
-    if n < 1:
-        raise ValueError("Invalid number of FFT data points (%d) specified."
-                         % n)
-
     fct = 1/inv_norm
 
     if a.shape[axis] != n:
@@ -79,13 +75,44 @@ def _raw_fft(a, n, axis, is_real, is_forward, inv_norm):
     return r
 
 
-def _unitary(norm):
-    if norm is None:
-        return False
-    if norm=="ortho":
-        return True
-    raise ValueError("Invalid norm value %s, should be None or \"ortho\"."
-                     % norm)
+def _get_forward_norm(n, norm):
+    if n < 1:
+        raise ValueError(f"Invalid number of FFT data points ({n}) specified.")
+
+    if norm is None or norm == "backward":
+        return 1
+    elif norm == "ortho":
+        return sqrt(n)
+    elif norm == "forward":
+        return n
+    raise ValueError(f'Invalid norm value {norm}; should be "backward",'
+                     '"ortho" or "forward".')
+
+
+def _get_backward_norm(n, norm):
+    if n < 1:
+        raise ValueError(f"Invalid number of FFT data points ({n}) specified.")
+
+    if norm is None or norm == "backward":
+        return n
+    elif norm == "ortho":
+        return sqrt(n)
+    elif norm == "forward":
+        return 1
+    raise ValueError(f'Invalid norm value {norm}; should be "backward", '
+                     '"ortho" or "forward".')
+
+
+_SWAP_DIRECTION_MAP = {"backward": "forward", None: "forward",
+                       "ortho": "ortho", "forward": "backward"}
+
+
+def _swap_direction(norm):
+    try:
+        return _SWAP_DIRECTION_MAP[norm]
+    except KeyError:
+        raise ValueError(f'Invalid norm value {norm}; should be "backward", '
+                         '"ortho" or "forward".')
 
 
 def _fft_dispatcher(a, n=None, axis=None, norm=None):
@@ -113,10 +140,16 @@ def fft(a, n=None, axis=-1, norm=None):
     axis : int, optional
         Axis over which to compute the FFT.  If not given, the last axis is
         used.
-    norm : {None, "ortho"}, optional
+    norm : {"backward", "ortho", "forward"}, optional
         .. versionadded:: 1.10.0
 
-        Normalization mode (see `numpy.fft`). Default is None.
+        Normalization mode (see `numpy.fft`). Default is "backward".
+        Indicates which direction of the forward/backward pair of transforms
+        is scaled and with what normalization factor.
+
+        .. versionadded:: 1.20.0
+
+            The "backward", "forward" values were added.
 
     Returns
     -------
@@ -175,13 +208,10 @@ def fft(a, n=None, axis=-1, norm=None):
     >>> plt.show()
 
     """
-
     a = asarray(a)
     if n is None:
         n = a.shape[axis]
-    inv_norm = 1
-    if norm is not None and _unitary(norm):
-        inv_norm = sqrt(n)
+    inv_norm = _get_forward_norm(n, norm)
     output = _raw_fft(a, n, axis, False, True, inv_norm)
     return output
 
@@ -222,10 +252,16 @@ def ifft(a, n=None, axis=-1, norm=None):
     axis : int, optional
         Axis over which to compute the inverse DFT.  If not given, the last
         axis is used.
-    norm : {None, "ortho"}, optional
+    norm : {"backward", "ortho", "forward"}, optional
         .. versionadded:: 1.10.0
 
-        Normalization mode (see `numpy.fft`). Default is None.
+        Normalization mode (see `numpy.fft`). Default is "backward".
+        Indicates which direction of the forward/backward pair of transforms
+        is scaled and with what normalization factor.
+
+        .. versionadded:: 1.20.0
+
+            The "backward", "forward" values were added.
 
     Returns
     -------
@@ -274,15 +310,11 @@ def ifft(a, n=None, axis=-1, norm=None):
     a = asarray(a)
     if n is None:
         n = a.shape[axis]
-    if norm is not None and _unitary(norm):
-        inv_norm = sqrt(max(n, 1))
-    else:
-        inv_norm = n
+    inv_norm = _get_backward_norm(n, norm)
     output = _raw_fft(a, n, axis, False, False, inv_norm)
     return output
 
 
-
 @array_function_dispatch(_fft_dispatcher)
 def rfft(a, n=None, axis=-1, norm=None):
     """
@@ -304,10 +336,16 @@ def rfft(a, n=None, axis=-1, norm=None):
     axis : int, optional
         Axis over which to compute the FFT. If not given, the last axis is
         used.
-    norm : {None, "ortho"}, optional
+    norm : {"backward", "ortho", "forward"}, optional
         .. versionadded:: 1.10.0
 
-        Normalization mode (see `numpy.fft`). Default is None.
+        Normalization mode (see `numpy.fft`). Default is "backward".
+        Indicates which direction of the forward/backward pair of transforms
+        is scaled and with what normalization factor.
+
+        .. versionadded:: 1.20.0
+
+            The "backward", "forward" values were added.
 
     Returns
     -------
@@ -363,11 +401,9 @@ def rfft(a, n=None, axis=-1, norm=None):
 
     """
     a = asarray(a)
-    inv_norm = 1
-    if norm is not None and _unitary(norm):
-        if n is None:
-            n = a.shape[axis]
-        inv_norm = sqrt(n)
+    if n is None:
+        n = a.shape[axis]
+    inv_norm = _get_forward_norm(n, norm)
     output = _raw_fft(a, n, axis, True, True, inv_norm)
     return output
 
@@ -402,10 +438,16 @@ def irfft(a, n=None, axis=-1, norm=None):
     axis : int, optional
         Axis over which to compute the inverse FFT. If not given, the last
         axis is used.
-    norm : {None, "ortho"}, optional
+    norm : {"backward", "ortho", "forward"}, optional
         .. versionadded:: 1.10.0
 
-        Normalization mode (see `numpy.fft`). Default is None.
+        Normalization mode (see `numpy.fft`). Default is "backward".
+        Indicates which direction of the forward/backward pair of transforms
+        is scaled and with what normalization factor.
+
+        .. versionadded:: 1.20.0
+
+            The "backward", "forward" values were added.
 
     Returns
     -------
@@ -465,9 +507,7 @@ def irfft(a, n=None, axis=-1, norm=None):
     a = asarray(a)
     if n is None:
         n = (a.shape[axis] - 1) * 2
-    inv_norm = n
-    if norm is not None and _unitary(norm):
-        inv_norm = sqrt(n)
+    inv_norm = _get_backward_norm(n, norm)
     output = _raw_fft(a, n, axis, True, False, inv_norm)
     return output
 
@@ -492,11 +532,17 @@ def hfft(a, n=None, axis=-1, norm=None):
     axis : int, optional
         Axis over which to compute the FFT. If not given, the last
         axis is used.
-    norm : {None, "ortho"}, optional
-        Normalization mode (see `numpy.fft`). Default is None.
-
+    norm : {"backward", "ortho", "forward"}, optional
         .. versionadded:: 1.10.0
 
+        Normalization mode (see `numpy.fft`). Default is "backward".
+        Indicates which direction of the forward/backward pair of transforms
+        is scaled and with what normalization factor.
+
+        .. versionadded:: 1.20.0
+
+            The "backward", "forward" values were added.
+
     Returns
     -------
     out : ndarray
@@ -559,8 +605,9 @@ def hfft(a, n=None, axis=-1, norm=None):
     a = asarray(a)
     if n is None:
         n = (a.shape[axis] - 1) * 2
-    unitary = _unitary(norm)
-    return irfft(conjugate(a), n, axis) * (sqrt(n) if unitary else n)
+    new_norm = _swap_direction(norm)
+    output = irfft(conjugate(a), n, axis, norm=new_norm)
+    return output
 
 
 @array_function_dispatch(_fft_dispatcher)
@@ -581,11 +628,17 @@ def ihfft(a, n=None, axis=-1, norm=None):
     axis : int, optional
         Axis over which to compute the inverse FFT. If not given, the last
         axis is used.
-    norm : {None, "ortho"}, optional
-        Normalization mode (see `numpy.fft`). Default is None.
-
+    norm : {"backward", "ortho", "forward"}, optional
         .. versionadded:: 1.10.0
 
+        Normalization mode (see `numpy.fft`). Default is "backward".
+        Indicates which direction of the forward/backward pair of transforms
+        is scaled and with what normalization factor.
+
+        .. versionadded:: 1.20.0
+
+            The "backward", "forward" values were added.
+
     Returns
     -------
     out : complex ndarray
@@ -619,9 +672,9 @@ def ihfft(a, n=None, axis=-1, norm=None):
     a = asarray(a)
     if n is None:
         n = a.shape[axis]
-    unitary = _unitary(norm)
-    output = conjugate(rfft(a, n, axis))
-    return output * (1 / (sqrt(n) if unitary else n))
+    new_norm = _swap_direction(norm)
+    output = conjugate(rfft(a, n, axis, norm=new_norm))
+    return output
 
 
 def _cook_nd_args(a, s=None, axes=None, invreal=0):
@@ -683,10 +736,16 @@ def fftn(a, s=None, axes=None, norm=None):
         axes are used, or all axes if `s` is also not specified.
         Repeated indices in `axes` means that the transform over that axis is
         performed multiple times.
-    norm : {None, "ortho"}, optional
+    norm : {"backward", "ortho", "forward"}, optional
         .. versionadded:: 1.10.0
 
-        Normalization mode (see `numpy.fft`). Default is None.
+        Normalization mode (see `numpy.fft`). Default is "backward".
+        Indicates which direction of the forward/backward pair of transforms
+        is scaled and with what normalization factor.
+
+        .. versionadded:: 1.20.0
+
+            The "backward", "forward" values were added.
 
     Returns
     -------
@@ -751,7 +810,6 @@ def fftn(a, s=None, axes=None, norm=None):
     >>> plt.show()
 
     """
-
     return _raw_fftnd(a, s, axes, fft, norm)
 
 
@@ -790,10 +848,16 @@ def ifftn(a, s=None, axes=None, norm=None):
         axes are used, or all axes if `s` is also not specified.
         Repeated indices in `axes` means that the inverse transform over that
         axis is performed multiple times.
-    norm : {None, "ortho"}, optional
+    norm : {"backward", "ortho", "forward"}, optional
         .. versionadded:: 1.10.0
 
-        Normalization mode (see `numpy.fft`). Default is None.
+        Normalization mode (see `numpy.fft`). Default is "backward".
+        Indicates which direction of the forward/backward pair of transforms
+        is scaled and with what normalization factor.
+
+        .. versionadded:: 1.20.0
+
+            The "backward", "forward" values were added.
 
     Returns
     -------
@@ -849,14 +913,13 @@ def ifftn(a, s=None, axes=None, norm=None):
     >>> plt.show()
 
     """
-
     return _raw_fftnd(a, s, axes, ifft, norm)
 
 
 @array_function_dispatch(_fftn_dispatcher)
 def fft2(a, s=None, axes=(-2, -1), norm=None):
     """
-    Compute the 2-dimensional discrete Fourier Transform
+    Compute the 2-dimensional discrete Fourier Transform.
 
     This function computes the *n*-dimensional discrete Fourier Transform
     over any axes in an *M*-dimensional array by means of the
@@ -880,10 +943,16 @@ def fft2(a, s=None, axes=(-2, -1), norm=None):
         axes are used.  A repeated index in `axes` means the transform over
         that axis is performed multiple times.  A one-element sequence means
         that a one-dimensional FFT is performed.
-    norm : {None, "ortho"}, optional
+    norm : {"backward", "ortho", "forward"}, optional
         .. versionadded:: 1.10.0
 
-        Normalization mode (see `numpy.fft`). Default is None.
+        Normalization mode (see `numpy.fft`). Default is "backward".
+        Indicates which direction of the forward/backward pair of transforms
+        is scaled and with what normalization factor.
+
+        .. versionadded:: 1.20.0
+
+            The "backward", "forward" values were added.
 
     Returns
     -------
@@ -940,7 +1009,6 @@ def fft2(a, s=None, axes=(-2, -1), norm=None):
               0.  +0.j        ,   0.  +0.j        ]])
 
     """
-
     return _raw_fftnd(a, s, axes, fft, norm)
 
 
@@ -978,10 +1046,16 @@ def ifft2(a, s=None, axes=(-2, -1), norm=None):
         axes are used.  A repeated index in `axes` means the transform over
         that axis is performed multiple times.  A one-element sequence means
         that a one-dimensional FFT is performed.
-    norm : {None, "ortho"}, optional
+    norm : {"backward", "ortho", "forward"}, optional
         .. versionadded:: 1.10.0
 
-        Normalization mode (see `numpy.fft`). Default is None.
+        Normalization mode (see `numpy.fft`). Default is "backward".
+        Indicates which direction of the forward/backward pair of transforms
+        is scaled and with what normalization factor.
+
+        .. versionadded:: 1.20.0
+
+            The "backward", "forward" values were added.
 
     Returns
     -------
@@ -1028,7 +1102,6 @@ def ifft2(a, s=None, axes=(-2, -1), norm=None):
            [0.+0.j,  1.+0.j,  0.+0.j,  0.+0.j]])
 
     """
-
     return _raw_fftnd(a, s, axes, ifft, norm)
 
 
@@ -1059,10 +1132,16 @@ def rfftn(a, s=None, axes=None, norm=None):
     axes : sequence of ints, optional
         Axes over which to compute the FFT.  If not given, the last ``len(s)``
         axes are used, or all axes if `s` is also not specified.
-    norm : {None, "ortho"}, optional
+    norm : {"backward", "ortho", "forward"}, optional
         .. versionadded:: 1.10.0
 
-        Normalization mode (see `numpy.fft`). Default is None.
+        Normalization mode (see `numpy.fft`). Default is "backward".
+        Indicates which direction of the forward/backward pair of transforms
+        is scaled and with what normalization factor.
+
+        .. versionadded:: 1.20.0
+
+            The "backward", "forward" values were added.
 
     Returns
     -------
@@ -1137,10 +1216,16 @@ def rfft2(a, s=None, axes=(-2, -1), norm=None):
         Shape of the FFT.
     axes : sequence of ints, optional
         Axes over which to compute the FFT.
-    norm : {None, "ortho"}, optional
+    norm : {"backward", "ortho", "forward"}, optional
         .. versionadded:: 1.10.0
 
-        Normalization mode (see `numpy.fft`). Default is None.
+        Normalization mode (see `numpy.fft`). Default is "backward".
+        Indicates which direction of the forward/backward pair of transforms
+        is scaled and with what normalization factor.
+
+        .. versionadded:: 1.20.0
+
+            The "backward", "forward" values were added.
 
     Returns
     -------
@@ -1158,7 +1243,6 @@ def rfft2(a, s=None, axes=(-2, -1), norm=None):
     For more details see `rfftn`.
 
     """
-
     return rfftn(a, s, axes, norm)
 
 
@@ -1190,17 +1274,23 @@ def irfftn(a, s=None, axes=None, norm=None):
         Along any axis, if the shape indicated by `s` is smaller than that of
         the input, the input is cropped.  If it is larger, the input is padded
         with zeros. If `s` is not given, the shape of the input along the axes
-        specified by axes is used. Except for the last axis which is taken to be
-        ``2*(m-1)`` where ``m`` is the length of the input along that axis.
+        specified by axes is used. Except for the last axis which is taken to
+        be ``2*(m-1)`` where ``m`` is the length of the input along that axis.
     axes : sequence of ints, optional
         Axes over which to compute the inverse FFT. If not given, the last
         `len(s)` axes are used, or all axes if `s` is also not specified.
         Repeated indices in `axes` means that the inverse transform over that
         axis is performed multiple times.
-    norm : {None, "ortho"}, optional
+    norm : {"backward", "ortho", "forward"}, optional
         .. versionadded:: 1.10.0
 
-        Normalization mode (see `numpy.fft`). Default is None.
+        Normalization mode (see `numpy.fft`). Default is "backward".
+        Indicates which direction of the forward/backward pair of transforms
+        is scaled and with what normalization factor.
+
+        .. versionadded:: 1.20.0
+
+            The "backward", "forward" values were added.
 
     Returns
     -------
@@ -1280,10 +1370,16 @@ def irfft2(a, s=None, axes=(-2, -1), norm=None):
     axes : sequence of ints, optional
         The axes over which to compute the inverse fft.
         Default is the last two axes.
-    norm : {None, "ortho"}, optional
+    norm : {"backward", "ortho", "forward"}, optional
         .. versionadded:: 1.10.0
 
-        Normalization mode (see `numpy.fft`). Default is None.
+        Normalization mode (see `numpy.fft`). Default is "backward".
+        Indicates which direction of the forward/backward pair of transforms
+        is scaled and with what normalization factor.
+
+        .. versionadded:: 1.20.0
+
+            The "backward", "forward" values were added.
 
     Returns
     -------
@@ -1300,5 +1396,4 @@ def irfft2(a, s=None, axes=(-2, -1), norm=None):
     For more details see `irfftn`.
 
     """
-
     return irfftn(a, s, axes, norm)


Test patch:
diff --git a/numpy/fft/tests/test_pocketfft.py b/numpy/fft/tests/test_pocketfft.py
--- a/numpy/fft/tests/test_pocketfft.py
+++ b/numpy/fft/tests/test_pocketfft.py
@@ -27,19 +27,22 @@ def test_identity(self):
         maxlen = 512
         x = random(maxlen) + 1j*random(maxlen)
         xr = random(maxlen)
-        for i in range(1,maxlen):
+        for i in range(1, maxlen):
             assert_allclose(np.fft.ifft(np.fft.fft(x[0:i])), x[0:i],
                             atol=1e-12)
-            assert_allclose(np.fft.irfft(np.fft.rfft(xr[0:i]),i),
+            assert_allclose(np.fft.irfft(np.fft.rfft(xr[0:i]), i),
                             xr[0:i], atol=1e-12)
 
     def test_fft(self):
         x = random(30) + 1j*random(30)
         assert_allclose(fft1(x), np.fft.fft(x), atol=1e-6)
+        assert_allclose(fft1(x), np.fft.fft(x, norm="backward"), atol=1e-6)
         assert_allclose(fft1(x) / np.sqrt(30),
                         np.fft.fft(x, norm="ortho"), atol=1e-6)
+        assert_allclose(fft1(x) / 30.,
+                        np.fft.fft(x, norm="forward"), atol=1e-6)
 
-    @pytest.mark.parametrize('norm', (None, 'ortho'))
+    @pytest.mark.parametrize('norm', (None, 'backward', 'ortho', 'forward'))
     def test_ifft(self, norm):
         x = random(30) + 1j*random(30)
         assert_allclose(
@@ -54,89 +57,138 @@ def test_fft2(self):
         x = random((30, 20)) + 1j*random((30, 20))
         assert_allclose(np.fft.fft(np.fft.fft(x, axis=1), axis=0),
                         np.fft.fft2(x), atol=1e-6)
+        assert_allclose(np.fft.fft2(x),
+                        np.fft.fft2(x, norm="backward"), atol=1e-6)
         assert_allclose(np.fft.fft2(x) / np.sqrt(30 * 20),
                         np.fft.fft2(x, norm="ortho"), atol=1e-6)
+        assert_allclose(np.fft.fft2(x) / (30. * 20.),
+                        np.fft.fft2(x, norm="forward"), atol=1e-6)
 
     def test_ifft2(self):
         x = random((30, 20)) + 1j*random((30, 20))
         assert_allclose(np.fft.ifft(np.fft.ifft(x, axis=1), axis=0),
                         np.fft.ifft2(x), atol=1e-6)
+        assert_allclose(np.fft.ifft2(x),
+                        np.fft.ifft2(x, norm="backward"), atol=1e-6)
         assert_allclose(np.fft.ifft2(x) * np.sqrt(30 * 20),
                         np.fft.ifft2(x, norm="ortho"), atol=1e-6)
+        assert_allclose(np.fft.ifft2(x) * (30. * 20.),
+                        np.fft.ifft2(x, norm="forward"), atol=1e-6)
 
     def test_fftn(self):
         x = random((30, 20, 10)) + 1j*random((30, 20, 10))
         assert_allclose(
             np.fft.fft(np.fft.fft(np.fft.fft(x, axis=2), axis=1), axis=0),
             np.fft.fftn(x), atol=1e-6)
+        assert_allclose(np.fft.fftn(x),
+                        np.fft.fftn(x, norm="backward"), atol=1e-6)
         assert_allclose(np.fft.fftn(x) / np.sqrt(30 * 20 * 10),
                         np.fft.fftn(x, norm="ortho"), atol=1e-6)
+        assert_allclose(np.fft.fftn(x) / (30. * 20. * 10.),
+                        np.fft.fftn(x, norm="forward"), atol=1e-6)
 
     def test_ifftn(self):
         x = random((30, 20, 10)) + 1j*random((30, 20, 10))
         assert_allclose(
             np.fft.ifft(np.fft.ifft(np.fft.ifft(x, axis=2), axis=1), axis=0),
             np.fft.ifftn(x), atol=1e-6)
+        assert_allclose(np.fft.ifftn(x),
+                        np.fft.ifftn(x, norm="backward"), atol=1e-6)
         assert_allclose(np.fft.ifftn(x) * np.sqrt(30 * 20 * 10),
                         np.fft.ifftn(x, norm="ortho"), atol=1e-6)
+        assert_allclose(np.fft.ifftn(x) * (30. * 20. * 10.),
+                        np.fft.ifftn(x, norm="forward"), atol=1e-6)
 
     def test_rfft(self):
         x = random(30)
         for n in [x.size, 2*x.size]:
-            for norm in [None, 'ortho']:
+            for norm in [None, 'backward', 'ortho', 'forward']:
                 assert_allclose(
                     np.fft.fft(x, n=n, norm=norm)[:(n//2 + 1)],
                     np.fft.rfft(x, n=n, norm=norm), atol=1e-6)
+            assert_allclose(
+                np.fft.rfft(x, n=n),
+                np.fft.rfft(x, n=n, norm="backward"), atol=1e-6)
             assert_allclose(
                 np.fft.rfft(x, n=n) / np.sqrt(n),
                 np.fft.rfft(x, n=n, norm="ortho"), atol=1e-6)
+            assert_allclose(
+                np.fft.rfft(x, n=n) / n,
+                np.fft.rfft(x, n=n, norm="forward"), atol=1e-6)
 
     def test_irfft(self):
         x = random(30)
         assert_allclose(x, np.fft.irfft(np.fft.rfft(x)), atol=1e-6)
-        assert_allclose(
-            x, np.fft.irfft(np.fft.rfft(x, norm="ortho"), norm="ortho"), atol=1e-6)
+        assert_allclose(x, np.fft.irfft(np.fft.rfft(x, norm="backward"),
+                        norm="backward"), atol=1e-6)
+        assert_allclose(x, np.fft.irfft(np.fft.rfft(x, norm="ortho"),
+                        norm="ortho"), atol=1e-6)
+        assert_allclose(x, np.fft.irfft(np.fft.rfft(x, norm="forward"),
+                        norm="forward"), atol=1e-6)
 
     def test_rfft2(self):
         x = random((30, 20))
         assert_allclose(np.fft.fft2(x)[:, :11], np.fft.rfft2(x), atol=1e-6)
+        assert_allclose(np.fft.rfft2(x),
+                        np.fft.rfft2(x, norm="backward"), atol=1e-6)
         assert_allclose(np.fft.rfft2(x) / np.sqrt(30 * 20),
                         np.fft.rfft2(x, norm="ortho"), atol=1e-6)
+        assert_allclose(np.fft.rfft2(x) / (30. * 20.),
+                        np.fft.rfft2(x, norm="forward"), atol=1e-6)
 
     def test_irfft2(self):
         x = random((30, 20))
         assert_allclose(x, np.fft.irfft2(np.fft.rfft2(x)), atol=1e-6)
-        assert_allclose(
-            x, np.fft.irfft2(np.fft.rfft2(x, norm="ortho"), norm="ortho"), atol=1e-6)
+        assert_allclose(x, np.fft.irfft2(np.fft.rfft2(x, norm="backward"),
+                        norm="backward"), atol=1e-6)
+        assert_allclose(x, np.fft.irfft2(np.fft.rfft2(x, norm="ortho"),
+                        norm="ortho"), atol=1e-6)
+        assert_allclose(x, np.fft.irfft2(np.fft.rfft2(x, norm="forward"),
+                        norm="forward"), atol=1e-6)
 
     def test_rfftn(self):
         x = random((30, 20, 10))
         assert_allclose(np.fft.fftn(x)[:, :, :6], np.fft.rfftn(x), atol=1e-6)
+        assert_allclose(np.fft.rfftn(x),
+                        np.fft.rfftn(x, norm="backward"), atol=1e-6)
         assert_allclose(np.fft.rfftn(x) / np.sqrt(30 * 20 * 10),
                         np.fft.rfftn(x, norm="ortho"), atol=1e-6)
+        assert_allclose(np.fft.rfftn(x) / (30. * 20. * 10.),
+                        np.fft.rfftn(x, norm="forward"), atol=1e-6)
 
     def test_irfftn(self):
         x = random((30, 20, 10))
         assert_allclose(x, np.fft.irfftn(np.fft.rfftn(x)), atol=1e-6)
-        assert_allclose(
-            x, np.fft.irfftn(np.fft.rfftn(x, norm="ortho"), norm="ortho"), atol=1e-6)
+        assert_allclose(x, np.fft.irfftn(np.fft.rfftn(x, norm="backward"),
+                        norm="backward"), atol=1e-6)
+        assert_allclose(x, np.fft.irfftn(np.fft.rfftn(x, norm="ortho"),
+                        norm="ortho"), atol=1e-6)
+        assert_allclose(x, np.fft.irfftn(np.fft.rfftn(x, norm="forward"),
+                        norm="forward"), atol=1e-6)
 
     def test_hfft(self):
         x = random(14) + 1j*random(14)
         x_herm = np.concatenate((random(1), x, random(1)))
         x = np.concatenate((x_herm, x[::-1].conj()))
         assert_allclose(np.fft.fft(x), np.fft.hfft(x_herm), atol=1e-6)
+        assert_allclose(np.fft.hfft(x_herm),
+                        np.fft.hfft(x_herm, norm="backward"), atol=1e-6)
         assert_allclose(np.fft.hfft(x_herm) / np.sqrt(30),
                         np.fft.hfft(x_herm, norm="ortho"), atol=1e-6)
+        assert_allclose(np.fft.hfft(x_herm) / 30.,
+                        np.fft.hfft(x_herm, norm="forward"), atol=1e-6)
 
-    def test_ihttf(self):
+    def test_ihfft(self):
         x = random(14) + 1j*random(14)
         x_herm = np.concatenate((random(1), x, random(1)))
         x = np.concatenate((x_herm, x[::-1].conj()))
         assert_allclose(x_herm, np.fft.ihfft(np.fft.hfft(x_herm)), atol=1e-6)
-        assert_allclose(
-            x_herm, np.fft.ihfft(np.fft.hfft(x_herm, norm="ortho"),
-                                 norm="ortho"), atol=1e-6)
+        assert_allclose(x_herm, np.fft.ihfft(np.fft.hfft(x_herm,
+                        norm="backward"), norm="backward"), atol=1e-6)
+        assert_allclose(x_herm, np.fft.ihfft(np.fft.hfft(x_herm,
+                        norm="ortho"), norm="ortho"), atol=1e-6)
+        assert_allclose(x_herm, np.fft.ihfft(np.fft.hfft(x_herm,
+                        norm="forward"), norm="forward"), atol=1e-6)
 
     @pytest.mark.parametrize("op", [np.fft.fftn, np.fft.ifftn,
                                     np.fft.rfftn, np.fft.irfftn])
@@ -161,7 +213,7 @@ def test_all_1d_norm_preserving(self):
                       ]
         for forw, back in func_pairs:
             for n in [x.size, 2*x.size]:
-                for norm in [None, 'ortho']:
+                for norm in [None, 'backward', 'ortho', 'forward']:
                     tmp = forw(x, n=n, norm=norm)
                     tmp = back(tmp, n=n, norm=norm)
                     assert_allclose(x_norm,