Repository: statsmodels/statsmodels
Pull request: ENH: Add fixed_params to Hannan Rissanen (GH7202)
- [x] closes #7202
- [x] xref #6159
- [x] tests added / passed. 
- [x] code/documentation is well formatted.  
- [x] properly formatted commit message. See 
      [NumPy's guide](https://docs.scipy.org/doc/numpy-1.15.1/dev/gitwash/development_workflow.html#writing-the-commit-message). 

<details>

**Notes**:

* It is essential that you add a test when making code changes. Tests are not 
  needed for doc changes.
* When adding a new function, test values should usually be verified in another package (e.g., R/SAS/Stata).
* When fixing a bug, you must add a test that would produce the bug in main and
  then show that it is fixed with the new code.
* New code additions must be well formatted. Changes should pass flake8. If on Linux or OSX, you can
  verify you changes are well formatted by running 
  ```
  git diff upstream/main -u -- "*.py" | flake8 --diff --isolated
  ```
  assuming `flake8` is installed. This command is also available on Windows 
  using the Windows System for Linux once `flake8` is installed in the 
  local Linux environment. While passing this test is not required, it is good practice and it help 
  improve code quality in `statsmodels`.
* Docstring additions must render correctly, including escapes and LaTeX.

</details>


Problem statement:
ENH: Fixed parameters in Hannan-Rissanen
#### Is your feature request related to a problem? Please describe
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

#### Describe the solution you'd like
A clear and concise description of what you want to happen.

#### Describe alternatives you have considered
A clear and concise description of any alternative solutions or features you have considered.

#### Additional context
Add any other context about the feature request here.

Hints from issue discussion:
Assigning the enhancement to myself.
Thanks for opening this issue. How are things going?
For reference, here is the note about this PR from a previous issue, #6159:

>> Hi Chad,
>I would like to pick up the "Support for fixed parameters". would you please share some more details on this enhancement?
>
>That would be much appreciated, thanks!
>
>First, a little background (maybe you already know this, but just in case you haven't run into this feature before). In our models, all of the parameters are typically estimated by maximum likelihood. But in some cases, you might want to "fix" one of the parameters to a particular value, and then estimate the *other* parameters by maximum likelihood. A simple example is:
>
>```python
>endog = np.random.normal(size=100)
>
># AR(1) model with an intercept
>mod = sm.tsa.SARIMAX(endog, order=(1, 0, 0), trend='c')
>
># Suppose we want to estimate the AR(1) coefficient, but we want to specify the intercept to be 0.5
>with mod.fix_params({'intercept': 0.5}):
>    res = mod.fit()
>print(res.summary())
>```
>
>this results in:
>
>```
>                               SARIMAX Results                                
>==============================================================================
>Dep. Variable:                      y   No. Observations:                  100
>Model:               SARIMAX(1, 0, 0)   Log Likelihood                -146.547
>Date:                Thu, 10 Dec 2020   AIC                            297.094
>Time:                        19:44:35   BIC                            302.305
>Sample:                             0   HQIC                           299.203
>                                - 100                                         
>Covariance Type:                  opg                                         
>=====================================================================================
>                        coef    std err          z      P>|z|      [0.025      0.975]
>-------------------------------------------------------------------------------------
>intercept (fixed)     0.5000        nan        nan        nan         nan         nan
>ar.L1                -0.1595      0.096     -1.655      0.098      -0.348       0.029
>sigma2                1.0972      0.144      7.631      0.000       0.815       1.379
>===================================================================================
>Ljung-Box (L1) (Q):                   0.13   Jarque-Bera (JB):                 2.30
>Prob(Q):                              0.72   Prob(JB):                         0.32
>Heteroskedasticity (H):               1.26   Skew:                            -0.37
>Prob(H) (two-sided):                  0.50   Kurtosis:                         2.97
>===================================================================================
>
>Warnings:
>[1] Covariance matrix calculated using the outer product of gradients (complex-step).
>```
>
>A more complete set of examples can be found in this notebook: https://www.statsmodels.org/stable/examples/notebooks/generated/statespace_fixed_params.html.
>
>This feature is available in state space models (and also in the new `ETSModel` class), but it's not yet been implemented for all of the new ARIMA-type estimators referenced here (including Innovations MLE, Hannan-Rissanen, and GLS).
>
>I think the easiest place to get started would be adding fixed parameters to the Hannan-Rissanen estimator, which is the `hannan_rissanen` function in the file `statsmodels/tsa/arima/estimators/hannan_rissanen.py`. The basic idea here, is that ultimately the parameters are estimated by least squares (see lines 121 and 141), and so fixing certain parameter values is straightforward.
>
>To see why it is straightforward with OLS, consider the following regression equation:
>
>`y = a + b x_1 + c x_2 + e`
>
>If we want to fix `b = 2`, then the equation is `y = a + 2 x_1 + c x_2 + e` or we can rewrite it as a different regression
>
>`z = a + c x_2 + e`
>
>where we have created the new variable `z` according to `z = y - 2 x_1`.
>
>So basically, what I suggest is the following:
>
>1. Create a new Github issue (e.g. `ENH: Fixed parameters in Hannan-Rissanen`) for a more convenient location for discussion and advice going forwards.
>
>2. Add a new argument `fixed_params` to `hannan_rissanen`, which should accept a dictionary with parameter names as keys and the fixed numbers as values.
>
>3. In each case (lines 121 and 141), if there are fixed parameters, you need to remove the columns that correspond to those parameters from the `exog` arrays being passed to `OLS` (e.g. in line 121, the `lagged_endog` variable is passed as the `exog` argument), multiply them by the given fixed parameter value, and then subtract that to create a new endogenous variable (i.e. like the `z` I mentioned above).
>
>Also, for the first attempt, I would just raise a `NotImplementedError` if `unbiased=True` (since I haven't really thought about fixed parameter values in this case).
>
>Thanks!
> Hi Chad, could you please help me how to identify the corresponding column when removing the parameter from the 2d array?

Sure! So to start, I would pull the `exog` array out of the `OLS` call, like:

```
exog = np.c_[lagged_endog[ix:], lagged_resid]
```

Now the `fixed_params` argument should be a dictionary with parameter names as keys and the fixed numbers as values, so you need to map parameter names to columns of `exog`.

The parameter names associated with the columns of this full `exog` array are given by `spec.param_names` (where `spec` is defined in line 91).

So for each `key, value` pair in `fixed_params`, you just need to get the index of `key` in `spec.param_names`, and that will tell you the column in `exog`.
Hi Chad, 
after subtraction, the array should be passed as the new exog to the OLS right?
You'll be creating both a new `endog` and a new `exog`:

- The new `endog` is the `z` I described in my earlier post, so it will end up being something like `new_endog = endog - fixed_params['ar.L1'] * exog[:, 0] - ...`
- The new `exog` is the original `exog` but with fewer columns, since you need to remove the columns associated with the fixed parameters (these are the same columns that were subtracted from `endog`)

Hope that helps
Hi Chad, I have made the required changed,,,, kindly review once
I think you still need to create a pull request so that we can review your code.

Commit message: ENH: Add fixed_params to Hannan Rissanen (GH7202)

Patch to review:
diff --git a/statsmodels/tsa/arima/estimators/hannan_rissanen.py b/statsmodels/tsa/arima/estimators/hannan_rissanen.py
index 77eff642373..ce02b51cd2e 100644
--- a/statsmodels/tsa/arima/estimators/hannan_rissanen.py
+++ b/statsmodels/tsa/arima/estimators/hannan_rissanen.py
@@ -16,7 +16,8 @@
 
 
 def hannan_rissanen(endog, ar_order=0, ma_order=0, demean=True,
-                    initial_ar_order=None, unbiased=None):
+                    initial_ar_order=None, unbiased=None,
+                    fixed_params=None):
     """
     Estimate ARMA parameters using Hannan-Rissanen procedure.
 
@@ -24,9 +25,9 @@ def hannan_rissanen(endog, ar_order=0, ma_order=0, demean=True,
     ----------
     endog : array_like
         Input time series array, assumed to be stationary.
-    ar_order : int
+    ar_order : int or list of int
         Autoregressive order
-    ma_order : int
+    ma_order : int or list of int
         Moving average order
     demean : bool, optional
         Whether to estimate and remove the mean from the process prior to
@@ -38,6 +39,10 @@ def hannan_rissanen(endog, ar_order=0, ma_order=0, demean=True,
         Whether or not to apply the bias correction step. Default is True if
         the estimated coefficients from the previous step imply a stationary
         and invertible process and False otherwise.
+    fixed_params : dict, optional
+        Dictionary with names of fixed parameters as keys (e.g. 'ar.L1',
+        'ma.L2'), which correspond to SARIMAXSpecification.param_names.
+        Dictionary values are the values of the associated fixed parameters.
 
     Returns
     -------
@@ -89,6 +94,9 @@ def hannan_rissanen(endog, ar_order=0, ma_order=0, demean=True,
        A Course in Time Series Analysis, 171–201.
     """
     spec = SARIMAXSpecification(endog, ar_order=ar_order, ma_order=ma_order)
+
+    fixed_params = _validate_fixed_params(fixed_params, spec.param_names)
+
     endog = spec.endog
     if demean:
         endog = endog - endog.mean()
@@ -106,11 +114,13 @@ def hannan_rissanen(endog, ar_order=0, ma_order=0, demean=True,
     # Create a spec, just to validate the initial autoregressive order
     _ = SARIMAXSpecification(endog, ar_order=initial_ar_order)
 
+    # Unpack fixed and free ar/ma lags, ix, and params (fixed only)
+    params_info = _package_fixed_and_free_params_info(
+        fixed_params, spec.ar_lags, spec.ma_lags
+    )
+
     # Compute lagged endog
-    # (`ar_ix`, and `ma_ix` below, are to account for non-consecutive lags;
-    # for indexing purposes, must have dtype int)
-    ar_ix = np.array(spec.ar_lags, dtype=int) - 1
-    lagged_endog = lagmat(endog, max_ar_order, trim='both')[:, ar_ix]
+    lagged_endog = lagmat(endog, max_ar_order, trim='both')
 
     # If no AR or MA components, this is just a variance computation
     if max_ma_order == 0 and max_ar_order == 0:
@@ -118,11 +128,32 @@ def hannan_rissanen(endog, ar_order=0, ma_order=0, demean=True,
         resid = endog.copy()
     # If no MA component, this is just CSS
     elif max_ma_order == 0:
-        mod = OLS(endog[max_ar_order:], lagged_endog)
-        res = mod.fit()
-        resid = res.resid
-        p.ar_params = res.params
-        p.sigma2 = res.scale
+        # extract 1) lagged_endog with free params; 2) lagged_endog with fixed
+        # params; 3) endog residual after applying fixed params if applicable
+        X_with_free_params = lagged_endog[:, params_info.free_ar_ix]
+        X_with_fixed_params = lagged_endog[:, params_info.fixed_ar_ix]
+        y = endog[max_ar_order:]
+        if X_with_fixed_params.shape[1] != 0:
+            y -= X_with_fixed_params.dot(params_info.fixed_ar_params)
+
+        # no free ar params -> variance computation on the endog residual
+        if X_with_free_params.shape[1] == 0:
+            p.ar_params = params_info.fixed_ar_params
+            p.sigma2 = np.var(y, ddof=0)
+            resid = y.copy()
+        # otherwise OLS with endog residual (after applying fixed params) as y,
+        # and lagged_endog with free params as X
+        else:
+            mod = OLS(y, X_with_free_params)
+            res = mod.fit()
+            resid = res.resid
+            p.sigma2 = res.scale
+            p.ar_params = _stitch_fixed_and_free_params(
+                fixed_ar_or_ma_lags=params_info.fixed_ar_lags,
+                fixed_ar_or_ma_params=params_info.fixed_ar_params,
+                free_ar_or_ma_lags=params_info.free_ar_lags,
+                free_ar_or_ma_params=res.params
+            )
     # Otherwise ARMA model
     else:
         # Step 1: Compute long AR model via Yule-Walker, get residuals
@@ -133,22 +164,57 @@ def hannan_rissanen(endog, ar_order=0, ma_order=0, demean=True,
         resid = y - X.dot(initial_ar_params)
 
         # Get lagged residuals for `exog` in least-squares regression
-        ma_ix = np.array(spec.ma_lags, dtype=int) - 1
-        lagged_resid = lagmat(resid, max_ma_order, trim='both')[:, ma_ix]
+        lagged_resid = lagmat(resid, max_ma_order, trim='both')
 
         # Step 2: estimate ARMA model via least squares
         ix = initial_ar_order + max_ma_order - max_ar_order
-        mod = OLS(endog[initial_ar_order + max_ma_order:],
-                  np.c_[lagged_endog[ix:], lagged_resid])
-        res = mod.fit()
-        p.ar_params = res.params[:spec.k_ar_params]
-        p.ma_params = res.params[spec.k_ar_params:]
-        resid = res.resid
-        p.sigma2 = res.scale
+        X_with_free_params = np.c_[
+            lagged_endog[ix:, params_info.free_ar_ix],
+            lagged_resid[:, params_info.free_ma_ix]
+        ]
+        X_with_fixed_params = np.c_[
+            lagged_endog[ix:, params_info.fixed_ar_ix],
+            lagged_resid[:, params_info.fixed_ma_ix]
+        ]
+        y = endog[initial_ar_order + max_ma_order:]
+        if X_with_fixed_params.shape[1] != 0:
+            y -= X_with_fixed_params.dot(
+                np.r_[params_info.fixed_ar_params, params_info.fixed_ma_params]
+            )
+
+        # Step 2.1: no free ar params -> variance computation on the endog
+        # residual
+        if X_with_free_params.shape[1] == 0:
+            p.ar_params = params_info.fixed_ar_params
+            p.ma_params = params_info.fixed_ma_params
+            p.sigma2 = np.var(y, ddof=0)
+            resid = y.copy()
+        # Step 2.2: otherwise OLS with endog residual (after applying fixed
+        # params) as y, and lagged_endog and lagged_resid with free params as X
+        else:
+            mod = OLS(y, X_with_free_params)
+            res = mod.fit()
+            k_free_ar_params = len(params_info.free_ar_lags)
+            p.ar_params = _stitch_fixed_and_free_params(
+                fixed_ar_or_ma_lags=params_info.fixed_ar_lags,
+                fixed_ar_or_ma_params=params_info.fixed_ar_params,
+                free_ar_or_ma_lags=params_info.free_ar_lags,
+                free_ar_or_ma_params=res.params[:k_free_ar_params]
+            )
+            p.ma_params = _stitch_fixed_and_free_params(
+                fixed_ar_or_ma_lags=params_info.fixed_ma_lags,
+                fixed_ar_or_ma_params=params_info.fixed_ma_params,
+                free_ar_or_ma_lags=params_info.free_ma_lags,
+                free_ar_or_ma_params=res.params[k_free_ar_params:]
+            )
+            resid = res.resid
+            p.sigma2 = res.scale
 
         # Step 3: bias correction (if requested)
         if unbiased is True or unbiased is None:
-            if p.is_stationary and p.is_invertible:
+            if len(fixed_params) != 0:
+                raise NotImplementedError
+            elif p.is_stationary and p.is_invertible:
                 Z = np.zeros_like(endog)
                 V = np.zeros_like(endog)
                 W = np.zeros_like(endog)
@@ -175,8 +241,15 @@ def hannan_rissanen(endog, ar_order=0, ma_order=0, demean=True,
                 lagged_W = lagmat(W, max_ma_order, trim='both')
 
                 exog = np.c_[
-                    lagged_V[max(max_ma_order - max_ar_order, 0):, ar_ix],
-                    lagged_W[max(max_ar_order - max_ma_order, 0):, ma_ix]]
+                    lagged_V[
+                        max(max_ma_order - max_ar_order, 0):,
+                        params_info.free_ar_ix
+                    ],
+                    lagged_W[
+                        max(max_ar_order - max_ma_order, 0):,
+                        params_info.free_ma_ix
+                    ]
+                ]
 
                 mod_unbias = OLS(Z[max(max_ar_order, max_ma_order):], exog)
                 res_unbias = mod_unbias.fit()
@@ -206,5 +279,125 @@ def hannan_rissanen(endog, ar_order=0, ma_order=0, demean=True,
         'initial_ar_order': initial_ar_order,
         'resid': resid
     })
-
     return p, other_results
+
+
+def _validate_fixed_params(fixed_params, spec_param_names):
+    """
+    Check that keys in fixed_params are a subset of spec.param_names except
+    "sigma2"
+
+    Parameters
+    ----------
+    fixed_params : dict
+    spec_param_names : list of string
+        SARIMAXSpecification.param_names
+    """
+    if fixed_params is None:
+        fixed_params = {}
+
+    assert isinstance(fixed_params, dict)
+
+    fixed_param_names = set(fixed_params.keys())
+    valid_param_names = set(spec_param_names) - {"sigma2"}
+
+    invalid_param_names = fixed_param_names - valid_param_names
+
+    if len(invalid_param_names) > 0:
+        raise ValueError(
+            f"Invalid fixed parameter(s): {sorted(list(invalid_param_names))}."
+            f" Please select among {sorted(list(valid_param_names))}."
+        )
+
+    return fixed_params
+
+
+def _package_fixed_and_free_params_info(fixed_params, spec_ar_lags,
+                                        spec_ma_lags):
+    """
+    Parameters
+    ----------
+    fixed_params : dict
+    spec_ar_lags : list of int
+        SARIMAXSpecification.ar_lags
+    spec_ma_lags : list of int
+        SARIMAXSpecification.ma_lags
+
+    Returns
+    -------
+    Bunch with
+    (lags) fixed_ar_lags, fixed_ma_lags, free_ar_lags, free_ma_lags;
+    (ix) fixed_ar_ix, fixed_ma_ix, free_ar_ix, free_ma_ix;
+    (params) fixed_ar_params, free_ma_params
+    """
+    # unpack fixed lags and params
+    fixed_ar_lags_and_params = []
+    fixed_ma_lags_and_params = []
+    for key, val in fixed_params.items():
+        lag = int(key.split(".")[-1].lstrip("L"))
+        if key.startswith("ar"):
+            fixed_ar_lags_and_params.append((lag, val))
+        elif key.startswith("ma"):
+            fixed_ma_lags_and_params.append((lag, val))
+
+    fixed_ar_lags_and_params.sort()
+    fixed_ma_lags_and_params.sort()
+
+    fixed_ar_lags = [lag for lag, _ in fixed_ar_lags_and_params]
+    fixed_ar_params = np.array([val for _, val in fixed_ar_lags_and_params])
+
+    fixed_ma_lags = [lag for lag, _ in fixed_ma_lags_and_params]
+    fixed_ma_params = np.array([val for _, val in fixed_ma_lags_and_params])
+
+    # unpack free lags
+    free_ar_lags = [lag for lag in spec_ar_lags
+                    if lag not in set(fixed_ar_lags)]
+    free_ma_lags = [lag for lag in spec_ma_lags
+                    if lag not in set(fixed_ma_lags)]
+
+    # get ix for indexing purposes: `ar_ix`, and `ma_ix` below, are to account
+    # for non-consecutive lags; for indexing purposes, must have dtype int
+    free_ar_ix = np.array(free_ar_lags, dtype=int) - 1
+    free_ma_ix = np.array(free_ma_lags, dtype=int) - 1
+    fixed_ar_ix = np.array(fixed_ar_lags, dtype=int) - 1
+    fixed_ma_ix = np.array(fixed_ma_lags, dtype=int) - 1
+
+    return Bunch(
+        # lags
+        fixed_ar_lags=fixed_ar_lags, fixed_ma_lags=fixed_ma_lags,
+        free_ar_lags=free_ar_lags, free_ma_lags=free_ma_lags,
+        # ixs
+        fixed_ar_ix=fixed_ar_ix, fixed_ma_ix=fixed_ma_ix,
+        free_ar_ix=free_ar_ix, free_ma_ix=free_ma_ix,
+        # fixed params
+        fixed_ar_params=fixed_ar_params, fixed_ma_params=fixed_ma_params,
+    )
+
+
+def _stitch_fixed_and_free_params(fixed_ar_or_ma_lags, fixed_ar_or_ma_params,
+                                  free_ar_or_ma_lags, free_ar_or_ma_params):
+    """
+    Stitch together fixed and free params, by the order of lags, for setting
+    SARIMAXParams.ma_params or SARIMAXParams.ar_params
+
+    Parameters
+    ----------
+    fixed_ar_or_ma_lags : list or np.array
+    fixed_ar_or_ma_params : list or np.array
+        fixed_ar_or_ma_params corresponds with fixed_ar_or_ma_lags
+    free_ar_or_ma_lags : list or np.array
+    free_ar_or_ma_params : list or np.array
+        free_ar_or_ma_params corresponds with free_ar_or_ma_lags
+
+    Returns
+    -------
+    list of fixed and free params by the order of lags
+    """
+    assert len(fixed_ar_or_ma_lags) == len(fixed_ar_or_ma_params)
+    assert len(free_ar_or_ma_lags) == len(free_ar_or_ma_params)
+
+    all_lags = np.r_[fixed_ar_or_ma_lags, free_ar_or_ma_lags]
+    all_params = np.r_[fixed_ar_or_ma_params, free_ar_or_ma_params]
+
+    sorted_idx = all_lags.argsort()
+    return list(all_params[sorted_idx])
diff --git a/statsmodels/tsa/arima/estimators/tests/test_hannan_rissanen.py b/statsmodels/tsa/arima/estimators/tests/test_hannan_rissanen.py
index 61c0125a98b..cb861f69758 100644
--- a/statsmodels/tsa/arima/estimators/tests/test_hannan_rissanen.py
+++ b/statsmodels/tsa/arima/estimators/tests/test_hannan_rissanen.py
@@ -5,7 +5,13 @@
 
 from statsmodels.tsa.innovations.arma_innovations import arma_innovations
 from statsmodels.tsa.arima.datasets.brockwell_davis_2002 import lake
-from statsmodels.tsa.arima.estimators.hannan_rissanen import hannan_rissanen
+from statsmodels.tsa.arima.estimators.hannan_rissanen import (
+    hannan_rissanen, _validate_fixed_params,
+    _package_fixed_and_free_params_info,
+    _stitch_fixed_and_free_params
+)
+from statsmodels.tsa.arima.specification import SARIMAXSpecification
+from statsmodels.tools.tools import Bunch
 
 
 @pytest.mark.low_precision('Test against Example 5.1.7 in Brockwell and Davis'
@@ -100,3 +106,190 @@ def test_unbiased_error():
     endog = (np.arange(1000) * 1.0)
     with pytest.raises(ValueError, match='Cannot perform third step'):
         hannan_rissanen(endog, ar_order=1, ma_order=1, unbiased=True)
+
+
+@pytest.mark.parametrize(
+    "ar_order, ma_order, fixed_params, invalid_fixed_params",
+    [
+        # no fixed param
+        (2, [1, 0, 1], None, None),
+        ([0, 1], 0, {}, None),
+        # invalid fixed params
+        (1, 3, {"ar.L2": 1, "ma.L2": 0}, ["ar.L2"]),
+        ([0, 1], [0, 0, 1], {"ma.L1": 0, "sigma2": 1}, ["ma.L2", "sigma2"]),
+        (0, 0, {"ma.L1": 0, "ar.L1": 0}, ["ar.L1", "ma.L1"]),
+        (5, [1, 0], {"random_param": 0, "ar.L1": 0}, ["random_param"]),
+        # valid fixed params
+        (0, 2, {"ma.L1": -1, "ma.L2": 1}, None),
+        (1, 0, {"ar.L1": 0}, None),
+        ([1, 0, 1], 3, {"ma.L2": 1, "ar.L3": -1}, None),
+        # all fixed
+        (2, 2, {"ma.L1": 1, "ma.L2": 1, "ar.L1": 1, "ar.L2": 1}, None)
+    ]
+)
+def test_validate_fixed_params(ar_order, ma_order, fixed_params,
+                               invalid_fixed_params):
+    # test validation with both _validate_fixed_params and directly with
+    # hannan_rissanen
+
+    endog = np.random.normal(size=100)
+    spec = SARIMAXSpecification(endog, ar_order=ar_order, ma_order=ma_order)
+
+    if invalid_fixed_params is None:
+        _validate_fixed_params(fixed_params, spec.param_names)
+        hannan_rissanen(
+            endog, ar_order=ar_order, ma_order=ma_order,
+            fixed_params=fixed_params, unbiased=False
+        )
+    else:
+        valid_params = sorted(list(set(spec.param_names) - {'sigma2'}))
+        msg = f"Invalid fixed parameter(s): {invalid_fixed_params}. " \
+              f"Please select among {valid_params}."
+        # using direct `assert` to test error message instead of `match` since
+        # the error message contains regex characters
+        with pytest.raises(ValueError) as e:
+            _validate_fixed_params(fixed_params, spec.param_names)
+            assert e.msg == msg
+        with pytest.raises(ValueError) as e:
+            hannan_rissanen(
+                endog, ar_order=ar_order, ma_order=ma_order,
+                fixed_params=fixed_params, unbiased=False
+            )
+            assert e.msg == msg
+
+
+@pytest.mark.parametrize(
+    "fixed_params, spec_ar_lags, spec_ma_lags, expected_bunch",
+    [
+        ({}, [1], [], Bunch(
+            # lags
+            fixed_ar_lags=[], fixed_ma_lags=[],
+            free_ar_lags=[1], free_ma_lags=[],
+            # ixs
+            fixed_ar_ix=np.array([], dtype=int),
+            fixed_ma_ix=np.array([], dtype=int),
+            free_ar_ix=np.array([0], dtype=int),
+            free_ma_ix=np.array([], dtype=int),
+            # fixed params
+            fixed_ar_params=np.array([]), fixed_ma_params=np.array([]),
+        )),
+        ({"ar.L2": 0.1, "ma.L1": 0.2}, [2], [1, 3], Bunch(
+            # lags
+            fixed_ar_lags=[2], fixed_ma_lags=[1],
+            free_ar_lags=[], free_ma_lags=[3],
+            # ixs
+            fixed_ar_ix=np.array([1], dtype=int),
+            fixed_ma_ix=np.array([0], dtype=int),
+            free_ar_ix=np.array([], dtype=int),
+            free_ma_ix=np.array([2], dtype=int),
+            # fixed params
+            fixed_ar_params=np.array([0.1]), fixed_ma_params=np.array([0.2]),
+        )),
+        ({"ma.L5": 0.1, "ma.L10": 0.2}, [], [5, 10], Bunch(
+            # lags
+            fixed_ar_lags=[], fixed_ma_lags=[5, 10],
+            free_ar_lags=[], free_ma_lags=[],
+            # ixs
+            fixed_ar_ix=np.array([], dtype=int),
+            fixed_ma_ix=np.array([4, 9], dtype=int),
+            free_ar_ix=np.array([], dtype=int),
+            free_ma_ix=np.array([], dtype=int),
+            # fixed params
+            fixed_ar_params=np.array([]), fixed_ma_params=np.array([0.1, 0.2]),
+        )),
+    ]
+)
+def test_package_fixed_and_free_params_info(fixed_params, spec_ar_lags,
+                                            spec_ma_lags, expected_bunch):
+    actual_bunch = _package_fixed_and_free_params_info(
+        fixed_params, spec_ar_lags, spec_ma_lags
+    )
+    assert isinstance(actual_bunch, Bunch)
+    assert len(actual_bunch) == len(expected_bunch)
+    assert actual_bunch.keys() == expected_bunch.keys()
+
+    # check lags
+    lags = ['fixed_ar_lags', 'fixed_ma_lags', 'free_ar_lags', 'free_ma_lags']
+    for k in lags:
+        assert isinstance(actual_bunch[k], list)
+        assert actual_bunch[k] == expected_bunch[k]
+
+    # check lags
+    ixs = ['fixed_ar_ix', 'fixed_ma_ix', 'free_ar_ix', 'free_ma_ix']
+    for k in ixs:
+        assert isinstance(actual_bunch[k], np.ndarray)
+        assert actual_bunch[k].dtype in [np.int64, np.int32]
+        np.testing.assert_array_equal(actual_bunch[k], expected_bunch[k])
+
+    params = ['fixed_ar_params', 'fixed_ma_params']
+    for k in params:
+        assert isinstance(actual_bunch[k], np.ndarray)
+        np.testing.assert_array_equal(actual_bunch[k], expected_bunch[k])
+
+
+@pytest.mark.parametrize(
+    "fixed_lags, free_lags, fixed_params, free_params, expected_all_params",
+    [
+        ([], [], [], [], []),
+        ([2], [], [0.2], [], [0.2]),
+        ([], [1], [], [0.2], [0.2]),
+        ([1], [3], [0.2], [-0.2], [0.2, -0.2]),
+        ([3], [1, 2], [0.2], [0.3, -0.2], [0.3, -0.2, 0.2]),
+        ([3, 1], [2, 4], [0.3, 0.1], [0.5, 0.], [0.1, 0.5, 0.3, 0.]),
+        ([3, 10], [1, 2], [0.2, 0.5], [0.3, -0.2], [0.3, -0.2, 0.2, 0.5]),
+    ]
+)
+def test_stitch_fixed_and_free_params(fixed_lags, free_lags, fixed_params,
+                                      free_params, expected_all_params):
+    actual_all_params = _stitch_fixed_and_free_params(
+        fixed_lags, fixed_params, free_lags, free_params
+    )
+    assert actual_all_params == expected_all_params
+
+
+def test_unbiased_error_with_fixed_params():
+    # unbiased=True or unbiased=None with fixed params should throw
+    # NotImplementedError for now
+    endog = np.random.normal(size=1000)
+    with pytest.raises(NotImplementedError):
+        hannan_rissanen(endog, ar_order=1, ma_order=1, unbiased=True,
+                        fixed_params={"ar.L1": 0})
+    with pytest.raises(NotImplementedError):
+        hannan_rissanen(endog, ar_order=1, ma_order=1, unbiased=None,
+                        fixed_params={"ar.L1": 0})
+
+
+@pytest.mark.parametrize(
+    "fixed_params",
+    [
+        {"ar.L1": 0.69607715},  # fix ar
+        {"ma.L1": 0.37879692},  # fix ma
+        {"ar.L1": 0.69607715, "ma.L1": 0.37879692},  # no free params
+    ]
+)
+def test_itsmr_with_fixed_params(fixed_params):
+    # This test is a variation of test_itsmr where we fix 1 or more parameters
+    # for Example 5.1.7 in Brockwell and Davis (2016) and check that free
+    # parameters are still correct'.
+
+    endog = lake.copy()
+    hr, _ = hannan_rissanen(
+        endog, ar_order=1, ma_order=1, demean=True,
+        initial_ar_order=22, unbiased=False,
+        fixed_params=fixed_params
+    )
+
+    assert_allclose(hr.ar_params, [0.69607715], atol=1e-4)
+    assert_allclose(hr.ma_params, [0.3787969217], atol=1e-4)
+
+    # Because our fast implementation of the innovations algorithm does not
+    # allow for non-stationary processes, the estimate of the variance returned
+    # by `hannan_rissanen` is based on the residuals from the least-squares
+    # regression, rather than (as reported by BD) based on the innovations
+    # algorithm output. Since the estimates here do correspond to a stationary
+    # series, we can compute the innovations variance manually to check
+    # against BD.
+    u, v = arma_innovations(endog - endog.mean(), hr.ar_params, hr.ma_params,
+                            sigma2=1)
+    tmp = u / v**0.5
+    assert_allclose(np.inner(tmp, tmp) / len(u), 0.4773580109, atol=1e-4)


Code review comments:
diff_hunk: @@ -133,22 +164,57 @@ def hannan_rissanen(endog, ar_order=0, ma_order=0, demean=True,
         resid = y - X.dot(initial_ar_params)
 
         # Get lagged residuals for `exog` in least-squares regression
-        ma_ix = np.array(spec.ma_lags, dtype=int) - 1
-        lagged_resid = lagmat(resid, max_ma_order, trim='both')[:, ma_ix]
+        lagged_resid = lagmat(resid, max_ma_order, trim='both')
 
         # Step 2: estimate ARMA model via least squares
         ix = initial_ar_order + max_ma_order - max_ar_order
-        mod = OLS(endog[initial_ar_order + max_ma_order:],
-                  np.c_[lagged_endog[ix:], lagged_resid])
-        res = mod.fit()
-        p.ar_params = res.params[:spec.k_ar_params]
-        p.ma_params = res.params[spec.k_ar_params:]
-        resid = res.resid
-        p.sigma2 = res.scale
+        X_with_free_params = np.c_[
+            lagged_endog[ix:, params_info.free_ar_ix],
+            lagged_resid[:, params_info.free_ma_ix]
+        ]
+        X_with_fixed_params = np.c_[
+            lagged_endog[ix:, params_info.fixed_ar_ix],
+            lagged_resid[:, params_info.fixed_ma_ix]
+        ]
+        y = endog[initial_ar_order + max_ma_order:]
+        if X_with_fixed_params.shape[1] != 0:
+            y -= X_with_fixed_params.dot(
+                np.r_[params_info.fixed_ar_params, params_info.fixed_ma_params]
+            )
+
+        # Step 2.1: no free ar params -> variance computation on the endog
+        # residual
+        if X_with_free_params.shape[1] == 0:
+            p.ar_params = params_info.fixed_ar_params
+            p.ma_params = params_info.fixed_ma_params
+            p.sigma2 = np.var(y, ddof=0)
+            resid = y.copy()
+        # Step 2.2: otherwise OLS with endog residual (after applying fixed
+        # params) as y, and lagged_endog and lagged_resid with free params as X
+        else:
+            mod = OLS(y, X_with_free_params)
+            res = mod.fit()
+            k_free_ar_params = len(params_info.free_ar_lags)
+            p.ar_params = _stitch_fixed_and_free_params(
+                fixed_ar_or_ma_lags=params_info.fixed_ar_lags,
+                fixed_ar_or_ma_params=params_info.fixed_ar_params,
+                free_ar_or_ma_lags=params_info.free_ar_lags,
+                free_ar_or_ma_params=res.params[:k_free_ar_params]
+            )
+            p.ma_params = _stitch_fixed_and_free_params(
+                fixed_ar_or_ma_lags=params_info.fixed_ma_lags,
+                fixed_ar_or_ma_params=params_info.fixed_ma_params,
+                free_ar_or_ma_lags=params_info.free_ma_lags,
+                free_ar_or_ma_params=res.params[k_free_ar_params:]
+            )
+            resid = res.resid
+            p.sigma2 = res.scale
 
         # Step 3: bias correction (if requested)
         if unbiased is True or unbiased is None:
-            if p.is_stationary and p.is_invertible:
+            if len(fixed_params) != 0:
+                raise NotImplementedError
path: statsmodels/tsa/arima/estimators/hannan_rissanen.py
text: @author:
Raising `NotImplementedError` here as instructed.

diff_hunk: @@ -100,3 +106,190 @@ def test_unbiased_error():
     endog = (np.arange(1000) * 1.0)
     with pytest.raises(ValueError, match='Cannot perform third step'):
         hannan_rissanen(endog, ar_order=1, ma_order=1, unbiased=True)
+
+
+@pytest.mark.parametrize(
+    "ar_order, ma_order, fixed_params, invalid_fixed_params",
+    [
+        # no fixed param
+        (2, [1, 0, 1], None, None),
+        ([0, 1], 0, {}, None),
+        # invalid fixed params
+        (1, 3, {"ar.L2": 1, "ma.L2": 0}, ["ar.L2"]),
+        ([0, 1], [0, 0, 1], {"ma.L1": 0, "sigma2": 1}, ["ma.L2", "sigma2"]),
+        (0, 0, {"ma.L1": 0, "ar.L1": 0}, ["ar.L1", "ma.L1"]),
+        (5, [1, 0], {"random_param": 0, "ar.L1": 0}, ["random_param"]),
+        # valid fixed params
+        (0, 2, {"ma.L1": -1, "ma.L2": 1}, None),
+        (1, 0, {"ar.L1": 0}, None),
+        ([1, 0, 1], 3, {"ma.L2": 1, "ar.L3": -1}, None),
+        # all fixed
+        (2, 2, {"ma.L1": 1, "ma.L2": 1, "ar.L1": 1, "ar.L2": 1}, None)
+    ]
+)
+def test_validate_fixed_params(ar_order, ma_order, fixed_params,
+                               invalid_fixed_params):
+    # test validation with both _validate_fixed_params and directly with
+    # hannan_rissanen
+
+    endog = np.random.normal(size=100)
+    spec = SARIMAXSpecification(endog, ar_order=ar_order, ma_order=ma_order)
+
+    if invalid_fixed_params is None:
+        _validate_fixed_params(fixed_params, spec.param_names)
+        hannan_rissanen(
+            endog, ar_order=ar_order, ma_order=ma_order,
+            fixed_params=fixed_params, unbiased=False
+        )
+    else:
+        valid_params = sorted(list(set(spec.param_names) - {'sigma2'}))
+        msg = f"Invalid fixed parameter(s): {invalid_fixed_params}. " \
path: statsmodels/tsa/arima/estimators/tests/test_hannan_rissanen.py
text: @user1:
Style-wise, we typically use parentheses instead of the `\` operator.