Repository: scipy/scipy

Issue:
BUG: step response not accurate at higher orders
A 6th order analog filter appears very distorted in scipy.signal.step compared to Octave:

```
z = array([], dtype=float64)
p = array([-2.8576, -2.8576, -2.8576, -2.8576, -2.8576, -2.8576])
k = 544.49693870986994
t_s, y_s = step((z, p, k), T=t)
plot(t_s, y_s)
```

![scipy 6th order step response screwed up](https://f.cloud.github.com/assets/58611/846316/58597c0c-f40b-11e2-8e7a-7a5c792c6b31.png)

vs Octave:

```
z = []
p = [-2.8576  -2.8576  -2.8576  -2.8576  -2.8576  -2.8576]
k = 544.49693870986994
step(zpk(z, p, k), 8)
```

![octave step response](https://f.cloud.github.com/assets/58611/846311/305721d2-f40b-11e2-9f2c-f4de4982dc83.png)

Even 4th order filters are visibly inaccurate, settling at 1.02 instead of 1.00, which is not a very high-order filter



Discussion:
This is a known weakness/bug in `step` (see also, for example, http://stackoverflow.com/questions/16114971/scipy-step-response-plot-seems-to-break-for-some-values/16115915#16115915).

Try it again with `step2`.

This comes up frequently enough that it is probably time to replace the implementation in `step` with something more robust.

Existing ticket on basically the same issue: https://github.com/scipy/scipy/issues/1104

Why are there two different functions for the same thing?

`step2` sometimes works well, but sometimes gives pages and pages of errors, which I think is https://github.com/scipy/scipy/issues/2570 ?

```
...

Original exception was:
TypeError: Cannot cast array data from dtype('complex128') to dtype('float64') according to the rule 'safe'
Error in sys.excepthook:
Traceback (most recent call last):
  File "C:\Python27\lib\site-packages\IPython\iplib.py", line 2031, in excepthook
    self.showtraceback((etype,value,tb),tb_offset=0)
  File "C:\Python27\lib\site-packages\IPython\iplib.py", line 1731, in showtraceback
    self.InteractiveTB(etype,value,tb,tb_offset=tb_offset)
  File "C:\Python27\lib\site-packages\IPython\ultraTB.py", line 998, in __call__
    print >> out, self.text(etype, evalue, etb)
  File "C:\Python27\lib\site-packages\IPython\ultraTB.py", line 1012, in text
    return FormattedTB.text(self,etype,value,tb,context=5,mode=mode)
  File "C:\Python27\lib\site-packages\IPython\ultraTB.py", line 937, in text
    if len(elist) > self.tb_offset:
TypeError: object of type 'NoneType' has no len()

Original exception was:
odepack.error: Result from function call is not a proper array of floats.
Error in sys.excepthook:
Traceback (most recent call last):
  File "C:\Python27\lib\site-packages\IPython\iplib.py", line 2031, in excepthook
    self.showtraceback((etype,value,tb),tb_offset=0)
  File "C:\Python27\lib\site-packages\IPython\iplib.py", line 1731, in showtraceback
    self.InteractiveTB(etype,value,tb,tb_offset=tb_offset)
  File "C:\Python27\lib\site-packages\IPython\ultraTB.py", line 998, in __call__
    print >> out, self.text(etype, evalue, etb)
  File "C:\Python27\lib\site-packages\IPython\ultraTB.py", line 1012, in text
    return FormattedTB.text(self,etype,value,tb,context=5,mode=mode)
  File "C:\Python27\lib\site-packages\IPython\ultraTB.py", line 937, in text
    if len(elist) > self.tb_offset:
TypeError: object of type 'NoneType' has no len()

...
```

> Why are there two different functions for the same thing?

Excellent question!  That's my fault.  I added `step2` about three years ago (https://github.com/scipy/scipy/commit/495185e02ca005f2885b8b34fa34009e1ac708a5).  I was relatively new to scipy development then, and wasn't comfortable rewriting existing code, so I made a new function.  No one objected at the time, so in it went.  Now I think it would be better to make `step` Just Work (tm) (possibly with a `method` argument if it was really necessary to provide multiple algorithms), and deprecate `step2`.  (See also `impulse2` and `lsim2`.)

Agreed, that needs to be cleaned up at some point.

As part of the rewriting process of `odeint` (gh-2818) tons of tests in `scipy.signal` fail. It made me remember this, just in case you are thinking of doing the clean-up soon.

> Now I think it would be better to make step Just Work (tm) (possibly with a method argument if it was really necessary to provide multiple algorithms), and deprecate step2. (See also impulse2 and lsim2.)

Would that be covered by this bug, or should it be filed separately?


Resolving patch:
diff --git a/scipy/signal/ltisys.py b/scipy/signal/ltisys.py
--- a/scipy/signal/ltisys.py
+++ b/scipy/signal/ltisys.py
@@ -13,6 +13,8 @@
 #   Rewrote abcd_normalize.
 # Jan 2015: Irvin Probst irvin DOT probst AT ensta-bretagne DOT fr
 #   Added pole placement
+# Mar 2015: Clancy Rowley
+#   Rewrote lsim
 #
 
 import warnings
@@ -617,7 +619,7 @@ def _cast_to_array_dtype(in1, in2):
     return in1
 
 
-def lsim(system, U, T, X0=None, interp=1):
+def lsim(system, U, T, X0=None, interp=True):
     """
     Simulate output of a continuous-time linear system.
 
@@ -635,14 +637,15 @@ def lsim(system, U, T, X0=None, interp=1):
         An input array describing the input at each time `T`
         (interpolation is assumed between given times).  If there are
         multiple inputs, then each column of the rank-2 array
-        represents an input.
+        represents an input.  If U = 0 or None, a zero input is used.
     T : array_like
         The time steps at which the input is defined and at which the
-        output is desired.
+        output is desired.  Must be nonnegative, increasing, and equally spaced.
     X0 : array_like, optional
         The initial conditions on the state vector (zero by default).
-    interp : {1, 0}, optional
-        Whether to use linear (1) or zero-order hold (0) interpolation.
+    interp : bool, optional
+        Whether to use linear (True, the default) or zero-order-hold (False)
+        interpolation for the input array.
 
     Returns
     -------
@@ -651,65 +654,123 @@ def lsim(system, U, T, X0=None, interp=1):
     yout : 1D ndarray
         System response.
     xout : ndarray
-        Time-evolution of the state-vector.
+        Time evolution of the state vector.
 
+    Examples
+    --------
+    Simulate a double integrator y'' = u, with a constant input u = 1
+
+    >>> from scipy import signal
+    >>> system = signal.lti([[0., 1.], [0., 0.]], [[0.], [1.]], [[1., 0.]], 0.)
+    >>> t = np.linspace(0, 5)
+    >>> u = np.ones_like(t)
+    >>> tout, y, x = signal.lsim(system, u, t)
+    >>> import matplotlib.pyplot as plt
+    >>> plt.plot(t, y)
     """
     if isinstance(system, lti):
         sys = system
     else:
         sys = lti(*system)
-    U = atleast_1d(U)
     T = atleast_1d(T)
-    if len(U.shape) == 1:
-        U = U.reshape((U.shape[0], 1))
-    sU = U.shape
     if len(T.shape) != 1:
         raise ValueError("T must be a rank-1 array.")
-    if sU[0] != len(T):
-        raise ValueError("U must have the same number of rows "
-                         "as elements in T.")
-    if sU[1] != sys.inputs:
-        raise ValueError("System does not define that many inputs.")
 
+    A, B, C, D = map(np.asarray, (sys.A, sys.B, sys.C, sys.D))
+    n_states = A.shape[0]
+    n_inputs = B.shape[1]
+
+    n_steps = T.size
     if X0 is None:
-        X0 = zeros(sys.B.shape[0], sys.A.dtype)
+        X0 = zeros(n_states, sys.A.dtype)
+    xout = zeros((n_steps, n_states), sys.A.dtype)
+
+    if T[0] == 0:
+        xout[0] = X0
+    elif T[0] > 0:
+        # step forward to initial time, with zero input
+        xout[0] = dot(X0, linalg.expm(transpose(A) * T[0]))
+    else:
+        raise ValueError("Initial time must be nonnegative")
+
+    no_input = (U is None
+                or (isinstance(U, (int, float)) and U == 0.)
+                or not np.any(U))
+
+    if n_steps == 1:
+        yout = squeeze(dot(xout, transpose(C)))
+        if not no_input:
+            yout += squeeze(dot(U, transpose(D)))
+        return T, squeeze(yout), squeeze(xout)
 
-    xout = zeros((len(T), sys.B.shape[0]), sys.A.dtype)
-    xout[0] = X0
-    A = sys.A
-    AT, BT = transpose(sys.A), transpose(sys.B)
     dt = T[1] - T[0]
-    lam, v = linalg.eig(A)
-    vt = transpose(v)
-    vti = linalg.inv(vt)
-    GT = dot(dot(vti, diag(numpy.exp(dt * lam))), vt)
-    GT = _cast_to_array_dtype(GT, xout)
-
-    ATm1 = linalg.inv(AT)
-    ATm2 = dot(ATm1, ATm1)
-    I = eye(A.shape[0], dtype=A.dtype)
-    GTmI = GT - I
-    F1T = dot(dot(BT, GTmI), ATm1)
-    if interp:
-        F2T = dot(BT, dot(GTmI, ATm2) / dt - ATm1)
-
-    for k in xrange(1, len(T)):
-        dt1 = T[k] - T[k - 1]
-        if dt1 != dt:
-            dt = dt1
-            GT = dot(dot(vti, diag(numpy.exp(dt * lam))), vt)
-            GT = _cast_to_array_dtype(GT, xout)
-            GTmI = GT - I
-            F1T = dot(dot(BT, GTmI), ATm1)
-            if interp:
-                F2T = dot(BT, dot(GTmI, ATm2) / dt - ATm1)
-
-        xout[k] = dot(xout[k - 1], GT) + dot(U[k - 1], F1T)
-        if interp:
-            xout[k] = xout[k] + dot((U[k] - U[k - 1]), F2T)
-
-    yout = (squeeze(dot(U, transpose(sys.D))) +
-            squeeze(dot(xout, transpose(sys.C))))
+    if not np.allclose((T[1:] - T[:-1]) / dt, 1.0):
+        warnings.warn("Non-uniform timesteps are deprecated. Results may be "
+                      "slow and/or inaccurate.", DeprecationWarning)
+        return lsim2(system, U, T, X0)
+
+    if no_input:
+        # Zero input: just use matrix exponential
+        # take transpose because state is a row vector
+        expAT_dt = linalg.expm(transpose(A) * dt)
+        for i in xrange(1, n_steps):
+            xout[i] = dot(xout[i-1], expAT_dt)
+        yout = squeeze(dot(xout, transpose(C)))
+        return T, squeeze(yout), squeeze(xout)
+
+    # Nonzero input
+    U = atleast_1d(U)
+    if U.ndim == 1:
+        U = U[:, np.newaxis]
+
+    if U.shape[0] != n_steps:
+        raise ValueError("U must have the same number of rows "
+                         "as elements in T.")
+
+    if U.shape[1] != n_inputs:
+        raise ValueError("System does not define that many inputs.")
+
+    if not interp:
+        # Zero-order hold
+        # Algorithm: to integrate from time 0 to time dt, we solve
+        #   xdot = A x + B u,  x(0) = x0
+        #   udot = 0,          u(0) = u0.
+        #
+        # Solution is
+        #   [ x(dt) ]       [ A*dt   B*dt ] [ x0 ]
+        #   [ u(dt) ] = exp [  0     0    ] [ u0 ]
+        M = np.vstack([np.hstack([A * dt, B * dt]),
+                       np.zeros((n_inputs, n_states + n_inputs))])
+        # transpose everything because the state and input are row vectors
+        expMT = linalg.expm(transpose(M))
+        Ad = expMT[:n_states, :n_states]
+        Bd = expMT[n_states:, :n_states]
+        for i in xrange(1, n_steps):
+            xout[i] = dot(xout[i-1], Ad) + dot(U[i-1], Bd)
+    else:
+        # Linear interpolation between steps
+        # Algorithm: to integrate from time 0 to time dt, with linear
+        # interpolation between inputs u(0) = u0 and u(dt) = u1, we solve
+        #   xdot = A x + B u,        x(0) = x0
+        #   udot = (u1 - u0) / dt,   u(0) = u0.
+        #
+        # Solution is
+        #   [ x(dt) ]       [ A*dt  B*dt  0 ] [  x0   ]
+        #   [ u(dt) ] = exp [  0     0    I ] [  u0   ]
+        #   [u1 - u0]       [  0     0    0 ] [u1 - u0]
+        M = np.vstack([np.hstack([A * dt, B * dt,
+                                  np.zeros((n_states, n_inputs))]),
+                       np.hstack([np.zeros((n_inputs, n_states + n_inputs)),
+                                  np.identity(n_inputs)]),
+                       np.zeros((n_inputs, n_states + 2 * n_inputs))])
+        expMT = linalg.expm(transpose(M))
+        Ad = expMT[:n_states, :n_states]
+        Bd1 = expMT[n_states+n_inputs:, :n_states]
+        Bd0 = expMT[n_states:n_states + n_inputs, :n_states] - Bd1
+        for i in xrange(1, n_steps):
+            xout[i] = (dot(xout[i-1], Ad) + dot(U[i-1], Bd0) + dot(U[i], Bd1))
+
+    yout = (squeeze(dot(xout, transpose(C))) + squeeze(dot(U, transpose(D))))
     return T, squeeze(yout), squeeze(xout)
 
 
@@ -780,9 +841,9 @@ def impulse(system, X0=None, T=None, N=None):
     else:
         sys = lti(*system)
     if X0 is None:
-        B = sys.B
+        X = squeeze(sys.B)
     else:
-        B = sys.B + X0
+        X = squeeze(sys.B + X0)
     if N is None:
         N = 100
     if T is None:
@@ -790,16 +851,7 @@ def impulse(system, X0=None, T=None, N=None):
     else:
         T = asarray(T)
 
-    h = zeros(T.shape, sys.A.dtype)
-    s, v = linalg.eig(sys.A)
-    vi = linalg.inv(v)
-    C = sys.C
-    for k in range(len(h)):
-        es = diag(numpy.exp(s * T[k]))
-        eA = dot(dot(v, es), vi)
-        eA = _cast_to_array_dtype(eA, h)
-        h[k] = squeeze(dot(dot(C, eA), B))
-
+    _, h, _ = lsim(sys, 0., T, X, interp=False)
     return T, h
 
 
@@ -928,7 +980,7 @@ def step(system, X0=None, T=None, N=None):
     else:
         T = asarray(T)
     U = ones(T.shape, sys.A.dtype)
-    vals = lsim(sys, U, T, X0=X0)
+    vals = lsim(sys, U, T, X0=X0, interp=False)
     return vals[0], vals[1]
 
 


Test patch:
diff --git a/scipy/signal/tests/test_ltisys.py b/scipy/signal/tests/test_ltisys.py
--- a/scipy/signal/tests/test_ltisys.py
+++ b/scipy/signal/tests/test_ltisys.py
@@ -6,7 +6,7 @@
 from numpy.testing import (assert_almost_equal, assert_equal, assert_allclose,
                            assert_, assert_raises, TestCase, run_module_suite)
 from scipy.signal.ltisys import (ss2tf, tf2ss, lsim2, impulse2, step2, lti,
-                                 bode, freqresp, impulse, step,
+                                 bode, freqresp, lsim, impulse, step,
                                  abcd_normalize, place_poles)
 from scipy.signal.filter_design import BadCoefficients
 import scipy.linalg as linalg
@@ -308,6 +308,91 @@ def test_multioutput(self):
         assert_allclose(b_all, np.vstack((b0, b1, b2)), rtol=1e-13, atol=1e-14)
 
 
+class TestLsim(object):
+    def lti_nowarn(self, *args):
+        with warnings.catch_warnings():
+            warnings.simplefilter("ignore", BadCoefficients)
+            system = lti(*args)
+        return system
+
+    def test_first_order(self):
+        # y' = -y
+        # exact solution is y(t) = exp(-t)
+        system = self.lti_nowarn(-1.,1.,1.,0.)
+        t = np.linspace(0,5)
+        u = np.zeros_like(t)
+        tout, y, x = lsim(system, u, t, X0=[1.0])
+        expected_x = np.exp(-tout)
+        assert_almost_equal(x, expected_x)
+        assert_almost_equal(y, expected_x)
+
+    def test_integrator(self):
+        # integrator: y' = u
+        system = self.lti_nowarn(0., 1., 1., 0.)
+        t = np.linspace(0,5)
+        u = t
+        tout, y, x = lsim(system, u, t)
+        expected_x = 0.5 * tout**2
+        assert_almost_equal(x, expected_x)
+        assert_almost_equal(y, expected_x)
+
+    def test_double_integrator(self):
+        # double integrator: y'' = 2u
+        A = np.mat("0. 1.; 0. 0.")
+        B = np.mat("0.; 1.")
+        C = np.mat("2. 0.")
+        system = self.lti_nowarn(A, B, C, 0.)
+        t = np.linspace(0,5)
+        u = np.ones_like(t)
+        tout, y, x = lsim(system, u, t)
+        expected_x = np.transpose(np.array([0.5 * tout**2, tout]))
+        expected_y = tout**2
+        assert_almost_equal(x, expected_x)
+        assert_almost_equal(y, expected_y)
+
+    def test_jordan_block(self):
+        # Non-diagonalizable A matrix
+        #   x1' + x1 = x2
+        #   x2' + x2 = u
+        #   y = x1
+        # Exact solution with u = 0 is y(t) = t exp(-t)
+        A = np.mat("-1. 1.; 0. -1.")
+        B = np.mat("0.; 1.")
+        C = np.mat("1. 0.")
+        system = self.lti_nowarn(A, B, C, 0.)
+        t = np.linspace(0,5)
+        u = np.zeros_like(t)
+        tout, y, x = lsim(system, u, t, X0=[0.0, 1.0])
+        expected_y = tout * np.exp(-tout)
+        assert_almost_equal(y, expected_y)
+
+    def test_miso(self):
+        # A system with two state variables, two inputs, and one output.
+        A = np.array([[-1.0, 0.0], [0.0, -2.0]])
+        B = np.array([[1.0, 0.0], [0.0, 1.0]])
+        C = np.array([1.0, 0.0])
+        D = np.zeros((1,2))
+        system = self.lti_nowarn(A, B, C, D)
+
+        t = np.linspace(0, 5.0, 101)
+        u = np.zeros_like(t)
+        tout, y, x = lsim(system, u, t, X0=[1.0, 1.0])
+        expected_y = np.exp(-tout)
+        expected_x0 = np.exp(-tout)
+        expected_x1 = np.exp(-2.0*tout)
+        assert_almost_equal(y, expected_y)
+        assert_almost_equal(x[:,0], expected_x0)
+        assert_almost_equal(x[:,1], expected_x1)
+
+    def test_nonzero_initial_time(self):
+        system = self.lti_nowarn(-1.,1.,1.,0.)
+        t = np.linspace(1,2)
+        u = np.zeros_like(t)
+        tout, y, x = lsim(system, u, t, X0=[1.0])
+        expected_y = np.exp(-tout)
+        assert_almost_equal(y, expected_y)
+
+
 class Test_lsim2(object):
 
     def test_01(self):
@@ -435,6 +520,15 @@ def test_05(self):
         expected_y = np.ones_like(tout)
         assert_almost_equal(y, expected_y)
 
+    def test_06(self):
+        # Second order system with a repeated root:
+        #     x''(t) + 2*x(t) + x(t) = u(t)
+        # The exact impulse response is t*exp(-t).
+        system = ([1.0], [1.0, 2.0, 1.0])
+        tout, y = self.func(system)
+        expected_y = tout * np.exp(-tout)
+        assert_almost_equal(y, expected_y)
+
     def test_array_like(self):
         # Test that function can accept sequences, scalars.
         system = ([1.0], [1.0, 2.0, 1.0])
@@ -442,24 +536,14 @@ def test_array_like(self):
         tout, y = self.func(system, X0=[3], T=[5, 6])
         tout, y = self.func(system, X0=[3], T=[5])
 
-
-class TestImpulse2(_TestImpulseFuncs):
-    def setup(self):
-        self.func = impulse2
-
     def test_array_like2(self):
         system = ([1.0], [1.0, 2.0, 1.0])
         tout, y = self.func(system, X0=3, T=5)
 
-    def test_06(self):
-        # Second order system with a repeated root:
-        #     x''(t) + 2*x(t) + x(t) = u(t)
-        # The exact impulse response is t*exp(-t).
-        # Doesn't pass for `impulse` (on some systems, see gh-2654)
-        system = ([1.0], [1.0, 2.0, 1.0])
-        tout, y = self.func(system)
-        expected_y = tout * np.exp(-tout)
-        assert_almost_equal(y, expected_y)
+
+class TestImpulse2(_TestImpulseFuncs):
+    def setup(self):
+        self.func = impulse2
 
 
 class TestImpulse(_TestImpulseFuncs):
@@ -510,6 +594,23 @@ def test_04(self):
         expected_y = 1 + 2.0*np.exp(-tout)
         assert_almost_equal(y, expected_y)
 
+    def test_05(self):
+        # Simple integrator: x'(t) = u(t)
+        # Exact step response is x(t) = t.
+        system = ([1.0],[1.0,0.0])
+        tout, y = self.func(system)
+        expected_y = tout
+        assert_almost_equal(y, expected_y)
+
+    def test_06(self):
+        # Second order system with a repeated root:
+        #     x''(t) + 2*x(t) + x(t) = u(t)
+        # The exact step response is 1 - (1 + t)*exp(-t).
+        system = ([1.0], [1.0, 2.0, 1.0])
+        tout, y = self.func(system)
+        expected_y = 1 - (1 + tout) * np.exp(-tout)
+        assert_almost_equal(y, expected_y)
+
     def test_array_like(self):
         # Test that function can accept sequences, scalars.
         system = ([1.0], [1.0, 2.0, 1.0])
@@ -522,6 +623,10 @@ def setup(self):
         self.func = step2
 
     def test_05(self):
+        # This test is almost the same as the one it overwrites in the base
+        # class.  The only difference is the tolerances passed to step2:
+        # the default tolerances are not accurate enough for this test
+
         # Simple integrator: x'(t) = u(t)
         # Exact step response is x(t) = t.
         system = ([1.0],[1.0,0.0])
@@ -529,15 +634,6 @@ def test_05(self):
         expected_y = tout
         assert_almost_equal(y, expected_y)
 
-    def test_06(self):
-        # Second order system with a repeated root:
-        #     x''(t) + 2*x(t) + x(t) = u(t)
-        # The exact step response is 1 - (1 + t)*exp(-t).
-        system = ([1.0], [1.0, 2.0, 1.0])
-        tout, y = self.func(system, atol=1e-10, rtol=1e-8)
-        expected_y = 1 - (1 + tout) * np.exp(-tout)
-        assert_almost_equal(y, expected_y)
-
 
 class TestStep(_TestStepFuncs):
     def setup(self):