Linear time-invariant system
In system analysis, among other fields of study, a linear time-invariant system (LTI system) is a system that produces an output signal from any input signal subject to the constraints of linearity and time-invariance; these terms are briefly defined below. These properties apply (exactly or approximately) to many important physical systems, in which case the response y"("t") of the system to an arbitrary input x"("t") can be found directly using convolution: y"("t") = "x"("t") ∗ "h"("t") where h"("t") is called the system's impulse response and ∗ represents convolution (not to be confused with  multiplication, as is frequently employed by the symbol in computer languages). What's more, there are systematic methods for solving any such system (determining "h"("t")), whereas systems not meeting both properties are generally more difficult (or impossible) to solve analytically. A good example of an LTI system is any electrical circuit consisting of resistors, capacitors, inductors and linear amplifiers. Linear time-invariant system theory is also used in image processing, where the systems have spatial dimensions instead of, or in addition to, a temporal dimension. These systems may be referred to as "linear translation-invariant" to give the terminology the  most general reach. In the case of generic discrete-time (i.e., sampled) systems, "linear shift-invariant" is the corresponding term. LTI system theory is an area of applied mathematics which has direct applications in electrical circuit analysis and design, signal processing and filter design, control theory, mechanical engineering, image processing, the design of measuring instruments of many sorts, NMR spectroscopy, and many other technical areas where systems of ordinary differential equations present themselves. Overview. The defining properties of any LTI system are "linearity" and "time invariance". The fundamental result in LTI system theory is that any LTI system can be  characterized entirely by a single function called the system's impulse response. The output of the system formula_2 is simply the convolution of the input to the system formula_1 with the system's impulse response formula_19. This is called a continuous time system. Similarly, a discrete-time linear time-invariant (or, more generally, "shift-invariant") system is defined as one operating in discrete time: formula_20 where "y", "x", and "h" are sequences and the convolution, in discrete time, uses a discrete summation rather than an integral. LTI systems can also be characterized in the "frequency domain" by the system's  transfer function, which is the Laplace transform of the system's impulse response (or Z transform in the case of discrete-time systems). As a result of the properties of these transforms, the output of the system in the frequency domain is the product of the transfer function and the transform of the input. In other words, convolution in the time domain is equivalent to multiplication in the frequency domain. For all LTI systems, the eigenfunctions, and the basis functions of the transforms, are complex exponentials. This is, if the input to a system is the complex waveform formula_21 for  some complex amplitude formula_22 and complex frequency formula_23, the output will be some complex constant times the input, say formula_24 for some new complex amplitude formula_25. The ratio formula_26 is the transfer function at frequency formula_23. Since sinusoids are a sum of complex exponentials with complex-conjugate frequencies, if the input to the system is a sinusoid, then the output of the system will also be a sinusoid, perhaps with a different amplitude and a different phase, but always with the same frequency upon reaching steady-state. LTI systems cannot produce frequency components that are not in the input . LTI system theory is good at describing many important systems. Most LTI systems are considered "easy" to analyze, at least compared to the time-varying and/or nonlinear case. Any system that can be modeled as a linear differential equation with constant coefficients is an LTI system. Examples of such systems are electrical circuits made up of resistors, inductors, and capacitors (RLC circuits). Ideal spring–mass–damper systems are also LTI systems, and are mathematically equivalent to RLC circuits. Most LTI system concepts are similar between the continuous-time and discrete-time (linear shift-invariant) cases. In image processing, the time  variable is replaced with two space variables, and the notion of time invariance is replaced by two-dimensional shift invariance. When analyzing filter banks and MIMO systems, it is often useful to consider vectors of signals. A linear system that is not time-invariant can be solved using other approaches such as the Green function method. Continuous-time systems. Impulse response and convolution. The behavior of a linear, continuous-time, time-invariant system with input signal "x"("t") and output signal "y"("t") is described by the convolution integral: where formula_28 is the system's response to an impulse: formula_29. formula_30 is  therefore proportional to a weighted average of the input function formula_31. The weighting function is formula_32, simply shifted by amount formula_33. As formula_33 changes, the weighting function emphasizes different parts of the input function. When formula_35 is zero for all negative formula_36, formula_37 depends only on values of formula_38 prior to time formula_33, and the system is said to be causal. To understand why the convolution produces the output of an LTI system, let the notation formula_40 represent the function formula_41 with variable formula_42 and constant formula_36. And let the shorter notation formula_44 represent formula_45. Then a continuous-time system  transforms an input function, formula_46 into an output function, formula_47. And in general, every value of the output can depend on every value of the input. This concept is represented by: formula_48 where formula_49 is the transformation operator for time formula_33. In a typical system, formula_37 depends most heavily on the values of formula_38 that occurred near time formula_33. Unless the transform itself changes with formula_33, the output function is just constant, and the system is uninteresting. For a linear system, formula_55 must satisfy : And the time-invariance requirement is: </math> In this notation, we can write the  impulse response as formula_56 Similarly: Substituting this result into the convolution integral: formula_57 which has the form of the right side of for the case formula_58 and formula_59 formula_60 In summary, the input function, formula_44, can be represented by a continuum of time-shifted impulse functions, combined "linearly", as shown at . The system's linearity property allows the system's response to be represented by the corresponding continuum of impulse responses, combined in the same way. And the time-invariance property allows that combination to be represented by the convolution integral. The mathematical operations above have a simple graphical simulation . Exponentials as eigenfunctions. An eigenfunction is a function for which the output of the operator is a scaled version of the same function. That is, formula_62 where "f" is the eigenfunction and formula_63 is the eigenvalue, a constant. The exponential functions formula_64, where formula_65, are eigenfunctions of a linear, time-invariant operator. A simple proof illustrates this concept. Suppose the input is formula_66. The output of the system with impulse response formula_19 is then formula_68 which, by the commutative property of convolution, is equivalent to formula_69 where the scalar formula_70 is dependent only on the parameter "s". So the system 's response is a scaled version of the input. In particular, for any formula_65, the system output is the product of the input formula_72 and the constant formula_73. Hence, formula_64 is an eigenfunction of an LTI system, and the corresponding eigenvalue is formula_73. Direct proof. It is also possible to directly derive complex exponentials as eigenfunctions of LTI systems. Let's set formula_76 some complex exponential and formula_77 a time-shifted version of it. formula_78 by linearity with respect to the constant formula_79. formula_80 by time invariance of formula_81. So formula_82. Setting formula_83 and renaming we get: formula_84 i.e. that  a complex exponential formula_85 as input will give a complex exponential of same frequency as output. Fourier and Laplace transforms. The eigenfunction property of exponentials is very useful for both analysis and insight into LTI systems. The one-sided Laplace transform formula_86 is exactly the way to get the eigenvalues from the impulse response. Of particular interest are pure sinusoids (i.e., exponential functions of the form formula_87 where formula_88 and formula_89). The Fourier transform formula_90 gives the eigenvalues for pure complex sinusoids. Both of formula_73 and formula_92 are called the "system function", "system response", or "transfer function". The Laplace transform  is usually used in the context of one-sided signals, i.e. signals that are zero for all values of "t" less than some value. Usually, this "start time" is set to zero, for convenience and without loss of generality, with the transform integral being taken from zero to infinity (the transform shown above with lower limit of integration of negative infinity is formally known as the bilateral Laplace transform). The Fourier transform is used for analyzing systems that process signals that are infinite in extent, such as modulated sinusoids, even though it cannot be directly applied to input and  output signals that are not square integrable. The Laplace transform actually works directly for these signals if they are zero before a start time, even if they are not square integrable, for stable systems. The Fourier transform is often applied to spectra of infinite signals via the Wiener–Khinchin theorem even when Fourier transforms of the signals do not exist. Due to the convolution property of both of these transforms, the convolution that gives the output of the system can be transformed to a multiplication in the transform domain, given signals for which the transforms exist formula_93 One can use  the system response directly to determine how any particular frequency component is handled by a system with that Laplace transform. If we evaluate the system response (Laplace transform of the impulse response) at complex frequency , where , we obtain |"H"("s")| which is the system gain for frequency "f". The relative phase shift between the output and input for that frequency component is likewise given by arg("H"("s")). When the Laplace transform of the derivative is taken, it transforms to a simple multiplication by the Laplace variable "s". Examples. formula_95 That the derivative has such a simple Laplace transform partly explains the utility  of the transform. formula_96 By the linearity of integration, formula_97 it is linear. Additionally, because formula_98 it is time invariant. In fact, formula_99 can be written as a convolution with the boxcar function formula_100. That is, formula_101 where the boxcar function formula_102 Important system properties. Some of the most important properties of a system are causality and stability. Causality is a necessity for a physical system whose independent variable is time, however this restriction is not present in other cases such as image processing. Causality. A system is causal if the output depends only on present and past, but not  future inputs. A necessary and sufficient condition for causality is formula_103 where formula_19 is the impulse response. It is not possible in general to determine causality from the two-sided Laplace transform. However when working in the time domain one normally uses the one-sided Laplace transform which requires causality. Stability. A system is bounded-input, bounded-output stable (BIBO stable) if, for every bounded input, the output is finite. Mathematically, if every input satisfying formula_105 leads to an output satisfying formula_106 (that is, a finite maximum absolute value of formula_1 implies a finite maximum absolute value of formula_2), then  the system is stable. A necessary and sufficient condition is that formula_19, the impulse response, is in L (has a finite L norm): formula_110 In the frequency domain, the region of convergence must contain the imaginary axis formula_111. As an example, the ideal low-pass filter with impulse response equal to a sinc function is not BIBO stable, because the sinc function does not have a finite L norm. Thus, for some bounded input, the output of the ideal low-pass filter is unbounded. In particular, if the input is zero for formula_112 and equal to a sinusoid at the  cut-off frequency for formula_113, then the output will be unbounded for all times other than the zero crossings. Discrete-time systems. Almost everything in continuous-time systems has a counterpart in discrete-time systems. Discrete-time systems from continuous-time systems. In many contexts, a discrete time (DT) system is really part of a larger continuous time (CT) system. For example, a digital recording system takes an analog sound, digitizes it, possibly processes the digital signals, and plays back an analog sound for people to listen to. In practical systems, DT signals obtained are usually uniformly sampled versions of  CT signals. If formula_1 is a CT signal, then the sampling circuit used before an analog-to-digital converter will transform it to a DT signal: formula_115 where "T" is the sampling period. Before sampling, the input signal is normally run through a so-called Nyquist filter which removes frequencies above the "folding frequency" 1/(2T); this guarantees that no information in the filtered signal will be lost. Without filtering, any frequency component "above" the folding frequency (or Nyquist frequency) is aliased to a different frequency (thus distorting the original signal), since a DT signal can only support frequency components  lower than the folding frequency. Impulse response and convolution. Let formula_116 represent the sequence formula_117 And let the shorter notation formula_118 represent formula_119 A discrete system transforms an input sequence, formula_118 into an output sequence, formula_121 In general, every element of the output can depend on every element of the input. Representing the transformation operator by formula_122, we can write: formula_123 Note that unless the transform itself changes with "n", the output sequence is just constant, and the system is uninteresting. (Thus the subscript, "n".) In a typical system, "y"["n"] depends most heavily on the elements of "x" whose indices  are near "n". For the special case of the Kronecker delta function, formula_124 the output sequence is the impulse response: formula_125 For a linear system, formula_122 must satisfy: And the time-invariance requirement is: In such a system, the impulse response, formula_127, characterizes the system completely. That is, for any input sequence, the output sequence can be calculated in terms of the input and the impulse response. To see how that is done, consider the identity: formula_128 which expresses formula_118 in terms of a sum of weighted delta functions. Therefore: formula_130 where we have invoked for the case formula_131 and  formula_132. And because of , we may write: formula_133 Therefore: which is the familiar discrete convolution formula. The operator formula_134 can therefore be interpreted as proportional to a weighted average of the function "x"["k"]. The weighting function is "h"[−"k"], simply shifted by amount "n". As "n" changes, the weighting function emphasizes different parts of the input function. Equivalently, the system's response to an impulse at "n"=0 is a "time" reversed copy of the unshifted weighting function. When "h"["k"] is zero for all negative "k", the system is said to be causal. Exponentials as eigenfunctions. An eigenfunction is a function for  which the output of the operator is the same function, scaled by some constant. In symbols, formula_135 where "f" is the eigenfunction and formula_63 is the eigenvalue, a constant. The exponential functions formula_137, where formula_138, are eigenfunctions of a linear, time-invariant operator. formula_139 is the sampling interval, and formula_140. A simple proof illustrates this concept. Suppose the input is formula_141. The output of the system with impulse response formula_142 is then formula_143 which is equivalent to the following by the commutative property of convolution formula_144 where formula_145 is dependent only on the parameter "z". So formula_146 is an eigenfunction  of an LTI system because the system response is the same as the input times the constant formula_147. Z and discrete-time Fourier transforms. The eigenfunction property of exponentials is very useful for both analysis and insight into LTI systems. The Z transform formula_148 is exactly the way to get the eigenvalues from the impulse response. Of particular interest are pure sinusoids; i.e. exponentials of the form formula_149, where formula_88. These can also be written as formula_146 with formula_152. The discrete-time Fourier transform (DTFT) formula_153 gives the eigenvalues of pure sinusoids. Both of formula_147 and formula_155 are called the  system function, "system response", or "transfer function". Like the one-sided Laplace transform, the Z transform is usually used in the context of one-sided signals, i.e. signals that are zero for t<0. The discrete-time Fourier transform Fourier series may be used for analyzing periodic signals. Due to the convolution property of both of these transforms, the convolution that gives the output of the system can be transformed to a multiplication in the transform domain. That is, formula_156 Just as with the Laplace transform transfer function in continuous-time system analysis, the Z transform makes it easier to analyze  systems and gain insight into their behavior. The Z transform of the delay operator is a simple multiplication by "z". That is, Examples. formula_159 formula_160 Because of the linearity of sums, formula_161 and so it is linear. Because, formula_162 it is also time invariant. Important system properties. The input-output characteristics of discrete-time LTI system are completely described by its impulse response formula_142. Two of the most important properties of a system are causality and stability. Non-causal (in time) systems can be defined and analyzed as above, but cannot be realized in real-time. Unstable systems can  also be analyzed and built, but are only useful as part of a larger system whose overall transfer function "is" stable. Causality. A discrete-time LTI system is causal if the current value of the output depends on only the current value and past values of the input. A necessary and sufficient condition for causality is formula_164 where formula_142 is the impulse response. It is not possible in general to determine causality from the Z transform, because the inverse transform is not unique. When a region of convergence is specified, then causality can be determined. Stability. A system is bounded  input, bounded output stable (BIBO stable) if, for every bounded input, the output is finite. Mathematically, if formula_166 implies that formula_167 (that is, if bounded input implies bounded output, in the sense that the maximum absolute values of formula_168 and formula_169 are finite), then the system is stable. A necessary and sufficient condition is that formula_142, the impulse response, satisfies formula_171 In the frequency domain, the region of convergence must contain the unit circle (i.e., the locus satisfying formula_172 for complex "z").