Filters and Stability
=====================

All filters we consider are linear, time invariant, and causal.

Linear, Time Invariant, Causal Filters
--------------------------------------

Every signal can be written as a linear combinations of sine functions
running at increasing frequencies, 
as :math:`\sin(2\pi k t)`, :math:`k=1, 2, \ldots`,
see :numref:`figSine1Hz` for the first sine function in this family.
Frequencies are measured in Hertz (Hz):
:math:`n` Hz = :math:`n` cycles per second.

.. _figSine1Hz:

.. figure:: ./figSine1Hz.png
   :align: center

   Sine function running at 1 Hertz, one cycle per second.

Processing a continuous signal happens by taking samples.
Sampling at 8 Hertz means taking 8 equidistant samples per second,
as illustrated in :numref:`figSine1HzSample8Hz`.
Our signal is the represented by
:math:`\{ u_0, u_1, u_2, u_3, u_4, u_5, u_6, u_7, u_0, \ldots \}`,
where :math:`u_k = \sin(2\pi k/8)`, :math:`k=0,1,2,\ldots,7`.

.. _figSine1HzSample8Hz:

.. figure:: ./figSine1HzSample8Hz.png
   :align: center

   Sine function running at 1 Hertz, sampled at 8 Hertz.

We can view the unit delay as a :index:`filter`,
as introduced in :numref:`figDelayFilter`.

.. _figDelayFilter:

.. figure:: ./figDelayFilter.png
   :align: center

   The unit delay as a filter.

The unit delay has as input
:math:`u = \{ u_0, u_1, u_2, \ldots \}`
has z-transform 
:math:`\displaystyle U(z) = \sum_{k=0}^\infty u_k z^{-k}`.
The output :math:`y = \{ 0, u_0, u_1, u_2, \ldots \}` 
has z-transform :math:`Y(z)`, computed below.

.. math::

   \begin{array}{rcl}
   Y(z) & = & {\displaystyle \sum_{k=1}^\infty u_{k-1} z^{-k}} \\
   & = & {\displaystyle z^{-1} \sum_{k=1}^\infty u_{k-1} z^{-(k-1)}}
   ~=~ {\displaystyle z^{-1} \sum_{k=0}^\infty u_k z^{-k}} \\
   & = & z^{-1} U(z).
   \end{array}

Therefore, 
the delay of a signal can be viewed as a filter,
schematically represented in :numref:`figDefinitionFilter`.

.. _figDefinitionFilter:

.. figure:: ./figDefinitionFilter.png
   :align: center

   Schematic representation of a filter.

.. index:: linear filter, time invariant filter, causal filter

.. topic:: Definition of a linear, time invariant, causal filter.

   A *filter* :math:`F` turns input 
   :math:`u = \displaystyle \left\{ u_k \vphantom{u^k} \right\}_{k=0}^\infty`
   into output
   :math:`y = \displaystyle \left\{ y_k \vphantom{y^k} \right\}_{k=0}^\infty`
   and is

   1. linear:
      :math:`F( \alpha u^{(1)} + \beta u^{(2)})
      = \alpha F(u^{(1)}) + \beta F(u^{(2)})`,

   2. time invariant:
      :math:`\displaystyle F\left( 
      \left\{ u_{k+k_0} \vphantom{u^k} \right\}_{k=0}^\infty \right)
      = \left\{ y_{k + k_0} \vphantom{y^k} \right\}_{k=0}^\infty`,
      :math:`k > k_0`, and

   3. causal:
      :math:`u_k = 0, k < k_0 \Rightarrow y_k = 0, k < k_0`.

For good understanding of this definition,
as an exercise, show that the unit delay
is a linear, time invariant, and causal filter.

Unit Impulse and Transfer Function
----------------------------------

The *unit impulse* is :math:`\delta = \{ 1, 0, 0, \ldots \}`.
The *impulse response*
is :math:`h = F \delta = \{ h_0, h_1, h_2, \ldots \}`.
Both :index:`unit impulse` and :index:`impulse response` are shown
in :numref:`figUnitImpulse`.

.. _figUnitImpulse:

.. figure:: ./figUnitImpulse.png
   :align: center

   Schematic representation of the unit impulse and the impulse response.

.. topic:: Theorem charactizing a linear, time invariant, causal filter.

   A linear, time invariant, causal filter is entirely determined
   by the impulse response.

For input :math:`u`, the :math:`k`-th element 
in the output :math:`y` is

.. math::

   y_k = u_k h_0 + u_{k-1} h_1 + \cdots + u_1 h_{k-1} + u_0 h_k
   = \sum_{j=0}^k h_j u_{k-j}.

The convolution is denoted by the 
operator :math:`\star`, as :math:`y = h \star u`.

In the frequency domain, the z-transforms of the input and output are

.. math::

   U(z) = \sum_{k=0}^\infty u_k z^{-k} \quad \mbox{and} \quad
   Y(z) = \sum_{k=0}^\infty y_k z^{-k}.

The impulse response has as z-transform
:math:`\displaystyle H(z) = \sum_{k=0}^\infty h_k z^{-k}`.
The z-transform of :math:`y = h \star u` is then

.. math::

   Y(z) = H(z) U(z).

.. index:: transfer function

.. topic:: Definition of the transfer function.

   The *transfer function of a filter* is the z-transform
   of the impulse response.

Now we can explain the :math:`z^{-1}`
in the schematic :numref:`figDelayFilter`
representing the unit delay as a filter by asking
*What is the transfer function of the unit delay filter?*
The transfer function is the z-transform of the impulse response:

* The response of :math:`\{1, 0, 0, \ldots \}` is 
  :math:`\{0, 1, 0, 0, \ldots \}`.

* The z-transform of :math:`\{0, 1, 0, 0, \ldots \}`
  is :math:`z^{-1}`.

Thus :math:`z^{-1}` is the transfer function of the unit delay filter.

Consider computing the output of a filter.
Denote

.. math::

   \begin{array}{rcl}
   \delta^{(0)} & = & \{ 1, 0, 0, \ldots \}, \\
   \delta^{(1)} & = & \{ 0, 1, 0, \ldots \}, \\
   \delta^{(2)} & = & \{ 0, 0, 1, \ldots \}, \quad \mbox{etc.}
   \end{array}

then we can write any input :math:`u` as
:math:`\displaystyle u = \sum_{k=0}^\infty u_k \delta^{(k)}`.

The output :math:`y = F u` can then be written as

.. math::

   y = F\left( \sum_{k=0}^\infty u_k \delta^{(k)} \right)
     = \sum_{k=0}^\infty u_k F \delta^{(k)},

because :math:`F` is linear.

After decomposing any input :math:`u` as linear combinations
of properly delayed unit impulses,
we construct a formula for the output:

.. math::

   \begin{array}{rcl}
   y & = & u_0 F \delta^{(0)}
   +  u_1 F \delta^{(1)}
   +  u_2 F \delta^{(2)} + \cdots \\
   & = & u_0 \{ h_0, h_1, h_2, \ldots \}
   + u_1 \{ 0, h_0, h_1, h_2, \ldots \} \\
   &   &    +~ u_2 \{ 0, 0, h_0, h_1, h_2, \ldots \} \\
   & = & \{ u_0 h_0, u_0 h_1 + u_1 h_0, u_0 h_2 + u_1 h_1 + u_2 h_0,
   \ldots \} \\
   & = & {\displaystyle \sum_{k=0}^\infty ~
   \underbrace{\sum_{j=0}^k u_j h_{k-j}}_{y_k}}
   \end{array}

The formula for :math:`y_k` is a convolution of the input 
with the impulse response.
We proved that the impulse response entirely determines a filter,
as stated earlier in the theorem characterizing
a linear, time invariant, and causal filter.

Let us now turn to the original function of a filter
and removing a signal shown earlier in :numref:`figSine1HzSample8Hz`.
Our signal is 
:math:`\{ u_0, u_1, u_2, u_3, u_4, u_5, u_6, u_7, u_0, \ldots \}`,
where :math:`u_k = \sin(2\pi k/8)`, :math:`k=0,1,2,\ldots,7`.
Observe, looking at :numref:`figSine1HzSample8Hz`,
the symmetry of the samples: 

.. math::

   u_5 = -u_1, \quad u_6 = -u_2, \quad u_7 = -u_3.

Adding a delay will remove the signal.
Let the output be defined as follows

.. math::

   y_0 = u_0 + u_4, \quad
   y_1 = u_1 + u_5, \quad
   y_2 = u_2 + u_6, \quad
   y_3 = u_3 + u_7.

This results in the removal of the signal, as 
:math:`y_0 = 0`, :math:`y_1 = 0`, :math:`y_2 = 0`, :math:`y_3 = 0`.
The z-transform of :math:`y` is :math:`Y(z)`:

.. math::

   Y(z) = U(z) + z^{-4} U(z) = \left( 1 + z^{-4} \right) U(z).

The transfer function of the filter 
is :math:`H(z) = 1 + z^{-4}`.

Stability
---------

In addition to working with linear, time invariant, and causal filters,
we want filters that are stable.

.. topic:: Definition of a stable filter.

   A filter is *stable* if bounded inputs
   produce bounded outputs.


What are bounded inputs and outputs?  Some examples follow.

* :math:`x = \{ 1, 1, 1, 1, 1, 1, 1, \ldots \}` is bounded,
  and so is any constant signal.

* :math:`x = \{ 1, 2, 4, 8, 16, 32, 64, \ldots \}` is not bounded.

* :math:`\displaystyle x = \left\{ 1, \frac{1}{2}, \frac{1}{4}, \frac{1}{8}, 
  \frac{1}{16}, \frac{1}{32}, \frac{1}{64}, \ldots \right\}` is bounded.

The impulse response determines
a linear, time invariant, causal filter.

.. topic:: The transfer function of a stable filter.

   Let :math:`\displaystyle H(z) = \sum_{k=0}^\infty h_k z^{-k}`
   be the transfer function of a filter :math:`F`.

   .. math::

      F \mbox{ is stable} \quad \Leftrightarrow \quad
      \sum_{k=0}^\infty |h_k| < \infty.

As a corollary, the transfer function :math:`H(z)` 
of a stable filter converges absolutely and uniformly 
for all :math:`|z| \geq 1` to a continuous function.

Amplitude Gain and Phase Shift
------------------------------

The theorem below provides another characterization of a filter.

.. index:: amplitude gain, phase shift

.. topic:: Theorem on the amplitude and phase shift of a filter.

   Let :math:`\displaystyle H(z) = \sum_{k=0}^\infty h_k z^{-k}`
   be the transfer function of a filter :math:`F`.

   For input
   :math:`\displaystyle u = \left\{ u_k = \sin(\omega k T) 
   \vphantom{u^k} \right\}_{k=0}^\infty`,
   :math:`\displaystyle y = \left\{ y_k = r \sin(\omega k T + \phi) 
   \vphantom{y^k} \right\}_{k=0}^\infty`
   is the output, where

   * :math:`r = |H(e^{i \omega T})|` is the amplitude gain, and

   * :math:`\phi = \mbox{arg} H(e^{i \omega T})` is the phase shift.

We interpret the input :math:`u` as 

* :math:`u = \sin(\omega t)`, :math:`\omega = 2 \pi n`,
  at :math:`n` Hertz,

* sampled at :math:`t = kT`, :math:`T` is the sampling rate.

Complex numbers appear, the imaginary unit 
is :math:`i = \sqrt{-1}`.

The proof of the theorem uses complex numbers.
We look at the response of the signal :math:`s = e^{i \omega T}`.

The input 
:math:`\displaystyle u = \left\{ s^k \vphantom{u^k} \right\}_{k=0}^\infty`.
Let us compute :math:`y = h \star u`.

.. math::

   y_k = \sum_{j=0}^k h_j u_{k-j}
   = \sum_{j=0}^k h_j s^{k-j}
   = s^k \sum_{j=0}^k h_j s^{-j}

By the definition of :math:`H`: 
:math:`\displaystyle H(s) = \sum_{j=0}^\infty h_j s^{-j}`.  So, we have:

.. math::

   y_k = s^k H(s) - s^k \sum_{j=k+1}^\infty h_j s^{-j}.

Consider :math:`k \rightarrow \infty`
and apply the complex exponentiation to the sine function.
Our input signal is :math:`s = e^{i \omega T}`.

.. math::

   \begin{array}{cccccr}
   & s & = & e^{i \omega T}  & = & \cos(\omega T) + i \sin(\omega T)
   \hphantom{\left. \right)} \\
   - \left( \right. 
   & s^{-1} & = & e^{-i \omega T} & = & \cos(\omega T) - i \sin(\omega T)
   \left. \right) \\
   \hline
   & s - s^{-1} & & & = & 2 i \sin(\omega T) \hphantom{\left. \right)} 
   \end{array}

It then follows:

.. math::

   \sin(\omega T) = \frac{1}{2i} 
   \left( s - s^{-1} \vphantom{\frac{1}{2}} \right)

and

.. math::

   \sin(k \omega T) = \frac{1}{2i} 
   \left( s^k - s^{-k} \vphantom{\frac{1}{2}} \right).

We derived :math:`y_k = s^k H(s)`.  
Suppose :math:`H(e^{i \omega T}) = r e^{i \phi}`.
What are :math:`r` and :math:`\phi`?

.. math::

   \begin{array}{rcl}
   { \displaystyle \frac{s^k H(s) - s^{-k} H(s^{-1})}{2 i} }
   & = & { \displaystyle \frac{1}{2i} \left( \vphantom{\frac{1}{2}}
   s^k r e^{i \phi} - s^{-k} r e^{-i \phi} \right),
   \quad s = e^{i \omega T} } \\
   & = & { \displaystyle \frac{1}{2i} \left( \vphantom{\frac{1}{2}}
   e^{k i \omega T + \phi} -  e^{-(k i \omega T + \phi)} \right) } \\
   & = & r \sin(k \omega T + \phi)
   \end{array}

Then indeed, :math:`r = |H(e^{i \omega T})|`
and :math:`\phi = \mbox{arg} H(e^{i \omega T})`.
The theorem is proven.

Exercises
---------

1. What is the z-transform of a one Hertz sine function 
   sampled at eight Hertz?

2. Show that the unit delay :math:`F = z^{-1}` is 
   linear, time invariant, and causal.

3. Noise often happens at high frequency.
   Give the transfer function of a filter
   to remove a signal at 60 Hz, sampled at 720 Hz.

   1. Justify the design of your filter.

   2. Which signals are also removed by your filter?

Bibliography
------------

1. Richard C. Dorf and Robert H. Bishop:
   *Modern Control Systems.*  Nineth Edition, Prentice Hall, 2001.

2. Charles R. MacCluer, middle of Chapter 3 of
   *Industrial Mathematics.  Modeling in Industry, Science, and Government*,
   Prentice Hall, 2000.