Review for the Final Exam
=========================

The questions following in the overview
are representative for the style of final exam questions,
suitable for what can be covered in a two hour session.
We have covered many more topics than the content of
those five sample review questions.
Homework problems must be reviewed as well.

Overview
--------

Although the final exam covers everything, in this review 
we consider the most important problems.

In addition to this review:

0. Install all Julia packages we have used.

1. Review the homework problems.

2. Consider the review for the midterm exam.

3. Answers to the midterm exam are posted.

One of the questions on the final exam will be
Describe in one paragraph what you learned 
from *another* project presentation.

The Quality of Watches
----------------------

Suppose we measure the quality of watches as the deviation
from the time at noon after one month without adjustments.
Watches which show a deviation of more than 3 minutes have
to be discarded at a cost of :math:`\$10`.

We have to choose between two ways to produces watches:

1. Watches produced by process :math:`A` give after one month an average
   time of :math:`12\!:\!01` with a standard deviation of 1 minute.

2. Watches produced by process :math:`B` show :math:`12\!:\!00` on average,
   but with a standard deviation of 2 minutes.

Which process is best? :math:`A` or :math:`B`?

1. Set up the quality loss function.

2. Use the expected loss to choose which process is best.

To solve this question, we start with
the quality loss function.

The quality loss function is :math:`L(X) = k(X - \theta)^2` where

* :math:`X` models the production,

* :math:`k` is the loss coefficient,

* :math:`\theta` is the target value of the production,

has expected 
loss :math:`E[L(X, \theta)] = k \sigma^2 + k (\mu - \theta)^2`, where

* :math:`\mu` is the mean value of the production,

* :math:`\sigma` is the standard deviation of the production.

The target :math:`\theta$ is $12\!:\!00`.

1. :math:`X = A`: :math:`\mu_A = 12\!:\!01`, :math:`\sigma_A = 1` minute.

2. :math:`X = B`: :math:`\mu_B = 12\!:\!00`, :math:`\sigma_B = 2` minutes.

The quality loss coefficient :math:`k` is the parameter in

.. math::

   L(x) = k (x - 12)^2.

The loss of a watch that deviates more than 3 minutes is :math:`\$10`.

.. math::

   L(15) = \$10 
   \quad \Rightarrow \quad k 3^2 = 10
   \quad \Rightarrow \quad k = 10/9

Then the expected loss is
:math:`\displaystyle E[L(X, \theta)] = \frac{10}{9} \sigma^2
+ \frac{10}{9} (\mu - \theta)^2`.

1. :math:`X = A`: :math:`\mu_A = 12\!:\!01`, :math:`\sigma_A = 1` minute.

   .. math::

      E[L(A, 12\!:\!00)] = \frac{10}{9} 1^2
      + \frac{10}{9} (12\!:\!01 - 12\!:\!00)^2 = \frac{20}{9}

2. :math:`X = B`: :math:`\mu_B = 12\!:\!00`, :math:`\sigma_B = 2` minutes.

   .. math::

      E[L(B, 12\!:\!00)] = \frac{10}{9} 2^2
      + \frac{10}{9} (12\!:\!00 - 12\!:\!00)^2 = \frac{40}{9}

Because the expected loss for process :math:`B` is higher than
for :math:`A`, process :math:`A` is best.

Maximizing Profit
-----------------

A manufacturer produces two types of products,
say :math:`A` and :math:`B`.
:math:`A` sells at :math:`\$5` per unit,
while :math:`B` sells at :math:`\$7` per unit.
We have to decide how many units of each product to produce,
with the aim of maximizing the profit, subject to constraints:

1. We must produce at least 20 units of :math:`A`, and
   offer at least 10 units of :math:`B`.

2. Our storage capacity is limited to 250 units.

3. We have only 600 hours of labor available, 

   * one unit of :math:`A` requires two hours of labor,

   * while one unit of :math:`B` takes three hours of labor.

Do the following:

1. Set up the linear programming model for this problem.

2. Formulate the dual problem.

Our objective is to maximize profit.

* Let :math:`x_A` be the number of units we produce of :math:`A`.

* Let :math:`x_B` be the number of units we produce of :math:`B`.

:math:`A` sells at :math:`\$5` per unit,
while :math:`B` sells at :math:`\$7` per unit.
So we then arrive at the objective

.. math::

   \max 5 x_A + 7 x_B.

The constraints are formulated as follows:

1. We must produce at least 20 units of :math:`A`, and
   offer at least 10 units of :math:`B`.

   .. math::

      x_A \geq 20 \quad \mbox{and} \quad x_B \geq 10

2. Our storage capacity is limited to 250 units.

   .. math::

      x_A + x_B \leq 250

3. We have only 600 hours of labor available, 

   * one unit of :math:`A` requires two hours of labor,

   * while one unit of :math:`B` takes three hours of labor.

   .. math::

      2 x_A + 3 x_B \leq 600

The maximization problem in standard form is then

.. math::

   \max 5 x_A + 7 x_B

subject to

.. math::

   \begin{array}{rcl}
   - x_A \hphantom{- x_B} & \leq & -20 \\
   \hphantom{-x_A} - x_B & \leq & -10 \\
   x_A + \hphantom{3} x_B & \leq & 250 \\
   2 x_A + 3 x_B & \leq & 600
   \end{array}

The dual of the maximization problem
is the minimization problem

.. math::

   \min -20 y_1 - 10 y_2 + 250 y_3 + 600 y_4

subject to

.. math::

   \begin{array}{rcl}
   - y_1 \hphantom{- y_2} + y_3 + 2 y_4 & \geq & 5 \\
   \hphantom{- y_1} - y_2 + y_3 + 3 y_4 & \geq & 7.
   \end{array}

Is an Investment Worth It?
--------------------------

An investment of :math:`\$3,000` in information technology is expected
to increase the monthly revenue by :math:`\$200`.
Suppose the equipment has a life span of five years.

1. Use a discount rate of :math:`3\%` to determine the payback period.

2. What is the return on investment?

The payback period is the number of months after which 
the increased revenue surpasses the cost of the investment.

1. Compute the present value of the increase in revenue,

2. compounding at the monthly rate of :math:`0.03/12`.

The present value :math:`p(m)` of the total increase in revenue
at month :math:`m` is

.. math::

   p(m) = \sum_{i=1}^m \left( \$200 e^{-i 0.03/12} \right),

where :math:`\$200 e^{-i 0.03/12}` is the present value
of the :math:`\$200` of the *i*-th month.

In Julia:

::

   julia> p(m) = sum([200*exp(-i*0.03/12) for i=1:m])
   p (generic function with 1 method)

   julia> p(15)
   2940.76755767211

   julia> p(16)
   3132.925445502574

We see that after 16 months, the present value of the increase
in revenue has surpassed :math:`\$3,000`, the cost of the investment.

The return on investment is the discount rate that yields zero savings.

* The cost of the investment is :math:`\$3,000`.

* The life span is five years, or 60 months.

For the present value :math:`p(m)` at :math:`m` months, we solve

.. math::

   p(60) = \$3,000,

where the discount rate :math:`d` is a variable:

.. math::

   p(60) = \sum_{i=1}^{60} \left( \$200 e^{-i~\!d/12} \right)
         = \$3,000.

We now solve for :math:`d` in :math:`\displaystyle
\sum_{i=1}^{60} \left( \$200 e^{-i~\!d/12} \right) - \$3,000 = 0`.

::

   using NLsolve

   """
       function f!(F, x)

   defines the input to nlsolve.
   """
   function f!(F, x)
       d = x[1]
       F[1] = 3000 - sum([200*exp(-i*d/12) for i=1:60])
   end

   roi = nlsolve(f!, [0.03])    # start at rate of 3%
   println("ROI : ", roi)

The output is

::

   ROI : Results of Nonlinear Solver Algorithm
    * Algorithm: Trust-region with dogleg and autoscaling
    * Starting Point: [0.03]
    * Zero: [0.7574574772572242]
    * Inf-norm of residuals: 0.000000
    * Iterations: 9
    * Convergence: true
      * |x - x'| < 0.0e+00: false
      * |f(x)| < 1.0e-08: true
    * Function Calls (f): 10
    * Jacobian Calls (df/dx): 10

The interpretation of the ``Zero: [0.7574574772572242]``
is that the return on investment is :math:`75.75\%`.

Predicting the Outcome of a Referendum
--------------------------------------

Consider the geographic prediction of the outcome of a referendum.
Voters have three choices, labeled by

.. math::

   y = -1 \mbox{ (if disapprove), or }
   y = +1 \mbox{ (if approve), or }
   y = 0 \mbox{ (if abstain).}

The pollsters collected 30 coordinates :math:`(x_1, x_2)`,
of 10 voters who disapproved, 10 who approved, and 10 who abstained.
With a neural network we want to geographically predict
the outcome of the referendum.

1. Define the setup of the training data.

2. Draw the two-layer neural network for this prediction,
   using two neurons in the layer following the input layer.

3. What are the dimensions of the neural network?

This is a multiple classes classification problem.
The labels :math:`y = -1, +1, 0` are represented by the vectors

.. math::

   \left[
      \begin{array}{c}
         1 \\ 0 \\ 0
      \end{array}
   \right], \quad
   \left[
      \begin{array}{c}
         0 \\ 1 \\ 0
      \end{array}
   \right], \quad \mbox{and} \quad
   \left[
      \begin{array}{c}
         0 \\ 0 \\ 1
      \end{array}
   \right] \quad \mbox{respectively.}

The training data consists of two matrices :math:`X` and :math:`Y`.

.. math::

   X = \left[
          \begin{array}{ccccc}
             x_{1,1} & x_{1,2} & \cdots & x_{1,30} \\
             x_{2,1} & x_{2,2} & \cdots & x_{2,30} \\
          \end{array}
       \right],

where the :math:`k` columns stores the coordinates
of the *k*-th voter.  The matrix :math:`Y` is a 3-by-30 matrix,
with in the *k*-th column
the vector encoding of the choice of the voter.

The two-layer neural network for this prediction,
using two neurons in the layer following the input layer
is shown in :numref:`figTwoLayerNetwork`.

.. _figTwoLayerNetwork:

.. figure:: ./figTwoLayerNetwork.png
   :align: center

   A neural network with two layers, 2 inputs and 3 outputs.

The weights and the bias vectors are

.. math::

   \mbox{W2} \in {\mathbb R}^{2 \times 2},
   \mbox{W3} \in {\mathbb R}^{3 \times 2},
   \mbox{b2} \in {\mathbb R}^{2},
   \mbox{b3} \in {\mathbb R}^{3},

so we have in total :math:`4 + 6 + 2 + 3 = 15` parameters to determine.

Does the Ship Sink?
-------------------

A crack in the hull of a ship at sea suddenly breaks open 
and the ship starts taking water in. 
As the crack is widening, the rate at which the volume of water 
inside the ship increases is one tenth of the volume already in the ship. 
The pump on board can pump out water at a rate 
of 0.5 :math:`\mbox{m}^3` per second.
When the pump starts pumping water out,
there is already 20 :math:`\mbox{m}^3` of water inside the ship.

1. Set up the ODE to model the evolution of the volume of water
   inside the ship as time advances.

2. Only when the volume of water inside the ship becomes zero,
   one can cover the crack in the hull and save the ship.

3. Solve the ODE.  Will the ship sink?

For the ODE model, we define :math:`V(t)` as the volume of water 
inside the ship in function of time :math:`t`.

Consider the facts:

* The rate at which :math:`V(t)` increases is one tenth of :math:`V(t)`.

* The pump works at the rate of 0.5 :math:`\mbox{m}^3` per second.

* When the pump starts, there is already 20 :math:`\mbox{m}^3` water.

This leads to the initial value problem

.. math::

   \frac{d~\!V}{d~\!t} = \frac{1}{10} V(t) - 0.5, \quad V(0) = 20.

The ODE is solved by the code below:

::

   using SymPy

   t = Sym("t")
   V = SymFunction("V")
   ode = diff(V(t), t) - V(t)/10 + 1/2
   sol = dsolve(ode, V(t), ics=Dict(V(0) => 20))
   println(sol)

The output is ``Eq(V(t), 15.0*exp(t/10) + 5.0``
which gives :math:`V(t) = 15 e^{t/10} + 5`.  The ship sinks.

Proportional Feedback
---------------------

Consider the plant with transfer function

.. math::

   P(s) = \frac{5 s^2 - 88 s - 43}{-73 s^2 + 25 s + 4}.

1. Show that the plant is unstable.

2. Apply the root locus method to find a simple proportional
   feedback constant :math:`\kappa` to stabilize the plant.

The roots of the denominator of

.. math::

   P(s) = \frac{5 s^2 - 88 s - 43}{-73 s^2 + 25 s + 4}

are

.. math::

   \frac{25}{146} - \frac{\sqrt{1793}}{146} \quad \mbox{and} \quad
   \frac{25}{146} + \frac{\sqrt{1793}}{146}.

As the (real part of the) second root is positive,
the plant is unstable.

In the root locus method, we look at the roots of

.. math::

   (-73 s^2 + 25 s + 4) - \kappa (5 s^2 - 88 s - 43)

for increasing values of :math:`\kappa`.  The :math:`\kappa` is 
the parameter in the controller :math:`H(s) = \kappa`.
The closed loop system has transfer function

.. math::

   \frac{P}{1 + \kappa P}.

We apply the adjusted code of an earlier lecture.
to make the plots in :numref:`figRootLocusKappa01`,
:numref:`figRootLocusKappa015`, and :numref:`figRootLocusKappa05`.
From these plots we see that proportional feedback can
stabilize the plant.

.. _figRootLocusKappa01:

.. figure:: ./figRootLocusKappa01.png
   :align: center

   Root locus plot for :math:`\kappa` from 0 to 0.1.

.. _figRootLocusKappa015:

.. figure:: ./figRootLocusKappa015.png
   :align: center

   Root locus plot for :math:`\kappa` from 0 to 0.15.

.. _figRootLocusKappa05:

.. figure:: ./figRootLocusKappa05.png
   :align: center

   Root locus plot for :math:`\kappa` from 0 to 0.5.

A Model of Education
--------------------

Model the education process by a PDE.  

* The function :math:`u(x,t)` models
  the distribution of the scores at time :math:`t`.  

* For :math:`s \in [0,1]`, :math:`\displaystyle \int_0^s u(x,t) dx`
  gives the number of students with score :math:`s` or less.
  The :math:`s` can be interpreted as a percentage.

* At :math:`t=0`, :math:`u(x,t)` obviously corresponds to the Bell
  curve which models the individual IQs.

* As the semester progresses, the profile of the Bell curve changes.

Answer the following questions:

1. What type of PDE would you choose to model the education process?
   Justify your choice.

2. What are the critical parameters in your model?
   How does one characterize the best education process?

Assuming bell shaped profiles of the scores,
as shown in :numref:`figScoreProfiles`.

.. _figScoreProfiles:

.. figure:: ./figScoreProfiles.png
   :align: center

   Bell shaped profiles of the score profiles.

.. math::

   u(x,t) = \frac{1}{\sigma \sqrt{2 \pi}} e^{-(x - \mu)^2/(2 \sigma^2)}

* At time :math:`t = 0`: :math:`\mu = 0.2`, :math:`\sigma = 0.2`.

* At time :math:`t = 1`: :math:`\mu = 0.8`, :math:`\sigma = 0.1`.

The advection diffusion equation is

.. math::

   \frac{\partial u}{\partial t}(x,t)
   + \frac{\partial (v~\! u)}{\partial x}(x,t)
   = k \frac{\partial^2 u}{\partial x^2}, \quad u(x, 0) = u_0(x).

Over time, we expect from the distribution of the scores:

* the mean :math:`\mu` shifts away from the low 0.2 to a higher value,

* the deviation :math:`\sigma` shrinks from 0.2 to a lower value.

The parameters in the model are

* :math:`(\mu, \sigma)`, the mean and deviation of the score distribution.

* :math:`(v, k)`, the speed and diffusion (or better: concentration) constant.

The best education process is characterized by

1. A high value of :math:`v`.

2. A strongly negative value of :math:`k`.

Ideally, it does not take long to get up to speed,
and there is little variation in the capabilities of the students.