Linear Programming

After the first week of project presentations, we start the sixth week looking at mathematical optimization.

Optimization Problems

A linear programming problem is particular form of a mathematical optimization problem, motivated by production planning. We want to maximize profit and minimize cost, expressed as

\[\begin{split}\begin{array}{lcl} \mbox{maximize profit} & & \mbox{minimize cost} \\ \mbox{subject to} & & \mbox{subject to} \\ \hspace{5mm} \mathrm{E}({\bf x}) = {\bf a} & equalities & \hspace{5mm} \mathrm{E}({\bf x}) = {\bf a} \\ \hspace{5mm} \mathrm{I}({\bf x}) \leq {\bf b} & inequalities & \hspace{5mm} \mathrm{I}({\bf x}) \geq {\bf b} \end{array}\end{split}\]

Definition of a linear programming problem.

A linear programming problem is an optimization problem

• with one linear objective functions; and

• all constraints (equalities and inequalities) are linear.

Viewing linear programming geometrically, the linear constraints define a polyhedron, illustrated by Fig. 44.

Fig. 44 A polyhedron as the intersection of half planes.

We want the optimization problem to be feasible and well posed defined below:

1. If the polyhedron is not empty, then the problem is feasible.

2. If the polyhedron is bounded, then the problem is well posed.

Consider the standard form for a linear minimization problem:

\[\begin{split}\begin{array}{lcl} {\rm min}~ {\bf c}^T {\bf x} & & {\bf c}, {\bf x} \in {\mathbb R}^n, \\ {\rm subject~to}~ A {\bf x} \geq {\bf b} & & A \in {\mathbb R}^{m \times n}, {\bf b} \in {\mathbb R}^m. \end{array}\end{split}\]

To turn the inequalities into equalities, we add slack variables \({\bf z} \in {\mathbb R}^m\):

\[A {\bf x} \geq {\bf b} \quad {\rm becomes} \quad A {\bf x} - {\bf z} = {\bf b}, \ {\bf z} \geq {\bf 0}.\]

To make all variables positive, split \({\bf x} = {\bf x}^+ - {\bf x}^-\), where \({\bf x}^+ \geq {\bf 0}$, ${\bf x}^- \geq {\bf 0}\):

\[\begin{split}\begin{array}{lcl} {\rm min} ~ \left[ {\bf c} ~~ {-\bf c} ~~ {\bf 0} \right] \left[ \begin{array}{l} {\bf x}^+ \\ {\bf x}^- \\ {\bf z} \end{array} \right] \\ {\rm subject~to}~ \left[ A ~~ -A ~~ -I \right] \left[ \begin{array}{l} {\bf x}^+ \\ {\bf x}^- \\ {\bf z} \end{array} \right] = {\bf b}, & & \left[ \begin{array}{l} {\bf x}^+ \\ {\bf x}^- \\ {\bf z} \end{array} \right] \geq {\bf 0}. \\ \end{array}\end{split}\]

A classical application of linear programming is the diet problem. The problem is to plan healthy but economical meals. We want to determine the number of quantities of each nutrient, subject to the minimal requirements, minimizing the cost.

\[\begin{split}\begin{array}{c|cccc|c} \mbox{nutritional} & \multicolumn{4}{c|}{\mbox{food quantities}} & \mbox{minimum} \\ \mbox{elements} & x_1 & x_2 & \cdots & x_n & \mbox{requirements} \\ \hline 1 & a_{11} & a_{12} & \cdots & a_{1n} & c_1 \\ 2 & a_{21} & a_{22} & \cdots & a_{2n} & c_2 \\ \vdots & \vdots & \vdots & \ddots & \vdots & \vdots \\ m & a_{m1} & a_{m2} & \cdots & a_{mn} & c_m \\ \hline \mbox{price} & p_1 & p_2 & \cdots & p_n & \end{array}\end{split}\]

Typically \(m > n\) in the linear programming problem:

\[\begin{split}\begin{array}{l} \min p_1 x_1 + p_2 x_2 + \cdots + p_n x_n \\ \mbox{subject to} \\ a_{i1} x_1 + a_{i2} x_2 + \cdots + a_{in} x_n \geq c_i, i=1,2,\ldots,m. \end{array}\end{split}\]

The Simplex Algorithm

The main idea of the simplex algorithm is to move from one vertex to another vertex in the polyhedron, improving the objective value in each move.

0. Bring the problem in standard form:

   \({\rm min}~ {\bf c}^T {\bf x}\) subject to \(A{\bf x} = {\bf b}\), \({\bf x} \geq {\bf 0}\).

1. Find one feasible point.

2. Find one feasible vertex point.

3. Move to vertex with improved objective value, repeat until at optimum.

To illustrate the simple algorithm, we consider a simple example:

\[\begin{split}\begin{array}{l} \max 3 x_1 + 5 x_2 \\ \mbox{subject to} \\ \left\{ \begin{array}{l} \hphantom{3} x_1~ \hphantom{~+ 4 x_2} \leq 8 \\ \hphantom{3 x_1 + 4} x_2 \leq 6 \\ 3 x_1 + 4 x_2 \leq 36 \\ \hphantom{x_1 \geq 0,} ~x_1 \geq 0 \\ \hphantom{x_1 \geq 0,} ~x_2 \geq 0 \end{array} \right. \end{array}\end{split}\]

Imagine a production plan:

• \(x_1\) can be sold at \(\$3\), while \(x_2\) yields \(\$5\) per unit.

• We have constraints on labor resources, storage capacity, etc.

The feasible region, the objective function and the steps in the simple algorithm are illustrated in Fig. 45.

Fig. 45 The feasible region is bounded by the blue lines. All points on the red lines have the same value of the objective. We move from (0,0) to (0,6), and then to (4,6). At the optimal solution we can no longer increase the value of the objective.

Operations Research in Julia

• JuMP is a modeling language and collection of packages for mathematical optimization in Julia.

• GLPK is used for linear programming. GLPK is the GNU Linear Programming Kit, free software.

• The first tutorial is an example of the diet problem.

• Changhyun Kwon: Julia Programming for Operations Research, second edition. Free to read online, examples on github.

Duality

Every LP problem has a dual formulation and interpretation. Mathematically this is expressed as below:

\[\begin{split}\begin{array}{l} {\rm max}~{\displaystyle \sum_{j=1}^n c_j x_j} \\ {\rm subject~to} \\ \quad \left\{ \begin{array}{r} {\displaystyle \sum_{j=1}^n a_{ij} x_j \leq b_i},~~~ \\ {~}^{i = 1,2,\ldots,m} \\ x_j \geq 0, {~}_{j=1,2,\ldots,n} \end{array} \right. \end{array} \quad \quad \begin{array}{l} {\rm min}~{\displaystyle \sum_{i=1}^m b_i y_i} \\ {\rm subject~to} \\ \quad \left\{ \begin{array}{r} {\displaystyle \sum_{j=1}^m a_{ji} y_j \geq c_i},~~~ \\ {~}^{i = 1,2,\ldots,n} \\ y_j \geq 0, {~}_{j=1,2,\ldots,m} \end{array} \right. \end{array}\end{split}\]

The economic interpretation is as follows:

• \(x_j\) : level of activity \(j\),

• \(c_j\) : unit profit of activity \(j\),

• \(b_i\) : amount available of resource \(i\),

• \(a_{ij}\) : amount of resource \(i\) consumed by each unit of activity \(j\),

• \(y_i\) : contribution to profit per unit of resource \(i\).

The dual of our numerical example:

\[\begin{split}\begin{array}{l} \max 3 x_1 + 5 x_2 \\ \mbox{subject to} \\ \left\{ \begin{array}{l} \hphantom{3} x_1~ \hphantom{~+ 4 x_2} \leq 8 \\ \hphantom{3 x_1 + 4} x_2 \leq 6 \\ 3 x_1 + 4 x_2 \leq 36 \\ \hphantom{x_1 \geq 0,} ~x_1 \geq 0 \\ \hphantom{x_1 \geq 0,} ~x_2 \geq 0 \end{array} \right. \end{array} \quad \quad \mbox{is} \quad \quad \begin{array}{l} \min 8 y_1 + 6 y_2 + 36 y_3 \\ \mbox{subject to} \\ \left\{ \begin{array}{r} y_1 \hphantom{+ y_2} + 3 y_3 \geq 3 \\ \hphantom{y_1 +} y_2 + 4 y_3 \geq 5 \\ \hphantom{y_1 \geq 0,} ~y_1 \geq 0 \\ \hphantom{y_1 \geq 0,} ~y_2 \geq 0 \\ \hphantom{y_1 \geq 0,} ~y_3 \geq 0 \end{array} \right. \end{array}\end{split}\]

Proposals of Project Topics

1. Read the paper by Ernest Koenigsberg: Some Industrial Applications of Linear Programming. Operational Research Society, Vol. 12, No. 2, pages 105-114, 1961.

   1. Select one problem and explain the LP formulation, in your own words.

   2. Work out an example with numerical data.

   3. Use JuMP to solve an instance of the same dimensions as in the paper.

   4. Report on the computational cost of the problem.

2. Read the first section of the book by Robert J. Vanderbei: Linear Programming. Foundations and Extensions. Fifth Edition, Springer-Verlag, 2020.

   1. Explain why the management of a production facility translates into an LP problem, in your own words.

   2. Setup a program to generate numerical data, for any dimension.

   3. Use JuMP to solve the generated instances.

   4. Report on the computational cost for various dimensions.

Exercises

1. Suppose General Motors makes a profit of \(\$100\) on each Chevrolet, \(\$200\) on each Buick, and \(\$400\) on each Cadillac.

   • These cars get 20, 17, and 14 miles a gallon respectively.

   • It takes respectively 1, 2, and 3 minutes to assemble one Chevrolet, one Buick, and one Cadillac.

   • Assume the company is mandated by the government that the average car has a fuel efficiency of at least 18 miles a gallon.

   Under these constraints, determine the optimal number of cars, maximizing the profit, which can be assembled in one 8-hour day. Formulate the linear programming problem.

2. Consider the diet problem, look at the tutorial example of JuMP.

   1. Write the dual of that example of the diet problem.

   2. What are the units of the dual variables?

3. Suppose an investor has a choice between three types of shares.

   Type A pays 4 percent, type B pays 6 percent, and type C pays 9 percent interest.

   The investor has \(\$100,000\) available to buy shares and wants to maximize the interest, under the following constraints:

   1. no more than \(\$20,000\) can be spent on shares of type C, and

   2. at least \(\$10,000\) should be spent on shares of type A.

   Answer the following questions:

   1. Give the mathematical description of the optimization problem.

   2. Bring the problem into the standard form.

   3. Solve the linear programming problem.