{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In Lecture 20 of mcs 320, we apply memoization to make recursive functions more efficient."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. Recursive Functions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The Fibonacci numbers are defined as $F(0) = 0$, $F(1) = 1$,\n",
    "and for $n > 1$: $F(n) = F(n-1) + F(n-2)$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below is a basic Python function, that follows the recursive definition."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "def F(n):\n",
    "    \"\"\"\n",
    "    Returns the n-th Fibonacci number.\n",
    "    \"\"\"\n",
    "    if n == 0:\n",
    "        return 0\n",
    "    elif n == 1:\n",
    "        return 1\n",
    "    else:\n",
    "        return F(n-1) + F(n-2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us look at the first ten Fibonacci numbers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0, 1, 1, 2, 3, 5, 8, 13, 21, 34]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[F(k) for k in range(10)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The main problem with the definition is that it is too inefficient."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "25 loops, best of 3: 7.92 ms per loop"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "timeit('F(20)')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "25 loops, best of 3: 13 ms per loop"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "timeit('F(21)')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "25 loops, best of 3: 21.6 ms per loop"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "timeit('F(22)')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "25 loops, best of 3: 34.5 ms per loop"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "timeit('F(23)')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Just to compute the next number in the sequence takes significantly more time."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The reason for this high cost is the repeated function calls.\n",
    "Consider the computation of $F(2) = F(0) + F(1) = 0 + 1$.\n",
    "As both $F(0)$ and $F(1)$ are base cases in the recursion,\n",
    "the calls $F(0)$ and $F(1)$ are leaves of the binary tree of calls."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "  _F(2)__\n",
       " /       \\\n",
       "F(0)      F(1)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "L0 = LabelledBinaryTree([], label='F(0)')\n",
    "L1 = LabelledBinaryTree([], label='F(1)')\n",
    "L2 = LabelledBinaryTree([L0, L1], label='F(2)')\n",
    "ascii_art(L2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Consider $F(3)$, $F(4)$, and $F(5)$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "       _______________F(5)_______________\n",
       "      /                                  \\\n",
       "  ___F(3)___                       _______F(4)________\n",
       " /          \\                     /                   \\\n",
       "F(1)        _F(2)__             _F(2)__             ___F(3)___\n",
       "           /       \\           /       \\           /          \\\n",
       "          F(0)      F(1)      F(0)      F(1)      F(1)        _F(2)__\n",
       "                                                             /       \\\n",
       "                                                            F(0)      F(1)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "L3 = LabelledBinaryTree([L1, L2], label='F(3)')\n",
    "L4 = LabelledBinaryTree([L2, L3], label='F(4)')\n",
    "L5 = LabelledBinaryTree([L3, L4], label='F(5)')\n",
    "ascii_art(L5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From the tree of function calls, we see many repeated instances\n",
    "of $F(0)$ and $F(1)$.  Observe too that $F(3)$ is computed twice, \n",
    "and $F(2)$ is computed three times."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. Memoization in Python"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Memoization is a technique to speed up programs by storing the results of function calls."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will memoize the Fibonacci function $F$,\n",
    "using a dictionary as an extra argument of the function\n",
    "to store the results of the function calls."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "def memoizedF(n, D={}):\n",
    "    \"\"\"\n",
    "    All calls made to memoizedF() are stored in D.\n",
    "    \"\"\"\n",
    "    if n in D:       # dictionary lookup\n",
    "        return D[n]\n",
    "    else:\n",
    "        if n == 0:\n",
    "            result = 0\n",
    "        elif n == 1:\n",
    "            result = 1\n",
    "        else:\n",
    "            result = memoizedF(n-1) + memoizedF(n-2)\n",
    "        D[n] = result # store the result in the dictionary\n",
    "        return result"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us check the computation of the first 10 Fibonacci numbers again."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0, 1, 1, 2, 3, 5, 8, 13, 21, 34]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[memoizedF(k) for k in range(10)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "625 loops, best of 3: 217 ns per loop"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "timeit('memoizedF(100)')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. Memoization in SageMath"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Consider the Chebyshev polynomials, defined as\n",
    "$T(0,x)=1$, $T(1,x) = x$ and for degree $n > 1$:\n",
    "$T(n,x) = 2 x T(n−1,x) − T(n−2,x)$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The Chebyshev polynomials are already available."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1,\n",
       " x,\n",
       " 2*x^2 - 1,\n",
       " 4*x^3 - 3*x,\n",
       " 8*x^4 - 8*x^2 + 1,\n",
       " 16*x^5 - 20*x^3 + 5*x,\n",
       " 32*x^6 - 48*x^4 + 18*x^2 - 1,\n",
       " 64*x^7 - 112*x^5 + 56*x^3 - 7*x,\n",
       " 128*x^8 - 256*x^6 + 160*x^4 - 32*x^2 + 1,\n",
       " 256*x^9 - 576*x^7 + 432*x^5 - 120*x^3 + 9*x]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[chebyshev_T(k,x) for k in range(10)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us define our own, efficient definition of the \n",
    "Chebyshev polynomials, with the definition of a function ``T``."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Because Chebyshev polynomials are defined by two parameters:\n",
    "\n",
    "1. ``n`` is the degree of the polynomial,\n",
    "\n",
    "2. ``x`` is the symbol of the variable in the polynomial,\n",
    "\n",
    "the keys of the dictionary are tuples ``(n, x)``."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "def T(n, x, D={}):\n",
    "    \"\"\"\n",
    "    Memoized Chebyshev function of degree n in x.\n",
    "    \"\"\"\n",
    "    if (n, x) in D:\n",
    "        return D[(n, x)]\n",
    "    else:\n",
    "        if n == 0:\n",
    "            result = 1\n",
    "        elif n == 1:\n",
    "            result = x\n",
    "        else:\n",
    "            result = expand(2*x*T(n-1,x) - T(n-2,x)) # normalize\n",
    "        D[(n, x)] = result # store the normalized result\n",
    "        return result"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Observe the normalization of the Chebyshev polynomial."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1,\n",
       " x,\n",
       " 2*x^2 - 1,\n",
       " 4*x^3 - 3*x,\n",
       " 8*x^4 - 8*x^2 + 1,\n",
       " 16*x^5 - 20*x^3 + 5*x,\n",
       " 32*x^6 - 48*x^4 + 18*x^2 - 1,\n",
       " 64*x^7 - 112*x^5 + 56*x^3 - 7*x,\n",
       " 128*x^8 - 256*x^6 + 160*x^4 - 32*x^2 + 1,\n",
       " 256*x^9 - 576*x^7 + 432*x^5 - 120*x^3 + 9*x]"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[T(k, x) for k in range(10)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "625 loops, best of 3: 474 ns per loop"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "timeit('T(100, x)')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As before, we see that this memoized version is very efficient."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "SageMath 10.3",
   "language": "sage",
   "name": "sagemath"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}