Enumeration and Backtracking ============================ Enumeration and backtracking are exhaustive problem solving methods, applying recursive algorithms. The exhaustiveness often leads to exponential time. Greedy algorithms and heuristic methods may prune the search space and give satisfactory practical performance. Enumerating Bit Combinations ---------------------------- Consider the automatic generation of truth tables. For example, we can simplify expressions via De Morgan's law: .. math:: {{\tt not~ (( not~x )~or~( not~y ))} ~=~ {\tt x~and~y}} A Truth Table proves De Morgan's law: .. math:: \begin{array}{cc|ccc|cc} {\tt x} & {\tt y} & {\rm (1)} & {\rm (2)} & {\rm (3)} & {\rm (4)} & {\rm (5)} \\ \hline 0 & 0 & 1 & 1 & 1 & 0 & 0 \\ 0 & 1 & 1 & 0 & 1 & 0 & 0 \\ 1 & 0 & 0 & 1 & 1 & 0 & 0 \\ 1 & 1 & 0 & 0 & 0 & 1 & 1 \end{array} where in the table above, we made the following abbreviations: :math:`0 = {\tt False}`, :math:`1 = {\tt True}`, :math:`{\rm (1)} = {\tt not~x}`, :math:`{\rm (2)} = {\tt not~y}`, :math:`{\rm (3)} = {\rm (1)} {\tt ~or~} {\rm (2)}`, :math:`{\rm (4)} = {\tt not~} {\rm (3)}`, :math:`{\rm (5)} = {\tt x~and~y}`. The proof of the law follows from the last two columns in the table being equal to each other. In writing all bit combinations, we will develop the script ``enumbits.py``. Its running is illustrated below. :: $ python3 enumbits.py Give number of bits : 3 [0, 0, 0] [0, 0, 1] [0, 1, 0] [0, 1, 1] [1, 0, 0] [1, 0, 1] [1, 1, 0] [1, 1, 1] Observe the pattern in this enumeration. The lists are list in lexicographic order, as words in a dictinary. In the recursive computation of all bit combinations, we view the bit sequences as the leaves of a binary tree. The internal nodes of the tree are only partially filled in, as in :numref:`figtreebits`. .. _figtreebits: .. figure:: ./figtreebits.png :align: center A binary tree to list all bit combinations. The tree grows from its root to the leaves: * Every leave is a print statement (base case). * Every internal node is a recursive function call. Let's define a recursive function ``enumbits``, to compute a combinations of :math:`n` bits. Two parameters suffice for this function: * ``bits`` is a list which serves as the accumulating parameter; and * ``k`` points to the current position in the list. Initially, ``k == 0`` and ``bits`` has space for :math:`n` objects. in the base case when ``k == len(bits)``, then we execute ``print bits``. In general, for ``k < len(bits)``: 1. ``bits[k] = 0``, call ``enumbits`` for ``k+1`` 2. ``bits[k] = 1``, call ``enumbits`` for ``k+1`` The main program is below: :: def main(): """ Prompts the user for number of bits and initializes the list. """ nbr = int(input('Give the number of bits : ')) lst = [0 for _ in range(nbr)] enumbits(0, lst) The prototype for the recursive function is :: def enumbits(k, bits): """ Writes all bit combinations of len(bits), starting at the k-th position. """ The definition of the parameters and their functionality counts often for more than half of the solution. The complete definition of the function comes next. :: def enumbits(k, bits): """ Writes all bit combinations of len(bits), starting at the k-th position. """ if k >= len(bits): print(bits) else: bits[k] = 0 enumbits(k+1, bits) bits[k] = 1 enumbits(k+1, bits) The first application is the computation of truth tables. We exploit Python's dynamic typing, as in the following session: :: >>> e = "not x or not y" >>> x = True >>> y = False >>> eval(e) True Logical expressions as strings delays their evaluation. We can apply ``eval`` only if all variables in an expression have values. An extension of the bit enumeration is for problems when there are more than two alternatives: Then the number ``N`` of alternatives is an extra parameter. In the general case, the loop ``for k in range(N)`` enumerate all choices. An illustration of combining many alternatives, consider a problem you may encounter when solving a crossword puzzle. Enumerate all three letter words: \begin{enumerate} \item starting with c, s, or v; \item with one vowel in the middle; and \item ending in d, t, or w. \end{enumerate} This problem is solved by the product of three lists: :: csv = ['c', 's', 'v'] vowels = ['a', 'e', 'i', 'o', 'u'] dtw = ['d', 't', 'w'] The number of plausible words equals :math:`3 \times 5 \times 3 = 45`. Our recursive function ``enumwords`` has 3 parameters: 1. ``letters`` is a list of lists to choose from; 2. ``k`` is the index to current list; and 3. ``accu`` is a string accumulating the word. Recall that adding a character ``c`` to a string ``s`` is done simply as ``s + c``. Initially, we have ``k == 0`` and ``accu == ''``. The recursive algorithm then compares ``k`` to ``len(letters)``: * If ``k == len(letter)``, then do ``print accu``. This is the base case. * Otherwise, for all characters ``letter`` in ``letters[k]``, call ``enumwords`` with ``k+1`` and ``accu + letter``. The main function is as follows: :: def main(): """ enumerates letter combinations """ csv = ['c', 's', 'v'] vowels = ['a', 'e', 'i', 'o', 'u'] dtw = ['d', 't', 'w'] let = [csv, vowels, dtw] enumwords(0, let, '') where the function ``enumwords`` is defined below: :: def enumwords(k, letters, accu): """ Starting with the k-th list in letters, adds letters to the current string accu. """ if k >= len(letters): print(accu) else: for letter in letters[k]: enumwords(k+1, letters, accu+letter) Another extension to the recursive enumeriation is to add a stopping condition. Consider a session where we ask the user whether to continue or not. :: $ python3 enumwstop.py the word is "cad" continue ? (y/n) y the word is "cat" continue ? (y/n) y the word is "caw" continue ? (y/n) y the word is "ced" continue ? (y/n) y the word is "cet" continue ? (y/n) n The changes to ``enumwords`` are summarized in the following items. * The base case calls an interactive function. * The return value of ``enumwords`` is a boolean: to continue or not. * In the general case, if a call to ``enumwords`` returns ``False``, then we break out of the loop. The question in the interactive session is encapsulated in the function ``ask_to_continue()``. :: def ask_to_continue(word): """ Shows the string word and then asks the user to continue or not. """ qst = 'the word is \"' + word + '\"' qst = qst + ' continue ? (y/n) ' ans = input(qst) return ans == 'y' Then the changed function ``enumwords`` is below. :: def enumwords(k, letters, accu): """ Starting with the k-th list in letters, adds letters to the current string accu, returns a boolean: to continue or not. """ if k >= len(letters): return ask_to_continue(accu) else: for letter in letters[k]: cont = enumwords(k+1, letters, accu+letter) if not cont: break return cont Backtracking ------------ To introduce backtracking, consider the knapsack problem. A traveler selects objects to put in a knapsack. The problem statement is the following: .. math:: \begin{array}{rcl} input & : & \mbox{a list of values } V, \mbox{a lower bound } L, \mbox{and an upper bound } U \\ output & : & \mbox{all selections } S \subseteq V: \sum_{s \in S} \in [L, U] \end{array} In an extended version the input is a list of weights. The upper bound is for the sum of weights. The lower bound is for the sum of values. A restricted problem is the subset sum problem. Find all sums of numbers in a given list that match a given value. Running the script ``knapsack.py`` could go as follows. :: $ python3 knapsack.py give number of objects : 5 give a list of 5 values : [2.1, 1.8, 3.2, 4.1, 0.8] lower bound on sum : 3 upper bound on sum : 5 V([0, 1, 4]) = 2.10 + 1.80 + 0.80 = 4.70 V([0, 1]) = 2.10 + 1.80 = 3.90 V([1, 2]) = 1.80 + 3.20 = 5.00 V([2, 4]) = 3.20 + 0.80 = 4.00 V([2]) = 3.20 = 3.20 V([3, 4]) = 4.10 + 0.80 = 4.90 V([3]) = 4.10 = 4.10 In designing a recursive algorithm, we first define the parameters. * The recursive function ``knapsack`` takes on input the list of values ``V`` and the bounds on the sum. * The parameter ``k`` is the index of the current object being considered for inclusion in the knapsack. In the base case, we have ``k == len(V)``. If the ``sum(V)`` is within bounds, then ``V`` is printed. In the general case, for ``k < len(V)``, consider adding object ``k`` * only if its value ``V[k]`` plus the current sum does not exceed the upper bound. * Also call for ``k+1`` *without* adding it. Finally, we have two accumulating parameters: The current selection and the list of the values of the current selection. In the definition of the recursive function ``knapsack`` observe the specification in the documentation string. :: def knapsack(things, low, upp, k, sel, val): """ Shows all selections sel of things whose sum of values in val: low <= sum(val) <= upp. Input parameters: things : a list of values; low : lower bound on sum; upp : upper bound on sum; k : index to the current thing. Accumulating parameters: sel : stores selection of things; if selected i-th thing in things, then sel[i] == 1, else sel[i] == 0; val : values of the selected things. Initialize sel and val both to [], k to 0. """ if k >= len(things): if low <= sum(val) <= upp: write(sel, val) if k < len(things): if sum(val) + things[k] <= upp: sel.append(k) val.append(things[k]) knapsack(things, low, upp, k+1, sel, val) del sel[len(sel)-1] del val[len(val)-1] knapsack(things, low, upp, k+1, sel, val) The helper function ``write()`` called in the base case of ``knapsack`` is defined below. :: def write(sel, val): """ Does formatted printing in knapsack, for the selection sel and values val. """ result = 'V(' + str(sel) + ') = ' for i in range(0, len(val)): if i > 0: result = result + ' + ' result = result + '%.2f' % val[i] result = result + ' = ' + '%.2f' % sum(val) print(result) We end with the code for the function ``main()``. :: def main(): """ Prompts user for the number of objects asks for the list of their values, a lower and an upper bound on the sum. """ from ast import literal_eval nbr = int(input('give number of objects : ')) qst = 'give a list of %d values : ' % nbr vals = literal_eval(input(qst)) low = float(input('lower bound on sum : ')) upp = float(input('upper bound on sum : ')) knapsack(vals, low, upp, 0, [], []) if __name__ == "__main__": main() Exercises --------- 1. Use the enumeration of all bit combinations in a script to verify using a truth table whether two logical expressions are equivalent. 2. Write code to enumerate all sublists of a given list. 3. Modify the code for the recursive function ``knapsack`` with an interactive stopping condition. 4. Let the script for the knapsack problem also take on input the weights of the objects. Prompt the user for an additional list of weights and use the upper bound for the total weight of the selected objects. 5. In chess, a queen can attack all squares that run horizontally, vertically, and diagonally starting at the queen's position. On an 8-by-8 chess board, compute in how many ways you can put 8 queens so they do not attack each other.