Polynomial-time reductions

Consider some of the problems we have encountered in Chapter 12:

1. The 3SAT problem: deciding whether a given 3CNF formula has a satisfying assignment.

2. Finding the longest path in a graph.

3. Finding the maximum cut in a graph.

4. Solving quadratic equations over \(n\) variables \(x_0,\ldots,x_{n-1} \in \R\).

All of these problems have the following properties:

• These are important problems, and people have spent significant effort on trying to find better algorithms for them.

• Each one of these is a search problem, whereby we search for a solution that is “good” in some easy to define sense (e.g., a long path, a satisfying assignment, etc.).

• Each of these problems has a trivial exponential time algorithm that involve enumerating all possible solutions.

• At the moment, for all these problems the best known algorithm is not much faster than the trivial one in the worst case.

In this chapter and in Chapter 15 we will see that, despite their apparent differences, we can relate the computational complexity of these and many other problems. In fact, it turns out that the problems above are computationally equivalent, in the sense that solving one of them immediately implies solving the others. This phenomenon, known as \(\mathbf{NP}\) completeness, is one of the surprising discoveries of theoretical computer science, and we will see that it has far-reaching ramifications.

In this chapter we will see that for each one of the problems of finding a longest path in a graph, solving quadratic equations, and finding the maximum cut, if there exists a polynomial-time algorithm for this problem then there exists a polynomial-time algorithm for the 3SAT problem as well. In other words, we will reduce the task of solving 3SAT to each one of the above tasks. Another way to interpret these results is that if there does not exist a polynomial-time algorithm for 3SAT then there does not exist a polynomial-time algorithm for these other problems as well. In Chapter 15 we will see evidence (though no proof!) that all of the above problems do not have polynomial-time algorithms and hence are inherently intractable.

Formal definitions of problems

For reasons of technical convenience rather than anything substantial, we concern ourselves with decision problems (i.e., Yes/No questions) or in other words Boolean (i.e., one-bit output) functions. We model the problems above as functions mapping \(\{0,1\}^*\) to \(\{0,1\}\) in the following way:

3SAT. The 3SAT problem can be phrased as the function \(3\ensuremath{\mathit{SAT}}:\{0,1\}^* \rightarrow \{0,1\}\) that takes as input a 3CNF formula \(\varphi\) (i.e., a formula of the form \(C_0 \wedge \cdots \wedge C_{m-1}\) where each \(C_i\) is the OR of three variables or their negation) and maps \(\varphi\) to \(1\) if there exists some assignment to the variables of \(\varphi\) that causes it to evalute to true, and to \(0\) otherwise. For example

since the assignment \(x = 1101\) satisfies the input formula. In the above we assume some representation of formulas as strings, and define the function to output \(0\) if its input is not a valid representation; we use the same convention for all the other functions below.

Quadratic equations. The quadratic equations problem corresponds to the function \(\ensuremath{\mathit{QUADEQ}}:\{0,1\}^* \rightarrow \{0,1\}\) that maps a set of quadratic equations \(E\) to \(1\) if there is an assignment \(x\) that satisfies all equations, and to \(0\) otherwise.

Longest path. The longest path problem corresponds to the function \(\ensuremath{\mathit{LONGPATH}}:\{0,1\}^* \rightarrow \{0,1\}\) that maps a graph \(G\) and a number \(k\) to \(1\) if there is a simple path in \(G\) of length at least \(k\), and maps \((G,k)\) to \(0\) otherwise. The longest path problem is a generalization of the well-known Hamiltonian Path Problem of determining whether a path of length \(n\) exists in a given \(n\) vertex graph.

Maximum cut. The maximum cut problem corresponds to the function \(\ensuremath{\mathit{MAXCUT}}:\{0,1\}^* \rightarrow \{0,1\}\) that maps a graph \(G\) and a number \(k\) to \(1\) if there is a cut in \(G\) that cuts at least \(k\) edges, and maps \((G,k)\) to \(0\) otherwise.

All of the problems above are in \(\mathbf{EXP}\) but it is not known whether or not they are in \(\mathbf{P}\). However, we will see in this chapter that if either \(\ensuremath{\mathit{QUADEQ}}\) , \(\ensuremath{\mathit{LONGPATH}}\) or \(\ensuremath{\mathit{MAXCUT}}\) are in \(\mathbf{P}\), then so is \(3\ensuremath{\mathit{SAT}}\).

Polynomial-time reductions

Suppose that \(F,G:\{0,1\}^* \rightarrow \{0,1\}\) are two Boolean functions. A polynomial-time reduction (or sometimes just “reduction” for short) from \(F\) to \(G\) is a way to show that \(F\) is “no harder” than \(G\), in the sense that a polynomial-time algorithm for \(G\) implies a polynomial-time algorithm for \(F\).

The following exercise justifies our intuition that \(F \leq_p G\) signifies that “\(F\) is no harder than \(G\)”.

Whistling pigs and flying horses

A reduction from \(F\) to \(G\) can be used for two purposes:

• If we already know an algorithm for \(G\) and \(F \leq_p G\) then we can use the reduction to obtain an algorithm for \(F\). This is a widely used tool in algorithm design. For example in Section 12.1.4 we saw how the Min-Cut Max-Flow theorem allows to reduce the task of computing a minimum cut in a graph to the task of computing a maximum flow in it.

• If we have proven (or have evidence) that there exists no polynomial-time algorithm for \(F\) and \(F \leq_p G\) then the existence of this reduction allows us to conclude that there exists no polynomial-time algorithm for \(G\). This is the “if pigs could whistle then horses could fly” interpretation we’ve seen in Section 9.4. We show that if there was an hypothetical efficient algorithm for \(G\) (a “whistling pig”) then since \(F \leq_p G\) then there would be an efficient algorithm for \(F\) (a “flying horse”). In this book we often use reductions for this second purpose, although the lines between the two is sometimes blurry (see the bibliographical notes in Section 14.10).

The most crucial difference between the notion in Definition 14.1 and the reductions we saw in the context of uncomputability (e.g., in Section 9.4) is that for relating time complexity of problems, we need the reduction to be computable in polynomial time, as opposed to merely computable. Definition 14.1 also restricts reductions to have a very specific format. That is, to show that \(F \leq_p G\), rather than allowing a general algorithm for \(F\) that uses a “magic box” that computes \(G\), we only allow an algorithm that computes \(F(x)\) by outputting \(G(R(x))\). This restricted form is convenient for us, but people have defined and used more general reductions as well (see Section 14.10).

In this chapter we use reductions to relate the computational complexity of the problems mentioned above: 3SAT, Quadratic Equations, Maximum Cut, and Longest Path, as well as a few others. We will reduce 3SAT to the latter problems, demonstrating that solving any one of them efficiently will result in an efficient algorithm for 3SAT. In Chapter 15 we show the other direction: reducing each one of these problems to 3SAT in one fell swoop.

Transitivity of reductions. Since we think of \(F \leq_p G\) as saying that (as far as polynomial-time computation is concerned) \(F\) is “easier or equal in difficulty to” \(G\), we would expect that if \(F \leq_p G\) and \(G \leq_p H\), then it would hold that \(F \leq_p H\). Indeed this is the case:

Reducing 3SAT to zero one and quadratic equations

We now show our first example of a reduction. The Zero-One Linear Equations problem corresponds to the function \(01\ensuremath{\mathit{EQ}}:\{0,1\}^* \rightarrow \{0,1\}\) whose input is a collection \(E\) of linear equations in variables \(x_0,\ldots,x_{n-1}\), and the output is \(1\) iff there is an assignment \(x\in \{0,1\}^n\) of \(0/1\) values to the variables that satisfies all the equations. For example, if the input \(E\) is a string encoding the set of equations

then \(01\ensuremath{\mathit{EQ}}(E)=1\) since the assignment \(x = 011\) satisfies all three equations. We specifically restrict attention to linear equations in variables \(x_0,\ldots, x_{n-1}\) in which every equation has the form \(\sum_{i \in S} x_i = b\) where \(S \subseteq [n]\) and \(b\in \N\).¹

If we asked the question of whether there is a solution \(x \in \R^n\) of real numbers to \(E\), then this can be solved using the famous Gaussian elimination algorithm in polynomial time. However, there is no known efficient algorithm to solve \(01\ensuremath{\mathit{EQ}}\). Indeed, such an algorithm would imply an algorithm for \(3\ensuremath{\mathit{SAT}}\) as shown by the following theorem:

Quadratic equations

Now that we reduced \(3\ensuremath{\mathit{SAT}}\) to \(01\ensuremath{\mathit{EQ}}\), we can use this to reduce \(3\ensuremath{\mathit{SAT}}\) to the quadratic equations problem. This is the function \(\ensuremath{\mathit{QUADEQ}}\) in which the input is a list of \(n\)-variate polynomials \(p_0,\ldots,p_{m-1}:\R^n \rightarrow \R\) that are all of degree at most two (i.e., they are quadratic) and with integer coefficients. (The latter condition is for convenience and can be achieved by scaling.) We define \(\ensuremath{\mathit{QUADEQ}}(p_0,\ldots,p_{m-1})\) to equal \(1\) if and only if there is a solution \(x\in \R^n\) to the equations \(p_0(x)=0\), \(p_1(x)=0\), \(\ldots\), \(p_{m-1}(x)=0\).

For example, the following is a set of quadratic equations over the variables \(x_0,x_1,x_2\):

The subset sum problem

As another consequence of the reduction of \(3\ensuremath{\mathit{SAT}}\) to \(01\ensuremath{\mathit{EQ}}\), we can also show that \(3\ensuremath{\mathit{SAT}}\) (through \(01\ensuremath{\mathit{EQ}}\)) reduces to the subset sum problem (also known as the knapsack problem). In the subset sum problem, we are given a list of integers \(x_0,\ldots,x_{n-1} \in \mathbb{Z}\) and an integer \(T\in\mathbb{Z}\). We need to determine whether or not there exists some set of the integers that sums up to \(T\). That is, for \(x_0,\ldots,x_{n-1},T \in \mathbb{Z}\), \(\ensuremath{\mathit{SSUM}}(x_0,\ldots,x_{n-1},T)=1\) if and only if there exists \(S\subseteq [n]\) such that \(\sum_{i\in S} x_i = T\). Note that the input length for the subset sum problem is the length of string needed to encode all the numbers, which will be approximately \(\lceil \log T \rceil + \sum_{i=0}^n \lceil \log x_i \rceil\), since encoding an integer \(x\) using the binary representation requires \(\lceil \log x \rceil\) bits.

The independent set problem

For a graph \(G=(V,E)\), an independent set (also known as a stable set) is a subset \(S \subseteq V\) such that there are no edges with both endpoints in \(S\) (in other words, \(E(S,S)=\emptyset\)). Every “singleton” (set consisting of a single vertex) is trivially an independent set, but finding larger independent sets can be challenging. The maximum independent set problem (henceforth simply “independent set”) is the task of finding the largest independent set in the graph. The independent set problem is naturally related to scheduling problems: if we put an edge between two conflicting tasks, then an independent set corresponds to a set of tasks that can all be scheduled together without conflicts. The independent set problem has been studied in a variety of settings, including for example in the case of algorithms for finding structure in protein-protein interaction graphs.

As mentioned in Section 14.1, we think of the independent set problem as the function \(\ensuremath{\mathit{ISET}}:\{0,1\}^* \rightarrow \{0,1\}\) that on input a graph \(G\) and a number \(k\) outputs \(1\) if and only if the graph \(G\) contains an independent set of size at least \(k\). We now reduce 3SAT to Independent set.

Some exercises and anatomy of a reduction.

Reductions can be confusing and working out exercises is a great way to gain more comfort with them. Here is one such example. As usual, I recommend you try it out yourself before looking at the solution.

Dominating set

In the two examples above, the reduction was almost “trivial”: the reduction from independent set to vertex cover merely changes the number \(k\) to \(n-k\), and the reduction from independent set to clique flips edges to non-edges and vice versa. The following exercise requires a somewhat more interesting reduction.

A corollary of Algorithm 14.10 and the other reduction we have seen so far is that if \(\ensuremath{\mathit{DS}} \in \mathbf{P}\) (i.e., dominating set has a polynomial-time algorithm) then \(3\ensuremath{\mathit{SAT}} \in \mathbf{P}\) (i.e., \(3\ensuremath{\mathit{SAT}}\) has a polynomial-time algorithm). By the contra-positive, if \(3\ensuremath{\mathit{SAT}}\) does not have a polynomial-time algorithm then neither does dominating set.

Anatomy of a reduction

The reduction of Solved Exercise 14.5 gives a good illustration of the anatomy of a reduction. A reduction consists of four parts:

• Algorithm description: This is the description of how the algorithm maps an input into the output. For example, in Solved Exercise 14.5 this is the description of how we map an instance \((G,k)\) of the vertex cover problem into an instance \((H,k')\) of the dominating set problem.

• Algorithm analysis: It is not enough to describe how the algorithm works but we need to also explain why it works. In particular we need to provide an analysis explaining why the reduction is both efficient (i.e., runs in polynomial time) and correct (satisfies that \(G(R(x))=F(x)\) for every \(x\)). Specifically, the components of analysis of a reduction \(R\) include:

  • Efficiency: We need to show that \(R\) runs in polynomial time. In most reductions we encounter this part is straightforward, as the reductions we typically use involve a constant number of nested loops, each involving a constant number of operations. For example, the reduction of Solved Exercise 14.5 just enumerates over the edges and vertices of the input graph.

  • Completeness: In a reduction \(R\) demonstrating \(F \leq_p G\), the completeness condition is the condition that for every \(x\in \{0,1\}^*\), if \(F(x) = 1\) then \(G(R(x))=1\). Typically we construct the reduction to ensure that this holds, by giving a way to map a “certificate/solution” certifying that \(F(x)=1\) into a solution certifying that \(G(R(x))=1\). For example, in Solved Exercise 14.5 we constructed the graph \(H\) such that for every vertex cover \(S\) in \(G\), the set \(S \cup I\) (where \(I\) is the isolated vertices) would be a dominating set in \(H\).

  • Soundness: This is the condition that if \(F(x)=0\) then \(G(R(x))=0\) or (taking the contrapositive) if \(G(R(x))=1\) then \(F(x)=1\). This is sometimes straightforward but can often be harder to show than the completeness condition, and in more advanced reductions (such as the reduction \(3\ensuremath{\mathit{SAT}} \leq_p \ensuremath{\mathit{ISET}}\) of Theorem 14.8) demonstrating soundness is the main part of the analysis. For example, in Solved Exercise 14.5 to show soundness we needed to show that for every dominating set \(D\) in the graph \(H\), there exists a vertex cover \(S\) of size at most \(|D|-\ell\) in the graph \(G\) (where \(\ell\) is the number of isolated vertices). This was challenging since the dominating set \(D\) might not be necessarily the one we “had in mind”. In particular, in the proof above we needed to modify \(D\) to ensure that it does not contain vertices of the form \(w_{u,v}\), and it was important to show that this modification still maintains the property that \(D\) is a dominating set, and also does not make it bigger.

Whenever you need to provide a reduction, you should make sure that your description has all these components. While it is sometimes tempting to weave together the description of the reduction and its analysis, it is usually clearer if you separate the two, and also break down the analysis to its three components of efficiency, completeness, and soundness.

Reducing Independent Set to Maximum Cut

We now show that the independent set problem reduces to the maximum cut (or “max cut”) problem, modeled as the function \(\ensuremath{\mathit{MAXCUT}}\) that on input a pair \((G,k)\) outputs \(1\) iff \(G\) contains a cut of at least \(k\) edges. Since both are graph problems, a reduction from independent set to max cut maps one graph into the other, but as we will see the output graph does not have to have the same vertices or edges as the input graph.

Reducing 3SAT to Longest Path

Note: This section is still a little messy; feel free to skip it or just read it without going into the proof details. The proof appears in Section 7.5 in Sipser’s book.

One of the most basic algorithms in Computer Science is Dijkstra’s algorithm to find the shortest path between two vertices. We now show that in contrast, an efficient algorithm for the longest path problem would imply a polynomial-time algorithm for 3SAT.

def TSAT2LONGPATH(φ):
    """Reduce 3SAT to LONGPATH"""
    def var(v): # return variable and True/False depending if positive or negated
        return int(v[2:]),False if v[0]=="¬" else int(v[1:]),True
    n = numvars(φ)
    clauses = getclauses(φ)
    m = len(clauses)
    G =Graph() 
    G.edge("start","start_0")
    for i in range(n): # add 2 length-m paths per variable
        G.edge(f"start_{i}",f"v_{i}_{0}_T")
        G.edge(f"start_{i}",f"v_{i}_{0}_F")
        for j in range(m-1): 
            G.edge(f"v_{i}_{j}_T",f"v_{i}_{j+1}_T")
            G.edge(f"v_{i}_{j}_F",f"v_{i}_{j+1}_F")
        G.edge(f"v_{i}_{m-1}_T",f"end_{i}")
        G.edge(f"v_{i}_{m-1}_F",f"end_{i}")
        if i<n-1:
            G.edge(f"end_{i}",f"start_{i+1}")
    G.edge(f"end_{n-1}","start_clauses")
    for j,C in enumerate(clauses): # add gadget for each clause
        for v in enumerate(C):
            i,sign = var(v[1])
            s = "F" if sign else "T"
            G.edge(f"C_{j}_in",f"v_{i}_{j}_{s}")
            G.edge(f"v_{i}_{j}_{s}",f"C_{j}_out")
        if j<m-1:
            G.edge(f"C_{j}_out",f"C_{j+1}_in")
    G.edge("start_clauses","C_0_in")
    G.edge(f"C_{m-1}_out","end")
    return G, 1+n*(m+1)+1+2*m+1

Summary of relations

We have shown that there are a number of functions \(F\) for which we can prove a statement of the form “If \(F\in \mathbf{P}\) then \(3\ensuremath{\mathit{SAT}} \in \mathbf{P}\)”. Hence coming up with a polynomial-time algorithm for even one of these problems will entail a polynomial-time algorithm for \(3\ensuremath{\mathit{SAT}}\) (see for example Figure 14.16). In Chapter 15 we will show the inverse direction (“If \(3\ensuremath{\mathit{SAT}} \in \mathbf{P}\) then \(F\in \mathbf{P}\)”) for these functions, hence allowing us to conclude that they have equivalent complexity to \(3\ensuremath{\mathit{SAT}}\).

Exercises

Bibliographical notes

Several notions of reductions are defined in the literature. The notion defined in Definition 14.1 is often known as a mapping reduction, many to one reduction or a Karp reduction.

The maximal (as opposed to maximum) independent set is the task of finding a “local maximum” of an independent set: an independent set \(S\) such that one cannot add a vertex to it without losing the independence property (such a set is known as a vertex cover). Unlike finding a maximum independent set, finding a maximal independent set can be done efficiently by a greedy algorithm, but this local maximum can be much smaller than the global maximum.

Reduction of independent set to max cut taken from these notes. Image of Hamiltonian Path through Dodecahedron by Christoph Sommer.

We have mentioned that the line between reductions used for algorithm design and showing hardness is sometimes blurry. An excellent example for this is the area of SAT Solvers (see (Gomes, Kautz, Sabharwal, Selman, 2008) ). In this field people use algorithms for SAT (that take exponential time in the worst case but often are much faster on many instances in practice) together with reductions of the form \(F \leq_p \ensuremath{\mathit{SAT}}\) to derive algorithms for other functions \(F\) of interest.

Comments

Comments are posted on the GitHub repository using the utteranc.es app. A GitHub login is required to comment. If you don't want to authorize the app to post on your behalf, you can also comment directly on the GitHub issue for this page.

Compiled on 12/06/2023 00:07:22

Copyright 2023, Boaz Barak.
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Produced using pandoc and panflute with templates derived from gitbook and bookdown.