Syntactic sugar, and computing every function

“[In 1951] I had a running compiler and nobody would touch it because, they carefully told me, computers could only do arithmetic; they could not do programs.”, Grace Murray Hopper, 1986.

“Syntactic sugar causes cancer of the semicolon.”, Alan Perlis, 1982.

The computational models we considered thus far are as “bare bones” as they come. For example, our NAND-CIRC “programming language” has only the single operation foo = NAND(bar,blah). In this chapter we will see that these simple models are actually equivalent to more sophisticated ones. The key observation is that we can implement more complex features using our basic building blocks, and then use these new features themselves as building blocks for even more sophisticated features. This is known as “syntactic sugar” in the field of programming language design since we are not modifying the underlying programming model itself, but rather we merely implement new features by syntactically transforming a program that uses such features into one that doesn’t.

This chapter provides a “toolkit” that can be used to show that many functions can be computed by NAND-CIRC programs, and hence also by Boolean circuits. We will also use this toolkit to prove a fundamental theorem: every finite function \(f:\{0,1\}^n \rightarrow \{0,1\}^m\) can be computed by a Boolean circuit, see Theorem 4.13 below. While the syntactic sugar toolkit is important in its own right, Theorem 4.13 can also be proven directly without using this toolkit. We present this alternative proof in Section 4.5. See Figure 4.1 for an outline of the results of this chapter.

Some examples of syntactic sugar

We now present some examples of “syntactic sugar” transformations that we can use in constructing straightline programs or circuits. We focus on the straight-line programming language view of our computational models, and specifically (for the sake of concreteness) on the NAND-CIRC programming language. This is convenient because many of the syntactic sugar transformations we present are easiest to think about in terms of applying “search and replace” operations to the source code of a program. However, by Theorem 3.19, all of our results hold equally well for circuits, whether ones using NAND gates or Boolean circuits that use the AND, OR, and NOT operations. Enumerating the examples of such syntactic sugar transformations can be a little tedious, but we do it for two reasons:

1. To convince you that despite their seeming simplicity and limitations, simple models such as Boolean circuits or the NAND-CIRC programming language are actually quite powerful.

2. So you can realize how lucky you are to be taking a theory of computation course and not a compilers course… :)

User-defined procedures

One staple of almost any programming language is the ability to define and then execute procedures or subroutines. (These are often known as functions in some programming languages, but we prefer the name procedures to avoid confusion with the function that a program computes.) The NAND-CIRC programming language does not have this mechanism built in. However, we can achieve the same effect using the time-honored technique of “copy and paste”. Specifically, we can replace code which defines a procedure such as

def Proc(a,b):
    proc_code
    return c
some_code
f = Proc(d,e)
some_more_code

with the following code where we “paste” the code of Proc

some_code
proc_code'
some_more_code

and where proc_code' is obtained by replacing all occurrences of a with d, b with e, and c with f. When doing that we will need to ensure that all other variables appearing in proc_code' don’t interfere with other variables. We can always do so by renaming variables to new names that were not used before. The above reasoning leads to the proof of the following theorem:

Theorem 4.1 can be proven using the transformation above, but since the formal proof is somewhat long and tedious, we omit it here.

def NOT(a):
    return NAND(a,a)
def AND(a,b):
    temp = NAND(a,b)
    return NOT(temp)
def OR(a,b):
    temp1 = NOT(a)
    temp2 = NOT(b)
    return NAND(temp1,temp2)

def MAJ(a,b,c):
    and1 = AND(a,b)
    and2 = AND(a,c)
    and3 = AND(b,c)
    or1 = OR(and1,and2)
    return OR(or1,and3)

print(MAJ(0,1,1))
# 1

Proof by Python (optional)

We can write a Python program that implements the proof of Theorem 4.1. This is a Python program that takes a NAND-CIRC-PROC program \(P\) that includes procedure definitions and uses simple “search and replace” to transform \(P\) into a standard (i.e., “sugar-free”) NAND-CIRC program \(P'\) that computes the same function as \(P\) without using any procedures. The idea is simple: if the program \(P\) contains a definition of a procedure Proc of two arguments x and y, then whenever we see a line of the form foo = Proc(bar,blah), we can replace this line by:

1. The body of the procedure Proc (replacing all occurrences of x and y with bar and blah respectively).

2. A line foo = exp, where exp is the expression following the return statement in the definition of the procedure Proc.

To make this more robust we add a prefix to the internal variables used by Proc to ensure they don’t conflict with the variables of \(P\); for simplicity we ignore this issue in the code below though it can be easily added.

The code of the Python function desugar below achieves such a transformation.

def desugar(code, func_name, func_args,func_body):
    """
    Replaces all occurences of 
       foo = func_name(func_args) 
    with
       func_body[x->a,y->b]
       foo = [result returned in func_body]    
    """
    # Uses Python regular expressions to simplify the search and replace,
    # see https://docs.python.org/3/library/re.html and Chapter 9 of the book

    # regular expression for capturing a list of variable names separated by commas
    arglist = ",".join([r"([a-zA-Z0-9\_\[\]]+)" for i in range(len(func_args))])
    # regular expression for capturing a statement of the form
    # "variable = func_name(arguments)"
    regexp = fr'([a-zA-Z0-9\_\[\]]+)\s*=\s*{func_name}\({arglist}\)\s*$'
    while True:
        m = re.search(regexp, code, re.MULTILINE)
        if not m: break
        newcode = func_body 
        # replace function arguments by the variables from the function invocation
        for i in range(len(func_args)): 
            newcode = newcode.replace(func_args[i], m.group(i+2))
        # Splice the new code inside
        newcode = newcode.replace('return', m.group(1) + " = ")
        code = code[:m.start()] + newcode + code[m.end()+1:]
    return code

Figure 4.2 shows the result of applying desugar to the program of Example 4.3 that uses syntactic sugar to compute the Majority function. Specifically, we first apply desugar to remove usage of the OR function, then apply it to remove usage of the AND function, and finally apply it a third time to remove usage of the NOT function.

def parse_func(code):
    """Parse a function definition into name, arguments and body"""
    lines = [l.strip() for l in code.split('\n')]
    regexp = r'def\s+([a-zA-Z\_0-9]+)\(([\sa-zA-Z0-9\_,]+)\)\s*:\s*'
    m = re.match(regexp,lines[0])
    return m.group(1), m.group(2).split(','), '\n'.join(lines[1:])

Conditional statements

Another sorely missing feature in NAND-CIRC is a conditional statement such as the if/then constructs that are found in many programming languages. However, using procedures, we can obtain an ersatz if/then construct. First we can compute the function \(\ensuremath{\mathit{IF}}:\{0,1\}^3 \rightarrow \{0,1\}\) such that \(\ensuremath{\mathit{IF}}(a,b,c)\) equals \(b\) if \(a=1\) and \(c\) if \(a=0\).

The \(\ensuremath{\mathit{IF}}\) function can be implemented from NANDs as follows (see Exercise 4.2):

def IF(cond,a,b):
    notcond = NAND(cond,cond)
    temp = NAND(b,notcond)
    temp1 = NAND(a,cond)
    return NAND(temp,temp1)

The \(\ensuremath{\mathit{IF}}\) function is also known as a multiplexing function, since \(cond\) can be thought of as a switch that controls whether the output is connected to \(a\) or \(b\). Once we have a procedure for computing the \(\ensuremath{\mathit{IF}}\) function, we can implement conditionals in NAND. The idea is that we replace code of the form

if (condition):  assign blah to variable foo

with code of the form

foo   = IF(condition, blah, foo)

that assigns to foo its old value when condition equals \(0\), and assign to foo the value of blah otherwise. More generally we can replace code of the form

if (cond):
    a = ...
    b = ...
    c = ...

with code of the form

temp_a = ...
temp_b = ...
temp_c = ...
a = IF(cond,temp_a,a)
b = IF(cond,temp_b,b)
c = IF(cond,temp_c,c)

Using such transformations, we can prove the following theorem. Once again we omit the (not too insightful) full formal proof, though see Section 4.1.2 for some hints on how to obtain it.

Extended example: Addition and Multiplication (optional)

Using “syntactic sugar”, we can write the integer addition function as follows:

# Add two n-bit integers
# Use LSB first notation for simplicity
def ADD(A,B):
    Result = [0]*(n+1)
    Carry  = [0]*(n+1)
    Carry[0] = zero(A[0])
    for i in range(n):
        Result[i] = XOR(Carry[i],XOR(A[i],B[i]))
        Carry[i+1] = MAJ(Carry[i],A[i],B[i])
    Result[n] = Carry[n]
    return Result

ADD([1,1,1,0,0],[1,0,0,0,0]);;
# [0, 0, 0, 1, 0, 0]

where zero is the constant zero function, and MAJ and XOR correspond to the majority and XOR functions respectively. While we use Python syntax for convenience, in this example \(n\) is some fixed integer and so for every such \(n\), ADD is a finite function that takes as input \(2n\) bits and outputs \(n+1\) bits. In particular for every \(n\) we can remove the loop construct for i in range(n) by simply repeating the code \(n\) times, replacing the value of i with \(0,1,2,\ldots,n-1\). By expanding out all the features, for every value of \(n\) we can translate the above program into a standard (“sugar-free”) NAND-CIRC program. Figure 4.4 depicts what we get for \(n=2\).

By going through the above program carefully and accounting for the number of gates, we can see that it yields a proof of the following theorem (see also Figure 4.5):

Once we have addition, we can use the grade-school algorithm to obtain multiplication as well, thus obtaining the following theorem:

We omit the proof, though in Exercise 4.7 we ask you to supply a “constructive proof” in the form of a program (in your favorite programming language) that on input a number \(n\), outputs the code of a NAND-CIRC program of at most \(1000n^2\) lines that computes the \(\ensuremath{\mathit{MULT}}_n\) function. In fact, we can use Karatsuba’s algorithm to show that there is a NAND-CIRC program of \(O(n^{\log_2 3})\) lines to compute \(\ensuremath{\mathit{MULT}}_n\) (and can get even further asymptotic improvements using better algorithms).

The LOOKUP function

The \(\ensuremath{\mathit{LOOKUP}}\) function will play an important role in this chapter and later. It is defined as follows:

See Figure 4.6 for an illustration of the LOOKUP function. It turns out that for every \(k\), we can compute \(\ensuremath{\mathit{LOOKUP}}_k\) using a NAND-CIRC program:

An immediate corollary of Theorem 4.10 is that for every \(k>0\), \(\ensuremath{\mathit{LOOKUP}}_k\) can be computed by a Boolean circuit (with AND, OR and NOT gates) of at most \(8 \cdot 2^k\) gates.

Constructing a NAND-CIRC program for \(\ensuremath{\mathit{LOOKUP}}\)

We prove Theorem 4.10 by induction. For the case \(k=1\), \(\ensuremath{\mathit{LOOKUP}}_1\) maps \((x_0,x_1,i) \in \{0,1\}^3\) to \(x_i\). In other words, if \(i=0\) then it outputs \(x_0\) and otherwise it outputs \(x_1\), which (up to reordering variables) is the same as the \(\ensuremath{\mathit{IF}}\) function presented in Section 4.1.3, which can be computed by a 4-line NAND-CIRC program.

As a warm-up for the case of general \(k\), let us consider the case of \(k=2\). Given input \(x=(x_0,x_1,x_2,x_3)\) for \(\ensuremath{\mathit{LOOKUP}}_2\) and an index \(i=(i_0,i_1)\), if the most significant bit \(i_0\) of the index is \(0\) then \(\ensuremath{\mathit{LOOKUP}}_2(x,i)\) will equal \(x_0\) if \(i_1=0\) and equal \(x_1\) if \(i_1=1\). Similarly, if the most significant bit \(i_0\) is \(1\) then \(\ensuremath{\mathit{LOOKUP}}_2(x,i)\) will equal \(x_2\) if \(i_1=0\) and will equal \(x_3\) if \(i_1=1\). Another way to say this is that we can write \(\ensuremath{\mathit{LOOKUP}}_2\) as follows:

def LOOKUP2(X[0],X[1],X[2],X[3],i[0],i[1]):
    if i[0]==1:
        return LOOKUP1(X[2],X[3],i[1])
    else:
        return LOOKUP1(X[0],X[1],i[1])

or in other words,

def LOOKUP2(X[0],X[1],X[2],X[3],i[0],i[1]):
    a = LOOKUP1(X[2],X[3],i[1])
    b = LOOKUP1(X[0],X[1],i[1])
    return IF( i[0],a,b)

More generally, as shown in the following lemma, we can compute \(\ensuremath{\mathit{LOOKUP}}_k\) using two invocations of \(\ensuremath{\mathit{LOOKUP}}_{k-1}\) and one invocation of \(\ensuremath{\mathit{IF}}\):

Proof of Theorem 4.10 from Lemma 4.11. Now that we have Lemma 4.11, we can complete the proof of Theorem 4.10. We will prove by induction on \(k\) that there is a NAND-CIRC program of at most \(4\cdot (2^k-1)\) lines for \(\ensuremath{\mathit{LOOKUP}}_k\). For \(k=1\) this follows by the four line program for \(\ensuremath{\mathit{IF}}\) we’ve seen before. For \(k>1\), we use the following pseudocode:

a = LOOKUP_(k-1)(X[0],...,X[2^(k-1)-1],i[1],...,i[k-1])
b = LOOKUP_(k-1)(X[2^(k-1)],...,X[2^(k-1)],i[1],...,i[k-1])
return IF(i[0],b,a)

If we let \(L(k)\) be the number of lines required for \(\ensuremath{\mathit{LOOKUP}}_k\), then the above pseudo-code shows that

Computing every function

At this point we know the following facts about NAND-CIRC programs (and so equivalently about Boolean circuits and our other equivalent models):

1. They can compute at least some non-trivial functions.

2. Coming up with NAND-CIRC programs for various functions is a very tedious task.

Thus I would not blame the reader if they were not particularly looking forward to a long sequence of examples of functions that can be computed by NAND-CIRC programs. However, it turns out we are not going to need this, as we can show in one fell swoop that NAND-CIRC programs can compute every finite function:

By Theorem 3.19, the models of NAND circuits, NAND-CIRC programs, AON-CIRC programs, and Boolean circuits, are all equivalent to one another, and hence Theorem 4.12 holds for all these models. In particular, the following theorem is equivalent to Theorem 4.12:

Improved bounds. Though it will not be of great importance to us, it is possible to improve on the proof of Theorem 4.12 and shave an extra factor of \(n\), as well as optimize the constant \(c\), and so prove that for every \(\epsilon>0\), \(m\in \N\) and sufficiently large \(n\), if \(f:\{0,1\}^n \rightarrow \{0,1\}^m\) then \(f\) can be computed by a NAND circuit of at most \((1+\epsilon)\tfrac{m\cdot 2^n}{n}\) gates. The proof of this result is beyond the scope of this book, but we do discuss how to obtain a bound of the form \(O(\tfrac{m \cdot 2^n}{n})\) in Section 4.4.2; see also the biographical notes.

Proof of NAND’s Universality

To prove Theorem 4.12, we need to give a NAND circuit, or equivalently a NAND-CIRC program, for every possible function. We will restrict our attention to the case of Boolean functions (i.e., \(m=1\)). Exercise 4.9 asks you to extend the proof for all values of \(m\). A function \(F: \{0,1\}^n\rightarrow \{0,1\}\) can be specified by a table of its values for each one of the \(2^n\) inputs. For example, the table below describes one particular function \(G: \{0,1\}^4 \rightarrow \{0,1\}\):²

For every \(x\in \{0,1\}^4\), \(G(x)=\ensuremath{\mathit{LOOKUP}}_4(1100100100001111,x)\), and so the following is NAND-CIRC “pseudocode” to compute \(G\) using syntactic sugar for the LOOKUP_4 procedure.

G0000 = 1
G1000 = 1
G0100 = 0
...
G0111 = 1
G1111 = 1
Y[0] = LOOKUP_4(G0000,G1000,...,G1111,
                X[0],X[1],X[2],X[3])

We can translate this pseudocode into an actual NAND-CIRC program by adding three lines to define variables zero and one that are initialized to \(0\) and \(1\) respectively, and then replacing a statement such as Gxxx = 0 with Gxxx = NAND(one,one) and a statement such as Gxxx = 1 with Gxxx = NAND(zero,zero). The call to LOOKUP_4 will be replaced by the NAND-CIRC program that computes \(\ensuremath{\mathit{LOOKUP}}_4\), plugging in the appropriate inputs.

There was nothing about the above reasoning that was particular to the function \(G\) above. Given every function \(F: \{0,1\}^n \rightarrow \{0,1\}\), we can write a NAND-CIRC program that does the following:

1. Initialize \(2^n\) variables of the form F00...0 till F11...1 so that for every \(z\in\{0,1\}^n\), the variable corresponding to \(z\) is assigned the value \(F(z)\).

2. Compute \(\ensuremath{\mathit{LOOKUP}}_n\) on the \(2^n\) variables initialized in the previous step, with the index variable being the input variables X[\(0\) ],…,X[\(n-1\) ]. That is, just like in the pseudocode for G above, we use Y[0] = LOOKUP(F00..00,...,F11..1,X[0],..,X[\(n-1\)])

The total number of lines in the resulting program is \(3+2^n\) lines for initializing the variables plus the \(4\cdot 2^n\) lines that we pay for computing \(\ensuremath{\mathit{LOOKUP}}_n\). This completes the proof of Theorem 4.12.

Improving by a factor of \(n\) (optional)

By being a little more careful, we can improve the bound of Theorem 4.12 and show that every function \(F:\{0,1\}^n \rightarrow \{0,1\}^m\) can be computed by a NAND-CIRC program of at most \(O(m 2^n/n)\) lines. In other words, we can prove the following improved version:

As before, it is enough to prove the case that \(m=1\). Hence we let \(f:\{0,1\}^n \rightarrow \{0,1\}\), and our goal is to prove that there exists a NAND-CIRC program of \(O(2^n/n)\) lines (or equivalently a Boolean circuit of \(O(2^n/n)\) gates) that computes \(f\).

We let \(k= \log(n-2\log n)\) (the reasoning behind this choice will become clear later on). We define the function \(g:\{0,1\}^k \rightarrow \{0,1\}^{2^{n-k}}\) as follows:

\[ g(a) = f(a0^{n-k})f(a0^{n-k-1}1) \cdots f(a1^{n-k}) \;. \]

In other words, if we use the usual binary representation to identify the numbers \(\{0,\ldots, 2^{n-k}-1 \}\) with the strings \(\{0,1\}^{n-k}\), then for every \(a\in \{0,1\}^k\) and \(b\in \{0,1\}^{n-k}\)

\[ g(a)_b = f(ab) \;. \;\;(4.2) \]

Equation 4.2 means that for every \(x\in \{0,1\}^n\), if we write \(x=ab\) with \(a\in \{0,1\}^k\) and \(b\in \{0,1\}^{n-k}\) then we can compute \(f(x)\) by first computing the string \(T=g(a)\) of length \(2^{n-k}\), and then computing \(\ensuremath{\mathit{LOOKUP}}_{n-k}(T\;,\; b)\) to retrieve the element of \(T\) at the position corresponding to \(b\) (see Figure 4.8). The cost to compute the \(\ensuremath{\mathit{LOOKUP}}_{n-k}\) is \(O(2^{n-k})\) lines/gates and the cost in NAND-CIRC lines (or Boolean gates) to compute \(f\) is at most

\[ cost(g) + O(2^{n-k}) \;, \;\;(4.3) \]

where \(cost(g)\) is the number of operations (i.e., lines of NAND-CIRC programs or gates in a circuit) needed to compute \(g\).

To complete the proof we need to give a bound on \(cost(g)\). Since \(g\) is a function mapping \(\{0,1\}^k\) to \(\{0,1\}^{2^{n-k}}\), we can also think of it as a collection of \(2^{n-k}\) functions \(g_0,\ldots, g_{2^{n-k}-1}: \{0,1\}^k \rightarrow \{0,1\}\), where \(g_i(x) = g(a)_i\) for every \(a\in \{0,1\}^k\) and \(i\in [2^{n-k}]\). (That is, \(g_i(a)\) is the \(i\)-th bit of \(g(a)\).) Naively, we could use Theorem 4.12 to compute each \(g_i\) in \(O(2^k)\) lines, but then the total cost is \(O(2^{n-k} \cdot 2^k) = O(2^n)\) which does not save us anything. However, the crucial observation is that there are only \(2^{2^k}\) distinct functions mapping \(\{0,1\}^k\) to \(\{0,1\}\). For example, if \(g_{17}\) is an identical function to \(g_{67}\) that means that if we already computed \(g_{17}(a)\) then we can compute \(g_{67}(a)\) using only a constant number of operations: simply copy the same value! In general, if you have a collection of \(N\) functions \(g_0,\ldots,g_{N-1}\) mapping \(\{0,1\}^k\) to \(\{0,1\}\), of which at most \(S\) are distinct then for every value \(a\in \{0,1\}^k\) we can compute the \(N\) values \(g_0(a),\ldots,g_{N-1}(a)\) using at most \(O(S\cdot 2^k + N)\) operations (see Figure 4.9).

In our case, because there are at most \(2^{2^k}\) distinct functions mapping \(\{0,1\}^k\) to \(\{0,1\}\), we can compute the function \(g\) (and hence by Equation 4.2 also \(f\)) using at most

\[O(2^{2^k} \cdot 2^k + 2^{n-k}) \;\;(4.4)\]

operations. Now all that is left is to plug into Equation 4.4 our choice of \(k = \log (n-2\log n)\). By definition, \(2^k = n-2\log n\), which means that Equation 4.4 can be bounded

\[ O\left(2^{n-2\log n} \cdot (n-2\log n) + 2^{n-\log(n-2\log n)}\right) \leq \]

\[ O\left(\tfrac{2^n}{n^2} \cdot n + \tfrac{2^n}{n-2\log n} \right) \leq O\left(\tfrac{2^n}{n} + \tfrac{2^n}{0.5n} \right) = O\left( \tfrac{2^n}{n} \right) \]

which is what we wanted to prove. (We used above the fact that \(n - 2\log n \geq 0.5 \log n\) for sufficiently large \(n\).)

Using the connection between NAND-CIRC programs and Boolean circuits, an immediate corollary of Theorem 4.15 is the following improvement to Theorem 4.13:

Computing every function: An alternative proof

Theorem 4.13 is a fundamental result in the theory (and practice!) of computation. In this section, we present an alternative proof of this basic fact that Boolean circuits can compute every finite function. This alternative proof gives a somewhat worse quantitative bound on the number of gates but it has the advantage of being simpler, working directly with circuits and avoiding the usage of all the syntactic sugar machinery. (However, that machinery is useful in its own right, and will find other applications later on.)

The class \(\ensuremath{\mathit{SIZE}}_{n,m}(s)\)

We have seen that every function \(f:\{0,1\}^n \rightarrow \{0,1\}^m\) can be computed by a circuit of size \(O(m\cdot 2^n)\), and some functions (such as addition and multiplication) can be computed by much smaller circuits. We define \(\ensuremath{\mathit{SIZE}}_{n,m}(s)\) to be the set of functions mapping \(n\) bits to \(m\) bits that can be computed by NAND circuits of at most \(s\) gates (or equivalently, by NAND-CIRC programs of at most \(s\) lines). Formally, the definition is as follows:

Figure 4.12 depicts the set \(\ensuremath{\mathit{SIZE}}_{n,1}(s)\). Note that \(\ensuremath{\mathit{SIZE}}_{n,m}(s)\) is a set of functions, not of programs! Asking if a program or a circuit is a member of \(\ensuremath{\mathit{SIZE}}_{n,m}(s)\) is a category error as in the sense of Figure 4.13. As we discussed in Section 3.7.2 (and Section 2.6.1), the distinction between programs and functions is absolutely crucial. You should always remember that while a program computes a function, it is not equal to a function. In particular, as we’ve seen, there can be more than one program to compute the same function.

While we defined \(\ensuremath{\mathit{SIZE}}_n(s)\) with respect to NAND gates, we would get essentially the same class if we defined it with respect to AND/OR/NOT gates:

The results we have seen in this chapter can be phrased as showing that \(\ensuremath{\mathit{ADD}}_n \in \ensuremath{\mathit{SIZE}}_{2n,n+1}(100 n)\) and \(\ensuremath{\mathit{MULT}}_n \in \ensuremath{\mathit{SIZE}}_{2n,2n}(10000 n^{\log_2 3})\). Theorem 4.12 shows that for some constant \(c\), \(\ensuremath{\mathit{SIZE}}_{n,m}(c m 2^n)\) is equal to the set of all functions from \(\{0,1\}^n\) to \(\{0,1\}^m\).

Y[0] = NAND(temp,temp)

Exercises

t = NAND(X[2],X[2])
u = NAND(X[0],t)
v = NAND(X[1],X[2])
Y[0] = NAND(u,v)

Temp[0] = NAND(X[0],X[0])
Temp[1] = NAND(X[1],X[1])
Temp[2] = NAND(Temp[0],Temp[1])
Temp[3] = NAND(X[2],X[2])
Temp[4] = NAND(X[3],X[3])
Temp[5] = NAND(Temp[3],Temp[4])
Temp[6] = NAND(Temp[2],Temp[2])
Temp[7] = NAND(Temp[5],Temp[5])
Y[0] = NAND(Temp[6],Temp[7])

In the following exercises you are asked to compare the power of pairs of programming languages. By “comparing the power” of two programming languages \(X\) and \(Y\) we mean determining the relation between the set of functions that are computable using programs in \(X\) and \(Y\) respectively. That is, to answer such a question you need to do both of the following:

1. Either prove that for every program \(P\) in \(X\) there is a program \(P'\) in \(Y\) that computes the same function as \(P\), or give an example for a function that is computable by an \(X\)-program but not computable by a \(Y\)-program.

and

2. Either prove that for every program \(P\) in \(Y\) there is a program \(P'\) in \(X\) that computes the same function as \(P\), or give an example for a function that is computable by a \(Y\)-program but not computable by an \(X\)-program.

When you give an example as above of a function that is computable in one programming language but not the other, you need to prove that the function you showed is (1) computable in the first programming language and (2) not computable in the second programming language.

Bibliographical notes

See Jukna’s and Wegener’s books (Jukna, 2012) (Wegener, 1987) for much more extensive discussion on circuits. Shannon showed that every Boolean function can be computed by a circuit of exponential size (Shannon, 1938) . The improved bound of \(c \cdot 2^n/n\) (with the optimal value of \(c\) for many bases) is due to Lupanov (Lupanov, 1958) . An exposition of this for the case of NAND (where \(c=1\)) is given in Chapter 4 of his book (Lupanov, 1984) . (Thanks to Sasha Golovnev for tracking down this reference!)

The concept of “syntactic sugar” is also known as “macros” or “meta-programming” and is sometimes implemented via a preprocessor or macro language in a programming language or a text editor. One modern example is the Babel JavaScript syntax transformer, that converts JavaScript programs written using the latest features into a format that older Browsers can accept. It even has a plug-in architecture, that allows users to add their own syntactic sugar to the language.

Comments

Comments are posted on the GitHub repository using the utteranc.es app. A GitHub login is required to comment. If you don't want to authorize the app to post on your behalf, you can also comment directly on the GitHub issue for this page.

Compiled on 12/06/2023 00:06:56

Copyright 2023, Boaz Barak.
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Produced using pandoc and panflute with templates derived from gitbook and bookdown.