Unbounded nondeterminism

Discussion of unbounded nondeterminism tends to get involved with discussions of fairness. The basic concept is that all computation paths must be "fair" in the sense that if the machine enters a state infinitely often, it must take every possible transition from that state. This amounts to requiring that the machine be guaranteed to service a request if it can, since an infinite sequence of states will only be allowed if there is no transition that leads to the request being serviced. Equivalently, every possible transition must occur eventually in an infinite computation, although it may take an unbounded amount of time for the transition to occur. This concept is to be distinguished from the local fairness of flipping a "fair" coin, by which it is understood that it is possible for the outcome to always be heads for any finite number of steps, although as the number of steps increases, this will almost surely not happen.

An example of the role of fair or unbounded nondeterminism in the merging of strings was given by William D. Clinger, in his 1981 thesis. He defined a "fair merge" of two strings to be a third string in which each character of each string must occur eventually. He then considered the set of all fair merges of two strings merge(S, T), assuming it to be a monotone function. Then he argued that merge(⊥,1^ω)⊆ merge(0,1^ω), where ⊥ is the empty stream. Now merge(⊥,1^ω) = {1^ω}, so it must be that 1^ω is an element of merge(0,1^ω), a contradiction. He concluded that:

It appears that a fair merge cannot be written as a nondeterministic data flow program operating on streams.[2]

Edsger Dijkstra argued that it is impossible to implement systems with unbounded nondeterminism[3]. For this reason, Tony Hoare suggested that "an efficient implementation should try to be reasonably fair."[4]

Nondeterministic Turing machines have only bounded nondeterminism. Likewise sequential programs containing guarded commands as the only sources of nondeterminism have only bounded nondeterminism.[3] Briefly, choice nondeterminism is bounded. Gordon Plotkin gave a proof in his original paper on powerdomains:

Now the set of initial segments of execution sequences of a given nondeterministic program P, starting from a given state, will form a tree. The branching points will correspond to the choice points in the program. Since there are always only finitely many alternatives at each choice point, the branching factor of the tree is always finite. That is, the tree is finitary. Now Kőnig's lemma says that if every branch of a finitary tree is finite, then so is the tree itself. In the present case this means that if every execution sequence of P terminates, then there are only finitely many execution sequences. So if an output set of P is infinite, it must contain [a nonterminating computation].[5]

William Clinger provided the following analysis of the above proof by Plotkin:

This proof depends upon the premise that if every node x of a certain infinite branch can be reached by some computation c, then there exists a computation c that visits every node x on the branch. ... Clearly this premise follows not from logic but rather from the interpretation given to choice points. This premise fails for arrival nondeterminism [in the arrival of messages in the Actor model] because of finite delay [in the arrival of messages]. Though each node on an infinite branch must lie on a branch with a limit, the infinite branch need not itself have a limit. Thus the existence of an infinite branch does not necessarily imply a nonterminating computation.[2]

Spaan et al. have argued that it is possible for an unboundedly nondeterministic program to solve the halting problem; their algorithm consists of two parts defined as follows:[6]

The first part of the program requests a natural number from the second part; after receiving it, it will iterate the desired Turing machine for that many steps, and accept or reject according to whether the machine has yet halted.

The second part of the program nondeterministically chooses a natural number on request. The number is stored in a variable which is initialized to 0; then the program repeatedly chooses whether to increment the variable, or service the request. The fairness constraint requires that the request eventually be serviced, for otherwise there is an infinite loop in which only the "increment the variable" branch is ever taken.

Clearly, if the machine does halt, this algorithm has a path which accepts. If the machine does not halt, this algorithm will always reject, no matter what number the second part of the program returns.

Clinger and Carl Hewitt have developed a model (known as the Actor model) of concurrent computation with the property of unbounded nondeterminism built in [Clinger 1981; [7]; [8]; [9]]; this allows computations that cannot be implemented by Turing Machines, as seen above. However, these researchers emphasize that their model of concurrent computations cannot implement any functions that are outside the class of recursive functions defined by Church, Kleene, Turing, etc. (See Indeterminacy in concurrent computation.)

Hewitt justified his use of unbounded nondeterminism by arguing that there is no bound that can be placed on how long it takes a computational circuit called an arbiter to settle (see metastability in electronics). Arbiters are used in computers to deal with the circumstance that computer clocks operate asynchronously with input from outside, e.g.., keyboard input, disk access, network input, etc. So it could take an unbounded time for a message sent to a computer to be received and in the meantime the computer could traverse an unbounded number of states.

He further argued that Electronic mail enables unbounded nondeterminism since mail can be stored on servers indefinitely before being delivered, and that data links to servers on the Internet can likewise be out of service indefinitely. This gave rise to the unbounded nondeterminism controversy.[10]

Hewitt argued that issues in fairness derive in part from the global state point of view. The oldest models of computation (e.g.. Turing machines, Post productions, the lambda calculus, etc.) are based on mathematics that makes use of a global state to represent a computational step. Each computational step is from one global state of the computation to the next global state. The global state approach was continued in automata theory for finite state machines and push down stack machines including their nondeterministic versions. All of these models have the property of bounded nondeterminism: if a machine always halts when started in its initial state, then there is a bound on the number of states in which it can halt.

Hewitt argued that there is a fundamental difference between choices in global state nondeterminism and the arrival order indeterminacy (nondeterminism) of his Actor model. In global state nondeterminism, a "choice" is made for the "next" global state. In arrival order indeterminacy, arbitration locally decides each arrival order in an unbounded amount of time. While a local arbitration is proceeding, unbounded activity can take place elsewhere. There is no global state and consequently no "choice" to be made as to the "next" global state.

• Hewitt, Carl; Bishop, Peter; Steiger, Richard (August 1973). "A universal modular ACTOR formalism for artificial intelligence". Proceedings of the 3rd international joint conference on Artificial intelligence. IJCAI'73. Stanford: Morgan Kaufmann. pp. 235–245.
• Milner, Robin (1973). "Processes: a mathematical model of computing agents". Proceedings of the Logic Colloquium. Logic Colloquium '73. Bristol: North Holland. pp. 157–173.
• Hewitt, Carl; Bishop, Peter; Greif, Irene; Smith, Brian; Matson, Todd; Steiger, Richard (October 1973). "Actor Induction and Meta-evaluation". Proceedings of the 1st annual ACM SIGACT-SIGPLAN symposium on Principles of programming languages. POPL'73. Boston, Massachusetts: Association for Computing Machinery. pp. 153–168. doi:10.1145/512927.512942.
• Hewitt, Carl; Bishop, Peter; Steiger, Richard; Greif, Irene; Smith, Brian; Matson, Todd; Hale, Roger (April 1974). "Behavioral semantics of nonrecursive control structures". In Robinet, B. (ed.). Proceedings of Colloque sur la programmation. Programming Symposium. Paris: Springer Berlin Heidelberg. pp. 385–407. doi:10.1007/3-540-06859-7_147. ISBN 9783540378198.
• Greif, Irene (August 1975). Semantics of communicating parallel processes (PhD thesis). Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science. hdl:1721.1/57710.
• Hewitt, Carl; Baker, Henry (August 1977). "Actors and Continuous Functionals". In Neuhold, Erich J. (ed.). Proceedings of the IFIP Working Conference on Formal Description of Programming Concepts. IFIP'78. St. Andrews, N.B., Canada: North-Holland. ISBN 9780444851079.
• Kahn, Gilles; MacQueen, David (1976). Coroutines and Networks of Parallel Processes (Research Report). Retrieved March 9, 2022.
• Baker, Henry (January 1978). Actor Systems for Real-Time Computation (PhD thesis). Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science.
• Smyth, Michael (1978). "Power domains". Journal of Computer and System Sciences. 16: 23–36. doi:10.1016/0022-0000(78)90048-X.
• Milne, George; Milner, Robin (April 1979). "Concurrent processes and their syntax". Journal of the ACM. 6 (2): 302–321. doi:10.1145/322123.322134. S2CID 16565064.
• Francez, Nissim; Hoare, C. A. R.; Lehmann, Daniel J.; de Roever, Willem P. (December 1979). "Semantics of nondeterminism, concurrency, and communication". Journal of Computer and System Sciences. 19 (3): 290–308. doi:10.1016/0022-0000(79)90006-0.
• Lynch, Nancy A.; Fischer, Michael J. (July 1979). "On describing the behavior and implementation of distributed systems". In Kahn, Gilles (ed.). Proceedings of the International Symposium on Semantics of Concurrent Computation. Semantics of Concurrent Computation. Evian, France: Springer-Verlag. pp. 147–171. doi:10.1007/BFb0022468. ISBN 9783540351634.
• Schwartz, Jerald S. (July 1979). "Denotational semantics of parallelism". In Kahn, Gilles (ed.). Proceedings of the International Symposium on Semantics of Concurrent Computation. Semantics of Concurrent Computation. Evian, France: Springer-Verlag. pp. 191–202. doi:10.1007/BFb0022470. ISBN 9783540351634.
• Wadge, William W. (July 1979). "An extensional treatment of dataflow deadlock". In Kahn, Gilles (ed.). Proceedings of the International Symposium on Semantics of Concurrent Computation. Semantics of Concurrent Computation. Evian, France: Springer-Verlag. pp. 285–299. doi:10.1007/BFb0022475. ISBN 9783540351634.
• Back, Ralph-Johan (July 1980). "Semantics of unbounded nondeterminism". In de Bakker, Jaco; van Leeuwen, Jan (eds.). Seventh Colloquium on Automata, Languages and Programming. International Colloquium on Automata, Languages, and Programming. Noordwijkerhout, the Netherlands: Springer-Verlag Berlin Heidelberg. pp. 51–63. doi:10.1007/3-540-10003-2_59.
• Park, David (1979). "On the semantics of fair parallelism". In Bjørner, Dines (ed.). Abstract Software Specifications. 1979 Copenhagen Winter School. Copenhagen: Springer-Verlag Berlin Heidelberg. pp. 504–526. doi:10.1007/3-540-10007-5_47.

• Dana Scott. What is Denotational Semantics? MIT Laboratory for Computer Science Distinguished Lecture Series. April 17, 1980.

• Clinger, William (August 1982). "Nondeterministic call by need is neither lazy nor by name". In Park, David M. R.; Friedman, Daniel P. (eds.). Proceedings of the 1982 ACM symposium on LISP and functional programming. LFP'82. Pittsburgh, Pennsylvania: Association for Computing Machinery. pp. 226–234. doi:10.1145/800068.802154.
• Brookes, S. D.; Hoare, C. A. R.; Roscoe, A. W. (July 1984). "A Theory of Communicating Sequential Processes". Journal of the ACM. 31 (3): 560–599. doi:10.1145/828.833. S2CID 488666.
• Roscoe, A. W. (January 1988). "Unbounded nondeterminism in CSP". Two Papers on CSP (PDF) (Technical monograph). Oxford University Computing Laboratory. PRG67. Retrieved March 10, 2022.
• Roscoe, A. W. (November 10, 1997). The Theory and Practice of Concurrency. Prentice-Hall. ISBN 9780136744092.
• Schmidt, David A. (March 1994). The Structure of Typed Programming Languages. The MIT Press. ISBN 9780262193498.
• Butler, Michael; Morgan, Carroll (January 1995). "Action Systems, Unbounded Nondeterminism, and Infinite Traces". Formal Aspects of Computing. 7 (1): 37–53. doi:10.1007/BF01214622. S2CID 2135743.
• Sudkamp, Thomas A. (January 3, 1997). Languages and Machines: An Introduction to the Theory of Computer Science (2nd ed.). Addison-Wesley. ISBN 9780201821369.
• Aceto, Luca; Gordon, Andrew D., eds. (August 2005). Algebraic Process Calculi: The First Twenty Five Years and Beyond. PA'05. University of Bologna Residential Center Bertinoro (Forlì), Italy: BRICS.
• Brookes, Stephen (August 2005). "Retracing CSP" (PDF). In Aceto, Luca; Gordon, Andrew D. (eds.). Algebraic Process Calculi: The First Twenty Five Years and Beyond. PA'05. University of Bologna Residential Center Bertinoro (Forlì), Italy: BRICS. pp. 75–80. Retrieved March 10, 2022.