Weighted automaton
In theoretical computer science and formal language theory, a weighted automaton or weighted finite-state machine is a generalization of a finite-state machine in which the edges have weights, for example real numbers or integers. Finite-state machines are only capable of answering decision problems; they take as input a string and produce a Boolean output, i.e. either "accept" or "reject". In contrast, weighted automata produce a quantitative output, for example a count of how many answers are possible on a given input string, or a probability of how likely the input string is according to a probability distribution.[2] They are one of the simplest studied models of quantitative automata.[1]: Fig.1
The definition of a weighted automaton is generally given over an arbitrary semiring , an abstract set with an addition operation and a multiplication operation . The automaton consists of a finite set of states, a finite input alphabet of characters and edges which are labeled with both a character in and a weight in . The weight of any path in the automaton is defined to be the product of weights along the path, and the weight of a string is the sum of the weights of all paths which are labeled with that string. The weighted automaton thus defines a function from to .[2]
Weighted automata generalize deterministic finite automata (DFAs) and nondeterministic finite automata (NFAs), which correspond to weighted automata over the Boolean semiring, where addition is logical disjunction and multiplication is logical conjunction. In the DFA case, there is only one accepting path for any input string, so disjunction is not applied. When the weights are real numbers and the outgoing weights for each state add to one, weighted automata can be considered a probabilistic model and are also known as probabilistic automata. These machines define a probability distribution over all strings, and are related to other probabilistic models such as Markov decision processes and Markov chains.
Weighted automata have applications in natural language processing where they are used to assign weights to words and sentences,[3][2]: 571–596 as well as in image compression.[2]: 453–480 They were first introduced by Marcel-Paul Schützenberger in his 1961 paper On the definition of a family of automata.[4] Since their introduction, many extensions have been proposed, for example nested weighted automata,[5] cost register automata,[6] and weighted finite-state transducers.[7] Researchers have studied weighted automata from the perspective of learning a machine from its input-output behavior[8] (see computational learning theory) and studying decidability questions.[9]
Definition
A commutative semiring (or rig) is a set R equipped with two distinguished elements and addition and multiplication operations and such that is commutative and associative with identity , is commutative and associative with identity , distributes over , and 0 is an absorbing element for .
A weighted automaton over is a tuple where:
- is a finite set of states.
- is a finite alphabet.
- is a finite set of transitions , where is called a character and is called a weight.
- is an initial weight function.
- is a final weight function.
A path on input is a finite path in the graph, where the concatenation of the character labels equals . The weight of the path is the product () of the weights along the path, additionally multiplied by the initial and final weights . The weight of the word is the sum () of the weights of all paths on input (or 0 if there are no accepting paths). In this way the machine defines a function .
The underlying NFA of is an NFA formed by removing all transitions with weight and then erasing all of the weights on the transitions , so that the new transition set lies in . The initial states and final states are the set of states such that and , respectively.
Variations
- A weighted automaton is deterministic if the underlying NFA is deterministic and unambiguous if the underlying NFA is unambiguous. In these cases, there is always exactly one accepting path, so the operation is never applied and can be omitted from the definition.
- It is possible to extend the definition to allow epsilon transitions , where is the empty string. In this case, one must then require that there are no cycles of epsilon transitions. This does not increase the expressiveness of weighted automata.[2][10] If epsilon transitions are allowed, the initial weights and final weights can be replaced by initial and final sets of states without loss of expressiveness.
- The transition function can be given as a matrix with entries in for each , rather than a set of transitions. The entry of the matrix at is the sum of all transitions labeled .[2][11]
- Some authors omit the initial and final weight functions and .[1] Instead, and are replaced by a set of initial and final states. If epsilon transitions are not present, this technically decreases expressiveness as it forces to depend only on the number of states that are both initial and final.
- Some authors restrict to specific semirings, such as or , particularly when studying decidability results.[1][9][4]
- The requirement that there is a zero element for is sometimes omitted; in this case the machine defines a partial function from to rather than a total function.
See also
References
- Chatterjee, Krishnendu; Henzinger, Thomas A.; Otop, Jan (2016). "Quantitative Monitor Automata". In Rival, Xavier (ed.). Static Analysis. Lecture Notes in Computer Science. Vol. 9837. Berlin, Heidelberg: Springer. pp. 23–38. doi:10.1007/978-3-662-53413-7_2. ISBN 978-3-662-53413-7.
- Droste, Manfred; Kuich, Werner; Vogler, Heiko, eds. (2009). Handbook of Weighted Automata. Bibcode:2009hwa..book.....D. doi:10.1007/978-3-642-01492-5. ISBN 978-3-642-01491-8. ISSN 1431-2654.
{{cite book}}
:|journal=
ignored (help) chs.1-4, pp.3-26, 69-71, 122-126. - Chiang, David. "Weighted Automata" (PDF). Natural Language Processing (CSE 40657/60657), course notes, Fall 2019. Retrieved 2021-11-09.
- Schützenberger, M. P. (1961-09-01). "On the definition of a family of automata". Information and Control. 4 (2): 245–270. doi:10.1016/S0019-9958(61)80020-X. ISSN 0019-9958.
- Chatterjee, Krishnendu; Henzinger, Thomas A.; Otop, Jan (2017-12-14). "Nested Weighted Automata". ACM Transactions on Computational Logic. 18 (4): 31:1–31:44. arXiv:1504.06117. doi:10.1145/3152769. ISSN 1529-3785. S2CID 15070803.
- Alur, Rajeev; D'Antoni, Loris; Deshmukh, Jyotirmoy; Raghothaman, Mukund; Yuan, Yifei (2013). "Regular Functions and Cost Register Automata". 2013 28th Annual ACM/IEEE Symposium on Logic in Computer Science. pp. 13–22. doi:10.1109/LICS.2013.65. ISBN 978-1-4799-0413-6. S2CID 1286659.
- Mohri, Mehryar; Pereira, Fernando; Riley, Michael (2002-01-01). "Weighted finite-state transducers in speech recognition". Computer Speech & Language. 16 (1): 69–88. doi:10.1006/csla.2001.0184. ISSN 0885-2308.
- Balle, Borja; Mohri, Mehryar (2015). "Learning Weighted Automata". In Maletti, Andreas (ed.). Algebraic Informatics. Lecture Notes in Computer Science. Vol. 9270. Cham: Springer International Publishing. pp. 1–21. doi:10.1007/978-3-319-23021-4_1. ISBN 978-3-319-23021-4.
- Almagor, Shaull; Boker, Udi; Kupferman, Orna (2011). "What's Decidable about Weighted Automata?". In Bultan, Tevfik; Hsiung, Pao-Ann (eds.). Automated Technology for Verification and Analysis. Lecture Notes in Computer Science. Vol. 6996. Berlin, Heidelberg: Springer. pp. 482–491. doi:10.1007/978-3-642-24372-1_37. ISBN 978-3-642-24372-1. S2CID 10159261.
- Mohri, Mehryar (2009), Droste, Manfred; Kuich, Werner; Vogler, Heiko (eds.), "Weighted Automata Algorithms", Handbook of Weighted Automata, Monographs in Theoretical Computer Science. An EATCS Series, Berlin, Heidelberg: Springer, pp. 213–254, Bibcode:2009hwa..book..213M, doi:10.1007/978-3-642-01492-5_6, ISBN 978-3-642-01492-5, retrieved 2021-11-11
- Droste, Manfred; Gastin, Paul (2007-06-21). "Weighted automata and weighted logics". Theoretical Computer Science. Automata, Languages and Programming. 380 (1): 69–86. doi:10.1016/j.tcs.2007.02.055. ISSN 0304-3975.