Proportional reduction in loss

Proportional reduction in loss (PRL) is a general framework for developing and evaluating measures of the reliability of particular ways of making observations which are possibly subject to errors of all types. Such measures quantify how much having the observations available has reduced the loss (cost) of the uncertainty about the intended quantity compared with not having those observations.

Proportional reduction in error is a more restrictive framework widely used in statistics, in which the general loss function is replaced by a more direct measure of error such as the mean square error. Examples are the coefficient of determination and Goodman and Kruskal's lambda.[1]

The concept of proportional reduction in loss was proposed by Bruce Cooil and Roland T. Rust in their 1994 paper Reliability and Expected Loss: A Unifying Principle.[2] Many commonly used reliability measures for quantitative data (such as continuous data in an experimental design) are PRL measures, including Cronbach's alpha and measures proposed by Ben J. Winer in 1971.[3] It also provides a general way of developing measures for the reliability of qualitative data. For example, this framework provides several possible measures that are applicable when a researcher wants to assess the consensus between judges who are asked to code a number of items into mutually exclusive qualitative categories.[4] Measures of this latter type have been proposed by several researchers, including Perrault and Leigh in 1989.[5]