Radon–Nikodym theorem

The Radon–Nikodym theorem involves a measurable space ${\displaystyle (X,\Sigma )}$ on which two σ-finite measures are defined, ${\displaystyle \mu }$ and ${\displaystyle \nu .}$ It states that, if ${\displaystyle \nu \ll \mu }$ (that is, if ${\displaystyle \nu }$ is absolutely continuous with respect to ${\displaystyle \mu }$), then there exists a ${\displaystyle \Sigma }$-measurable function ${\displaystyle f:X\to [0,\infty ),}$ such that for any measurable set ${\displaystyle A\subseteq X,}$

$${\displaystyle \nu (A)=\int _{A}f\,d\mu .}$$

The function ${\displaystyle f}$ satisfying the above equality is uniquely defined up to a ${\displaystyle \mu }$-null set, that is, if ${\displaystyle g}$ is another function which satisfies the same property, then ${\displaystyle f=g}$ ${\displaystyle \mu }$-almost everywhere. The function ${\displaystyle f}$ is commonly written ${\frac {d\nu }{d\mu }}$ and is called the Radon–Nikodym derivative. The choice of notation and the name of the function reflects the fact that the function is analogous to a derivative in calculus in the sense that it describes the rate of change of density of one measure with respect to another (the way the Jacobian determinant is used in multivariable integration).

A similar theorem can be proven for signed and complex measures: namely, that if ${\displaystyle \mu }$ is a nonnegative σ-finite measure, and ${\displaystyle \nu }$ is a finite-valued signed or complex measure such that ${\displaystyle \nu \ll \mu ,}$ that is, ${\displaystyle \nu }$ is absolutely continuous with respect to ${\displaystyle \mu ,}$ then there is a ${\displaystyle \mu }$-integrable real- or complex-valued function ${\displaystyle g}$ on ${\displaystyle X}$ such that for every measurable set ${\displaystyle A,}$

$${\displaystyle \nu (A)=\int _{A}g\,d\mu .}$$

The theorem is very important in extending the ideas of probability theory from probability masses and probability densities defined over real numbers to probability measures defined over arbitrary sets. It tells if and how it is possible to change from one probability measure to another. Specifically, the probability density function of a random variable is the Radon–Nikodym derivative of the induced measure with respect to some base measure (usually the Lebesgue measure for continuous random variables).

For example, it can be used to prove the existence of conditional expectation for probability measures. The latter itself is a key concept in probability theory, as conditional probability is just a special case of it.

Amongst other fields, financial mathematics uses the theorem extensively, in particular via the Girsanov theorem. Such changes of probability measure are the cornerstone of the rational pricing of derivatives and are used for converting actual probabilities into those of the risk neutral probabilities.

If μ and ν are measures over X, and μ ≪ ν

• The Kullback–Leibler divergence from ν to μ is defined to be
  $${\displaystyle D_{\text{KL}}(\mu \parallel \nu )=\int _{X}\log \left({\frac {d\mu }{d\nu }}\right)\;d\mu .}$$

  
• For α > 0, α ≠ 1 the Rényi divergence of order α from ν to μ is defined to be
  $${\displaystyle D_{\alpha }(\mu \parallel \nu )={\frac {1}{\alpha -1}}\log \left(\int _{X}\left({\frac {d\mu }{d\nu }}\right)^{\alpha -1}\;d\mu \right).}$$

  

Here is an example when μ is not σ-finite and the Radon–Nikodym theorem fails to hold.

Consider the Borel σ-algebra on the real line. Let the counting measure, μ, of a Borel set A be defined as the number of elements of A if A is finite, and ∞ otherwise. One can check that μ is indeed a measure. It is not σ-finite, as not every Borel set is at most a countable union of finite sets. Let ν be the usual Lebesgue measure on this Borel algebra. Then, ν is absolutely continuous with respect to μ, since for a set A one has μ(A) = 0 only if A is the empty set, and then ν(A) is also zero.

Assume that the Radon–Nikodym theorem holds, that is, for some measurable function f one has

${\displaystyle \nu (A)=\int _{A}f\,d\mu }$

for all Borel sets. Taking A to be a singleton set, A = {a}, and using the above equality, one finds

${\displaystyle 0=f(a)}$

for all real numbers a. This implies that the function  f , and therefore the Lebesgue measure ν, is zero, which is a contradiction.

Assuming ${\displaystyle \nu \ll \mu ,}$ the Radon–Nikodym theorem also holds if ${\displaystyle \mu }$ is localizable and ${\displaystyle \nu }$ is accessible with respect to ${\displaystyle \mu }$,[5]^{: p. 189, Exercise 9O} i.e., ${\displaystyle \nu (A)=\sup\{\nu (B):B\in {\cal {P}}(A)\cap \mu ^{\operatorname {pre} }(\mathbb {R} _{\geq 0})\}}$ for all ${\displaystyle A\in \Sigma .}$[6]^{: Theorem 1.111 (Radon–Nikodym, II)}[5]^{: p. 190, Exercise 9T(ii)}

Constructing an extended-valued candidate First, suppose μ and ν are both finite-valued nonnegative measures. Let F be the set of those extended-value measurable functions f  : X → [0, ∞] such that:

${\displaystyle \forall A\in \Sigma$ :\qquad \int _{A}f\,d\mu \leq \nu (A)}

F ≠ ∅, since it contains at least the zero function. Now let f₁,  f₂ ∈ F, and suppose A is an arbitrary measurable set, and define:

${\displaystyle {\begin{aligned}A_{1}&=\left\{x\in A:f_{1}(x)>f_{2}(x)\right\},\\A_{2}&=\left\{x\in A:f_{2}(x)\geq f_{1}(x)\right\}.\end{aligned}}}$

Then one has

${\displaystyle \int _{A}\max \left\{f_{1},f_{2}\right\}\,d\mu =\int _{A_{1}}f_{1}\,d\mu +\int _{A_{2}}f_{2}\,d\mu \leq \nu \left(A_{1}\right)+\nu \left(A_{2}\right)=\nu (A),}$

and therefore, max{ f ₁,  f ₂} ∈ F.

Now, let { f_n } be a sequence of functions in F such that

${\displaystyle \lim _{n\to \infty }\int _{X}f_{n}\,d\mu =\sup _{f\in F}\int _{X}f\,d\mu .}$

By replacing  f_n  with the maximum of the first n functions, one can assume that the sequence { f_n } is increasing. Let g be an extended-valued function defined as

${\displaystyle g(x):=\lim _{n\to \infty }f_{n}(x).}$

By Lebesgue's monotone convergence theorem, one has

${\displaystyle \lim _{n\to \infty }\int _{A}f_{n}\,d\mu =\int _{A}\lim _{n\to \infty }f_{n}(x)\,d\mu (x)=\int _{A}g\,d\mu \leq \nu (A)}$

for each A ∈ Σ, and hence, g ∈ F. Also, by the construction of g,

${\displaystyle \int _{X}g\,d\mu =\sup _{f\in F}\int _{X}f\,d\mu .}$

Proving equality Now, since g ∈ F,

${\displaystyle \nu _{0}(A):=\nu (A)-\int _{A}g\,d\mu }$

defines a nonnegative measure on Σ. To prove equality, we show that ν₀ = 0.

Suppose ν₀ ≠ 0; then, since μ is finite, there is an ε > 0 such that ν₀(X) > ε μ(X). To derive a contradiction from ν₀ ≠ 0, we look for a positive set P ∈ Σ for the signed measure ν₀ − ε μ (i.e. a measurable set P, all of whose measurable subsets have non-negative ν₀ − εμ measure), where also P has positive μ-measure. Conceptually, we're looking for a set P, where ν₀ ≥ ε μ in every part of P. A convenient approach is to use the Hahn decomposition (P, N) for the signed measure ν₀ − ε μ.

Note then that for every A ∈ Σ one has ν₀(A ∩ P) ≥ ε μ(A ∩ P), and hence,

${\displaystyle {\begin{aligned}\nu (A)&=\int _{A}g\,d\mu +\nu _{0}(A)\\&\geq \int _{A}g\,d\mu +\nu _{0}(A\cap P)\\&\geq \int _{A}g\,d\mu +\varepsilon \mu (A\cap P)=\int _{A}\left(g+\varepsilon 1_{P}\right)\,d\mu ,\end{aligned}}}$

where 1_P is the indicator function of P. Also, note that μ(P) > 0 as desired; for if μ(P) = 0, then (since ν is absolutely continuous in relation to μ) ν₀(P) ≤ ν(P) = 0, so ν₀(P) = 0 and

${\displaystyle \nu _{0}(X)-\varepsilon \mu (X)=\left(\nu _{0}-\varepsilon \mu \right)(N)\leq 0,}$

contradicting the fact that ν₀(X) > εμ(X).

Then, since also

${\displaystyle \int _{X}\left(g+\varepsilon 1_{P}\right)\,d\mu \leq \nu (X)<+\infty ,}$

g + ε 1_P ∈ F and satisfies

${\displaystyle \int _{X}\left(g+\varepsilon 1_{P}\right)\,d\mu >\int _{X}g\,d\mu =\sup _{f\in F}\int _{X}f\,d\mu .}$

This is impossible because it violates the definition of a supremum; therefore, the initial assumption that ν₀ ≠ 0 must be false. Hence, ν₀ = 0, as desired.

Restricting to finite values Now, since g is μ-integrable, the set {x ∈ X : g(x) = ∞} is μ-null. Therefore, if a  f  is defined as

${\displaystyle f(x)={\begin{cases}g(x)&{\text{if }}g(x)<\infty \\0&{\text{otherwise,}}\end{cases}}}$

then f has the desired properties.

Uniqueness As for the uniqueness, let  f, g : X → [0, ∞) be measurable functions satisfying

${\displaystyle \nu (A)=\int _{A}f\,d\mu =\int _{A}g\,d\mu }$

for every measurable set A. Then, g − f  is μ-integrable, and

${\displaystyle \int _{A}(g-f)\,d\mu =0.}$

In particular, for A = {x ∈ X : f(x) > g(x)}, or {x ∈ X : f(x) < g(x)}. It follows that

${\displaystyle \int _{X}(g-f)^{+}\,d\mu =0=\int _{X}(g-f)^{-}\,d\mu ,}$

and so, that (g − f )⁺ = 0 μ-almost everywhere; the same is true for (g − f )⁻, and thus, f = g μ-almost everywhere, as desired.

If μ and ν are σ-finite, then X can be written as the union of a sequence {B_n}_n of disjoint sets in Σ, each of which has finite measure under both μ and ν. For each n, by the finite case, there is a Σ-measurable function  f_n  : B_n → [0, ∞) such that

${\displaystyle \nu _{n}(A)=\int _{A}f_{n}\,d\mu }$

for each Σ-measurable subset A of B_n. The sum ${\textstyle \left(\sum _{n}f_{n}1_{B_{n}}\right):=f}$ of those functions is then the required function such that ${\textstyle \nu (A)=\int _{A}f\,d\mu }$.

As for the uniqueness, since each of the f_n is μ-almost everywhere unique, so is f.

If ν is a σ-finite signed measure, then it can be Hahn–Jordan decomposed as ν = ν⁺ − ν⁻ where one of the measures is finite. Applying the previous result to those two measures, one obtains two functions, g, h : X → [0, ∞), satisfying the Radon–Nikodym theorem for ν⁺ and ν⁻ respectively, at least one of which is μ-integrable (i.e., its integral with respect to μ is finite). It is clear then that f = g − h satisfies the required properties, including uniqueness, since both g and h are unique up to μ-almost everywhere equality.

If ν is a complex measure, it can be decomposed as ν = ν₁ + iν₂, where both ν₁ and ν₂ are finite-valued signed measures. Applying the above argument, one obtains two functions, g, h : X → [0, ∞), satisfying the required properties for ν₁ and ν₂, respectively. Clearly, f = g + ih is the required function.