This article was published in the International Journal of Quantum Information. It should be cited as:

Bond, R. L., He, Y.‑H., & Ormerod, T. C. (2018). A quantum framework for likelihood ratios. International Journal of Quantum Information, 16(1), 1850002. doi: 10.1142/S0219749918500028

International Journal of Quantum Information
Vol. 16, No. 1 (2018) 1850002 (14 pages)
© World Scientific Publishing Company
DOI: 10.1142/S0219749918500028

1. Introduction

In recent years, Bayesian statistical research has often been epistemologically driven, guided by de Finetti’s famous quote that “probability does not exist” (de Finetti, 1974). For example, the “quantum Bayesian” methodology of Caves, Fuchs & Schack has applied de Finetti’s ideas to Bayes’ theorem for use in quantum mechanics (Caves, Fuchs, & Schack, 2002). In doing so, Caves et al. have argued that statistical systems are best interpreted by methods in which the Bayesian likelihood ratio is seen to be both external to the system and subjectively imposed on it by the observer (Timpson, 2008). However, the Caves et al. approach is problematic. At a human scale, for instance, an observer’s belief as to the chances of a fair coin landing either “heads” or “tails” has no known effect. Indeed, for all practical purposes, the “heads:tails” likelihood ratio of 0.5:0.5 is only meaningful when considered as a property of the coin’s own internal statistical system rather than as some ephemeral and arbitrary qualia.

Yet, to date, the axiomatic difficulties associated with Bayes’ theorem, notably its reliance upon the use of marginal probabilities in the absence of structural statistical information (eg., estimates of covariate overlap), as well as the assumed conditional independence of data, have largely been approached from a “mend and make do” standpoint. For instance, the “maximum likelihood” approach of Dempster, Laird, & Rubin calculates iteratively derived measures of covariate overlap which not only lack a sense of natural authenticity, but also introduce fundamental assumptions into the statistical analysis (Dempster, Laird, & Rubin, 1977).

Instead, this paper adopts a different approach to the analysis of statistical systems. By using quantum mechanical mathematical spaces, it is demonstrated that the creation of isomorphic representations of classical data-sets as entangled systems allows for a natural, albeit non-trivial, calculation of likelihood ratios. It is expected that this technique will find applications within the fields of quantum state estimation, and quantum information theory.

2. The limits of Bayes’ theorem

Bayes’ theorem is used to calculate the conditional probability of a statement, or hypothesis, being true given that other information is also true. It is usually written as

Here, \(P(H_{i}|D)\) is the conditional probability of hypothesis \(H_{i}\) being true given that the information \(D\) is true; \(P(D|H_{i})\) is the conditional probability of \(D\) being true if \(H_{i}\) is true; and \(\sum\limits_{j}P(H_{j})P(D|H_{j})\) is the sum of the probabilities of all hypotheses multiplied by the conditional probability of \(D\) being true for each hypothesis (Oaksford & Chater, 2007).

To exemplify using the contingency information in \(\eqref{figure1}\), if one wishes to calculate the nature of a randomly selected particle from a set of 20, given that it has spin \(\uparrow\), then using Bayes’ theorem it is trivial to calculate that particle \(\alpha\) is the most likely type with a likelihood ratio of approximately 0.53:0.47,

However, difficulties arise in the use of Bayes’ theorem for the calculation of likelihood ratios where there are multiple non-exclusive data sets. For instance, if the information in \(\eqref{figure1}\) is expanded to include data about particle charge \(\eqref{figure2}\) then the precise covariate overlap (ie., \(D_1 \cap D_2\)) for each particle becomes an unknown.

All that may be shown is that, for each particle, the occurrence of both features forms a range described by

where \(n(H_i)\) is the total number of exemplars \(i\), \(n(D_1|H_i)\) is the total number of \(i\) with spin \(\uparrow\), and \(n(D_2|H_i)\) is the total number of \(i\) with a positive charge.

Specifically for \(\eqref{figure2}\) these ranges equate to

The simplest approach to resolving this problem is to naively ignore any intersection, or co-dependence, of the data and to directly multiply the marginal probabilities. Hence, given \(\eqref{figure2}\), the likelihood of particle \(\alpha\) having the greatest occurrence of both spin \(\uparrow\) and a positive charge would be calculated as

Yet, because the data intersect, this probability value is only one of a number which may be reasonably calculated. Alternatives include calculating a likelihood ratio using the mean value \(\mu\) of the frequency ranges for each hypothesis

and taking the mean value of the probability range derived from the frequency range

Given this multiplicity of probability values, it would seem that none of these methods may lay claim to normativity. This problem of covariate overlap has, of course, been previously addressed within statistical literature. For instance, the “maximum likelihood” approach of Dempster, Laird, & Rubin has demonstrated how an “expectation-maximization” algorithm may be used to derive appropriate covariate overlap measures (Dempster et al., 1977). Indeed, the mathematical efficacy of this technique has been confirmed by (Wu, 1983). However, it is difficult to see how such an iterative methodology can be employed without introducing axiomatic assumptions. Further, since any assumptions, irrespective of how benign they may appear, have the potential to skew results, what is required is an approach in which covariate overlaps can be automatically, and directly, calculated from contingency data.

3. A quantum mechanical proof of Bayes’ theorem for independent data

Previously unconsidered, the quantum mechanical von Neumann axioms would seem to offer the most promise in this regard, since the re-conceptualization of covariate data as a quantum entangled system allows for statistical analysis with few, non-arbitrary assumptions. Unfortunately there are many conceptual difficulties that can arise here. For instance, a Dirac notation representation of \(\eqref{figure2}\) as a standard quantum superposition is

In this example, \(\eqref{superpos}\) cannot be solved since the possible values of \(D_1 \cap D_2\) for each hypothesis \(\eqref{min_max_actual}\) have been described as equal chance outcomes within a general superposition of \(H_1\) and \(H_2\), with the unknown coefficients \(\alpha\) and \(\beta\) assuming the role of the classical Bayesian likelihood ratio.

The development of an alternative quantum mechanical description necessitates a return to the simplest form of Bayes’ theorem using the case of exclusive populations \(H_i\) and data sets \(D\), \(\bar{D}\), such as given in \(\eqref{figure1}\). Here, the overall probability of \(H_1\) may be simply calculated as

The a priori uncertainty in \(\eqref{figure1}\) may be expressed by constructing a wave function in which the four data points are encoded as a linear superposition

Since there is no overlap between either \(D\) and \(\bar{D}\) or the populations \(H_1\) and \(H_2\), each datum automatically forms an eigenstate basis with the orthonormal conditions

where the normalization of the wave function demands that

so that the sum of the modulus squares of the coefficients \(\alpha_{i,j}\) gives a total probability of 1

For simplicity let

If the coefficients \(\alpha_{i,j}\) from \(\eqref{superpos2}\) are set as required by \(\eqref{figure1}\), it follows that

so that the normalised wave function \(\newcommand\ket[1]{\left|#1\right>} \ket{\Psi}\) is described by

for some normalization constant \(N\).

The orthonormality condition \(\eqref{ortho2}\) implies that

thereby giving the full wave function description

If the value of \(P(H_1|D)\) is to be calculated, ie., the property \(D\) is observed, then the normalized wave function \(\eqref{superpos2}\) necessarily collapses to

where the coefficients \(\alpha_{1,2}\) may be determined by projecting \(\newcommand\ket[1]{\left|#1\right>} \ket{\Psi}\) on to the two terms in \(\newcommand\ket[1]{\left|#1\right>} \ket{\Psi’}\) using \(\eqref{ortho1}\), giving

Normalizing \(\eqref{collapse1}\) with the coefficient \(N’\)

and using the normalization condition \(\eqref{ortho2}\), implies that

Thus, after collapse, the properly normalized wave function \(\eqref{collapse2}\) becomes

which means that the probability of observing \(\newcommand\ket[1]{\left|#1\right>} \ket{H_1 \otimes D}\) is

This is entirely consistent with Bayes’ theorem and demonstrates its derivation using quantum mechanical axioms.

4. Quantum likelihood ratios for co‑dependent data

Having established the principle of using a quantum mechanical approach for the calculation of simple likelihood ratios with mutually exclusive data \(\eqref{figure1}\), it is now possible to consider the general case of \(n\) hypotheses and \(m\) data \(\eqref{figure3}\), where the data are co‑dependent, or intersect.

Here the contingency table in \(\eqref{figure3}\) has been indexed using

While the general wave function remains the same as before, the overlapping data create non-orthonormal inner products which can be naturally defined as

Assuming, for simplicity, that the overlaps \(c_{ij}^{\alpha}\) are real, then there is a symmetry in that \(c_{ij}^\alpha = c_{ji}^\alpha\) for each \(\alpha\). Further, for each \(\alpha\) and \(i\), the state is normalized, ie., \(c_{ii}^\alpha = 1\). The given independence of the hypotheses \(H_\alpha\) also enforces the Kroenecker delta function, \(\delta_{\alpha\beta}\).

The Hilbert space \(V\) spanned by the kets \(\newcommand\ket[1]{\left|#1\right>} \ket{H_\alpha \otimes D_i}\) is \(mn\)‑dimensional and, because of the independence of \(H_\alpha\), naturally decomposes into the direct sum

with respect to the inner product, thereby demonstrating that the non‑orthonormal conditions are the direct sum of \(m\) vector spaces \(V^\alpha\).

Since the inner products are non-orthonormal, each \(V^{\alpha}\) must be individually orthonormalised. Given that \(V\) splits into a direct sum, this may be achieved for each subspace \(V^{\alpha}\) by applying the Gram‑Schmidt algorithm to \(\newcommand\ket[1]{\left|#1\right>} \{ \ket{H_\alpha \otimes D_i} \}\) of \(V\). Consequently, the orthonormal basis may be defined as

for each \(\alpha=1,2,\ldots,n\) with \(m\times m\) matrices \(A_{i,k}^{\alpha}\), for each \(\alpha\).

Substituting the inner products \(\eqref{innerGen}\) gives

The wave-function may now be written as a linear combination of the orthonormalised kets \(\newcommand\ket[1]{\left|#1\right>} \ket{K^\alpha_i}\) with the coefficients \(b_i^\alpha\), and may be expanded into the \(\newcommand\ket[1]{\left|#1\right>} \ket{H_\alpha \otimes D_i}\) basis using \(\eqref{KHD}\), ie.,

As with \(\eqref{ortho4}\) from earlier, the coefficients in \(\eqref{PsiGen}\) should be set as required by the contingency table

where, to solve for the \(b\)‑coefficients, \(\eqref{AAc}\) may be used to invert

giving

Having relabelled the indices as necessary, a back-substitution of \(\eqref{bAx}\) into the expansion \(\eqref{PsiGen}\) gives

which is the same as having simply assigned each ket’s coefficient to the square root of its associated entry in the contingency table.

The normalization factor for \(\newcommand\ket[1]{\left|#1\right>} \ket{\Psi}\) is simply \(1/\sqrt{N}\), where \(N\) is the sum of the squares of the coefficients \(b\) of the orthonormalised bases \(\newcommand\ket[1]{\left|#1\right>} \ket{K^\alpha_i}\),

Thus, the final normalized wave function is

where \(\alpha\) is summed from 1 to \(n\), and \(i,j\) are summed from 1 to \(m\). Note that, in the denominator, the diagonal term \(\sqrt{x_{i \alpha}x_{j \alpha}}c_{ij}^\alpha\), which occurs whenever \(i=j\), simplifies to \(x_{i \alpha}\) since \(c_{ii}^\alpha = 1\) for all \(\alpha\).

From \(\eqref{NormalWave}\) it follows that, exactly in parallel to the non‑intersecting case, if all properties \(D_i\) are observed simultaneously, the probability of any hypothesis \(H_\alpha\), for a fixed \(\alpha\), is

In the case of non‑even populations for each hypothesis (ie., non‑even priors), the calculated probabilities should be appropriately weighted.

5. Example solution

Returning to the problem presented in the contingency table \(\eqref{figure2}\), it is now possible to calculate the precise probability for a randomly selected particle with the properties of “spin \(\uparrow\)” and “charge +” being particle \(\alpha\) (\(H_1\)). For this \(2 \times 2\) matrix, recalling from \(\eqref{innerGen}\) that \(c_{ii}^\alpha=1\) and \(c_{ij}^\alpha = c_{ji}^\alpha\), the general expression \(\eqref{PHalpha}\) may be written as

where, adhering to the earlier notation \(\eqref{griddef}\),

and, for brevity, \(c_1 := c_{1,2}^1\), \(c_2 := c_{1,2}^2\). For simplicity, \(P(H_i|D_1 \cap D_2)\) will henceforth be denoted as \(P_i\).

Implementing \(\eqref{entang2}\) is dependent upon deriving solutions for the yet unknown expressions \(c_{i}\), \(i=1,2\) which govern the extent of the intersection in \(\eqref{innerGen}\). This can only be achieved by imposing reasonable constraints upon \(c_i\) which have been inferred from expected behaviour and known outcomes, ie., through the use of boundary values and symmetries. Specifically, these constraints are:

(i) Data dependence

The expressions \(c_i\) must, in some way, be dependent upon the data given in the contingency table, ie.,

(ii) Probability

The calculated values for \(P_i\) must fall between 0 and 1. Since \(x_i\) and \(y_i\) are positive, it suffices to take

(iii) Complementarity

The law of total probability dictates that

which can easily be seen to hold.

(iv) Symmetry

The exchanging of rows within the contingency tables should not affect the calculation of \(P_i\). In other words, for each \(i=1,2\), \(P_i\) is invariant under \(x_i \leftrightarrow y_i\). This constraint implies that

Equally, if the columns are exchanged then \(P_i\) must map to each other, ie., for each \(i=1,2\) then \(P_1 \leftrightarrow P_2\) under \(x_1 \leftrightarrow x_2,\: y_1 \leftrightarrow y_2\) which gives the further constraint that

(v) Known values

There are a number of contingency table structures which give rise to a known probability, ie.,

where \(m,n\) are positively valued probabilities. For such contingency tables the correct probabilities should always be returned by \(c_i\). Applying this principle to \(\eqref{entang2}\) gives the constraints

(vi) Non‑homogeneity

Bayes’ theorem returns the same probability for any linearly scaled contingency tables, eg.,

While homogeneity may be justified for conditionally independent data, this is not the case for intersecting, co‑dependent data since the act of scaling changes the nature of the intersections and the relationship between them. This may be easily demonstrated by taking the possible value ranges for \(\eqref{homogen1}\) and \(\eqref{homogen2}\), calculated using \(\eqref{min_max}\), which are

The effect of scaling has not only introduced uncertainty where previously there had been none, but has also introduced the possibility of 0 as a valid answer for both hypotheses. Further, the spatial distance between the hypotheses has also decreased. For these reasons it would seem unreasonable to assert that \(\eqref{homogen1}\) and \(\eqref{homogen2}\) share the same likelihood ratio.Using these principles and constraints it becomes possible to solve \(c_i\). From the principle of symmetry it follows that

and that the equalities \(\eqref{movern2}\), \(\eqref{movern3}\) for \(P_i=0.5\) automatically hold. Further, \(\eqref{movern}\) solves to give

which, because \(c_1(n,1,m,1)=c_2(m,1,n,1)\), finally gives

Substituting \(g(m,n) := \sqrt{n} c_1(m,1,n,1)\) transforms \(\eqref{function1}\) into an anti‑symmetric bivariate functional equation in \(m,n\),

whose solution is \(g(m,n)=\frac{m}{2\sqrt{mn}} \ .\)

This gives a final solution for the coefficients \(c_{1,2}\) of

Thus, substituting \(\eqref{c12sol}\) into \(\eqref{entang2}\) gives the likelihood ratio expression of,

Given that the population sizes of \(H_1\) and \(H_2\) are the same, no weighting needs to take place. Hence, the value of \(P(H_1|D_1 \cap D_2)\) for \(\eqref{figure2}\) may now be calculated to be

6. Discussion

One of the greatest obstacles in developing any statistical approach is demonstrating correctness. This formula is no different in that respect. If correctness could be demonstrated then, a priori, there would be an appropriate existing method which would negate the need for a new one. All that may be hoped for in any approach is that it generates appropriate answers when they are known, reasonable answers for all other cases, and that these answers follow logically from the underlying mathematics.

However, what is clear is that the limitations of the naive Bayes’ classifier render any calculations derived from it open to an unknown margin of error. Given the importance of accurately deriving likelihood ratios this is troubling. This is especially true when the statistical tolerance of calculations is marginal.

As a quantum mechanical methodology this result is able to calculate accurate, iteration free, likelihood ratios which fall beyond the scope of existing statistical techniques, and offers a new theoretical approach within both statistics and physics. Further, through the addition of a Hamiltonian operator to introduce time‑evolution, it can offer likelihood ratios for future system states with appropriate updating of the contingency table. In contrast, Bayes’ theorem is unable to distinguish directly between time‑dependent and time‑independent systems. This may lead to situations where the process of contingency table updating results in the same decisions being made repeatedly with the appearance of an ever increasing degree of certainty. Indeed, from \(\eqref{bayesproof}\), it would seem that the naive Bayes’ classifier is only a special case of a more complex quantum mechanical framework, and may only be used where the exclusivity of data is guaranteed.

The introduction of a Hamiltonian operator, and a full quantum dynamical formalism, is in progress, and should have profound implications for the physical sciences. Inevitably, such a formalism will require a sensible continuous classical limit. In other words, the final expressions for the likelihood ratios should contain a parameter, in some form of \(\hbar\), which, when going to 0, reproduces a classically known result. For example, the solutions to \(\eqref{c12sol}\) could be moderated as

so that in the limit of \(\hbar \rightarrow 0\), the intersection parameters, \(c_1\) and \(c_2\), vanish to return the formalism to the classical situation of independent data.

7. Conclusion

This article has demonstrated both theoretically, and practically, that a quantum mechanical methodology can overcome the axiomatic limitations of classical statistics. In doing so, it challenges the orthodoxy of de Finetti’s epistemological approach to statistics by demonstrating that it is possible to derive “real” likelihood ratios from information systems without recourse to arbitrary and subjective evaluations.

While further theoretical development work needs to be undertaken, particularly with regards to the application of these mathematics in other domains, it is hoped that this article will help advance the debate over the nature and meaning of statistics within the physical sciences.

Acknowledgements

YHH would like to thank the Science and Technology Facilities Council, UK, for grant ST/J00037X/1, the Chinese Ministry of Education for a Chang‑Jiang Chair Professorship at NanKai University, as well as the City of Tian‑Jin for a Qian‑Ren Scholarship, and Merton College, Oxford, for her enduring support.