My PhD topic: Nonlinear Experimental Design

Bayesian experimental design provides a general probability-theoretical framework from which other theories on experimental design can me derived. It is based on Bayesian inference to interpret the observations/data acquired during the experiment. This allows accounting for both any prior knowledge on the parameters to be determined as well as uncertainties in observations.

In Bayesian experimental design, the aim when designing an experiment is to maximize the expected utility of the experiment outcome. The utility is most commonly defined in terms of a measure of the accuracy of the information provided by the experiment (e.g. the Shannon information or the negative variance), but may also involve factors such as the financial cost of performing the experiment. What will be the optimal experiment design depends on the particular utility criterion chosen.

Relations to more specialized optimal design theory

Notation parameters to be determined observation or data design PDF for making observation , given parameter values and design prior PDF marginal PDF in observation space posterior PDF utility of the design utility of the experiment outcome after observation with design

Linear theory
If the model is linear, the prior probability density function (PDF) is homogeneous and observational errors are Multivariate normal distribution | normally distributed, the theory simplifies to the classical optimal design | optimal experimental design theory.

Approximate normality
In numerous publications on Bayesian experimental design, it is (often implicitly) assumed that all posterior PDFs will be approximately normal. This allows for the expected utility to be calculated using linear theory, averaging over the space of model parameters, an approach reviewed in Chaloner & Verdinelli (1995). Caution must however be taken when applying this method, since approximately normality of all possible posteriors is difficult to verify, even in cases of normal observational errors and uniform a prior PDF.

Mathematical formulation

Given a vector of parameters to determine, a Prior probability | prior PDF over those parameters and a PDF for making observation , given parameter values and design , the posterior PDF can be calculated using Bayes’ theorem

,

(1)

where is the marginal probability density in observation space

.

(2)

The expected utility of an experiment with design can then be defined

,

(3)

where is some real-valued functional of the Posterior probability | posterior PDF after making observation using an experiment design .

Gain in Shannon information as utility

If the utility is defined as the prior-posterior gain in Differential entropy | Shannon information

.

(4)

Lindley (1956) noted that the expected utility will then be coordinate-independent and can be written in two forms

of which the latter can be evaluated without the need for evaluating individual posterior PDFs
for all possible observations . Worth noting is that the first term on the second equation line will be a constant, as long as the observational uncertainty does not depend on the design . Several authors have considered numerical techniques for evaluating and optimizing this criterion, e.g. van den Berg, Curtis & Trampert (2003) and Ryan (2003).

© 2008 Emanuel Winterfors code can be used in comments: $latex p(\theta)$ gives