A computational method for processing measurement data is discussed that applies the law of propagation of uncertainty, which is described in the Guide to the Expression of Uncertainty in Measurement. The method introduces the notion of an 'uncertain number', which is an entity that encapsulates information about the value and uncertainty of a quantity. Basic mathematical operations can be defined for uncertain numbers so that relationships between quantities can be expressed in a familiar way. Uncertain number equations can be arbitrarily combined, or decomposed into expressions for intermediate results, providing flexibility and convenience for automatic data processing while rigorously supporting uncertainty propagation. A geometrical description of simple uncertain number operations is also given.

The uncertain-numbers method (Hall B D 2006 Metrologia 43 L56–61) is an alternative computational procedure to the Law of Propagation of Uncertainty (LPU) described in the Guide to the Expression of Uncertainty in Measurement. One advantage of the method is that data processing can be carried out in an arbitrary series of steps and much of the mathematical analysis normally associated with the LPU can be automated by software. This has applications for measuring systems, which are modular in design and use internal data processing to apply corrections to raw data. Several scenarios involving radio frequency power measurements are used to illustrate the new method in this context. The scenarios show something of the difficulty inherent in calculating uncertainty for modern measurement systems and in particular highlight the occurrence of systematic errors arising from internal instrument correction factors. Such errors introduce correlation to a series of measurements and must be handled with care when functions of results, such as means, differences and ratios, are required.

An earlier publication (Hall 2006 Metrologia 43 L56–61) introduced the notion of an uncertain number that can be used in data processing to represent quantity estimates with associated uncertainty. The approach can be automated, allowing data processing algorithms to be decomposed into convenient steps, so that complicated measurement procedures can be handled. This paper illustrates the uncertain-number approach using several simple measurement scenarios and two different software tools. One is an extension library for Microsoft Excel®. The other is a special-purpose calculator using the Python programming language.

This paper presents methods for evaluating, expressing and using the uncertainty associated with complex S-parameter measurements. The methods are based on internationally recommended guidelines, published by the International Organization for Standardization (ISO), with extensions to accommodate the complex nature of the measurands. The treatment of measurements from both one-port and multi-port devices is presented and this is used to propagate uncertainty from complex-valued S-parameters to radio frequency (rf) and microwave quantities derived from these. A simple example, involving the comparison loss correction used during a microwave power meter calibration, is included to demonstrate the principles of propagating the uncertainty in complex S-parameter measurements.

On the propagation of uncertainty in complex-valued quantities

This paper explores a recent suggestion to use a bivariate form of the Gaussian 'error propagation law' to propagate uncertainty in the measurement of complex-valued quantities (Ridler N M and Salter M J 2002 Metrologia 39 295–302). Several alterative formulations of the law are discussed in which the contributions from individual input terms are more explicit. The calculation of complex-valued sensitivity coefficients is discussed and a matrix generalization of the notion of a 'component of uncertainty' in a measurement result is introduced. A form of a 'chain rule' is given for the propagation of uncertainty when several intermediate equations are involved.

An algorithm for propagating measurement uncertainty in a system of interconnected modules is presented. The method adheres strictly to current best-practice in the evaluation and reporting of measurement uncertainty. It allows modular instrumentation systems to be designed that will propagate uncertainty automatically. The algorithm is simple, general, efficient, and can be implemented with little difficulty. It inherently provides the kind of dynamic "plug-and-play" flexibility expected of modern instrumentation.

The 'GUM Tree' design pattern for uncertainty software

B. D. Hall

in: Advanced Computational Tools in Metrology & Testing VI, pp 199-208, (2004, World Scientific Series on Advances in Mathematics for Applied Sciences, Vol.66, editors: P Ciarlini, M G Cox, F Pavese and G B Rossi)

Abstract

The paper describes a simple approach to software design in which the ‘Law of propagation of uncertainty' is used to obtain measurement results that include a statement of uncertainty, as described in the Guide to the Expression of Uncertainty in Measurement (ISO, Geneva, 1995). The technique can be used directly for measurement uncertainty calculations, but is of particular interest when applied to the design of instrumentation systems. It supports modularity and extensibility, which are key requirements of modern instrumentation, without imposing an additional performance burden. The technique automates the evaluation and propagation of components of uncertainty in an arbitrary network of modular measurement components.

An extension of the 'GUM Tree' method for complex numbers

B. D. Hall

in: Advanced Computational Tools in Metrology & Testing VIII, pp 158-163, (2009, World Scientific Series on Advances in Mathematics for Applied Sciences, Vol.78, editors: F Pavese, M Bär, A B Forbes, J M Linares, C Perruchet and N F Zhang)

Abstract

An uncertain complex number is an entity that encapsulates information about an estimate of a complex quantity. Simple algorithms can propagate uncertain complex numbers through an arbitrary sequence of data-processing steps, providing a flexible tool for uncertainty calculations supported by software. The technique has important applications in modern instrumentation systems. Some radio-frequency measurements are discussed.

A method of uncertainty analysis based on classical statistical principles is presented for a measurand that is a linear combination of multidimensional input quantities. The method assigns the measurand a combined standard uncertainty matrix and an effective degrees of freedom, which allows the measurand to be estimated by an ellipsoidal confidence region in the multidimensional space. Simulations for a 95 % nominal confidence level show the ellipsoids to contain the measurand with probability approximately 0.95, as required. The derivation of the method assumes all input uncertainties to be evaluated by the Type A method. So the method is analogous to the Welch-Satterthwaite formula for one-dimensional data, a derivation of which is given in an appendix.

The Welch–Satterthwaite (W–S) formula described in the Guide to the Expression of Uncertainty in Measurement enables an effective number of degrees of freedom to be associated with the standard uncertainty of a measurement estimate. This facilitates the calculation of an expanded uncertainty interval for the value of the measurand. However, the W–S formula is only applicable when the components of measurement error with finite degrees of freedom are uncorrelated. This paper considers the generalization of the formula to accommodate correlated components with finite degrees of freedom. We show that the number of degrees of freedom to be associated with any estimate derived from repeated observation of several quantities is n - 1, where n is the number of sets of observations, and we then give two equations by which the number of effective degrees may be calculated.

An extension to GUM methodology: degrees-of-freedom calculations for correlated multidimensional estimates

The Guide to the Expression of Uncertainty in Measurement advocates the use of an 'effective number of degrees of freedom' for the calculation of an interval of measurement uncertainty. However, it does not describe how this number is to be calculated when (i) the measurand is a vector quantity or (ii) when the errors in the estimates of the quantities defining the measurand (the 'input quantities') are not incurred independently. An appropriate analysis for a vector-valued measurand has been described (Metrologia 39 (2002) 361-9), and a method for a one-dimensional measurand with dependent errors has also been given (Metrologia 44 (2007) 340-9). This paper builds on those analyses to present a method for the situation where the problem is multidimensional and involves correlated errors. The result is an explicit general procedure that reduces to simpler procedures where appropriate. The example studied is from the field of radio-frequency metrology, where measured quantities are often complex-valued and can be regarded as vectors of two elements.

This communication demonstrates the need for independent validation when an uncertainty calculation procedure is applied to a particular type of measurement problem. A simple measurement scenario is used to highlight differences in the performance of two general methods of uncertainty calculation, one from the Guide to the Expression of Uncertainty in Measurement (GUM) and one from Supplement 1 to the 'Guide to the Expression of Uncertainty in Measurement'—Propagation of Distributions using a Monte Carlo method. The performance of these methods is investigated in terms of the long-run success rate when applied to many simulated measurements in the scenario. An individual application of the method is deemed successful if an uncertainty interval containing the measurand is obtained. The alternative approach to validation taken in the Supplement, that an uncertainty interval calculated by a Monte Carlo method can be used to validate the GUM method, is not consistent with the results of this study.

This paper continues a discussion generated by a recent paper of Hall (2008 Metrologia 45 L5–8) regarding the performance of methods of uncertainty evaluation. The 'validity' of a method of generating intervals of measurement uncertainty is identified principally with the frequency with which these intervals contain the measurand. Two approaches to the evaluation of such intervals are described, and their performances are compared for the simple measurement function appearing in Hall's paper. The first is a Bayesian approach, which is consistent with the numerical method described in Supplement 1 to the Guide to the Expression of Uncertainty in Measurement. The second is an approach based on frequentist principles. Simulations with fixed values of the unknown parameters are conducted to find the rate at which the methods generate intervals containing the value of the measurand and to find the mean widths of the intervals produced. The results show that the standard Bayesian procedure and its modifications can perform poorly. In contrast, the frequentist procedure achieves the required rate of 0.95 while generating intervals of similar width.

Using simulation to check uncertainty calculations

A simulation approach is described for testing the performance of uncertainty calculations based on the Guide to the Expression of Uncertainty in Measurement (GUM) 1st edn (Sèvres, Paris: BIPM Joint Committee for Guides in Metrology). Performance is measured in terms of the long-run success rate of an uncertainty calculation when applied to many simulated independent measurements. An individual calculation is deemed successful if the uncertainty interval obtained covers the measurand used in the simulation. Several examples, including two from the GUM, illustrate the approach. Software implementing the method is described in detail. Simulation is a practical method that can provide useful insights into specific measurement problems when there is any doubt about the validity of GUM calculations.

A simulation approach is described for testing the performance of uncertainty calculations based on the Guide to the Expression of Uncertainty in Measurement (GUM) 1st edn (Sèvres, Paris: BIPM Joint Committee for Guides in Metrology). Performance is measured in terms of the long-run success rate of an uncertainty calculation when applied to many simulated independent measurements. An individual calculation is deemed successful if the uncertainty interval obtained covers the measurand used in the simulation. Several examples, including two from the GUM, illustrate the approach. Software implementing the method is described in detail. Simulation is a practical method that can provide useful insights into specific measurement problems when there is any doubt about the validity of GUM calculations.

This introduction to measurement uncertainty is intended for metrology professionals working in calibration laboratories and metrology institutes, as well as students in tertiary-level science and engineering programmes. The subject matter is presented with an emphasis on developing models of the physical measurement process. The level of mathematics and statistics used is basic and is typically covered by high school studies.

A measurement result is incomplete without a statement of its 'uncertainty' or 'margin of error'. But what does this statement actually tell us? By examining the practical meaning of probability, this book discusses what is meant by a '95 percent interval of measurement uncertainty', and how such an interval can be calculated. The book argues that the concept of an unknown 'target value' is essential if probability is to be used as a tool for evaluating measurement uncertainty. It uses statistical concepts, such as a conditional confidence interval, to present 'extended' classical methods for evaluating measurement uncertainty. The use of the Monte Carlo principle for the simulation of experiments is described. Useful for researchers and graduate students, the book also discusses other philosophies relating to the evaluation of measurement uncertainty. It employs clear notation and language to avoid the confusion that exists in this controversial field of science.