Confidence in dependability cases
dependability case; confidence; ALARP; ACARP; SIL; IEC61508
We define a dependability case as a convincing and valid argument that a system is adequately dependable in a given application and environment. In DIRC we have started to formalise this notion of a 'case'. To make a case we need to reason, based on assumptions and evidence, that a dependability claim is true at some level of confidence. The aim is to obtain a more rigorous and formal understanding of the role of confidence in dependability arguments. Just as we find diversity everywhere so do we see issues of confidence. It arises from a realization that the uncertainty involved in dependability cases - even safety cases for critical systems - is always present and that it is not dealt with explicitly yet many costly assurance activities are undertaken to increase confidence in the system (rather than change say its reliability). For example, it is common to find requirements of the kind 'this system needs to be SIL x', or 'the probability of failure on demand (pfd) of this system should be less than 10-3'. In fact, one can never know with certainty that such requirements have been met: there are uncertainties arising from, e.g., weaknesses of evidence, possible fallibility in reasoning, etc. What is needed is an ability to say 'this argument supports this claim at this level of confidence'.
We have illustrated these issues by examining and mathematically modeling a number of judgements about the safety integrity level (SIL) of a system. We have explored how the confidence in these judgments effects the overall judgement of a safety related failure rate and have illustrated this with an example of SIL membership showing how our confidence in our claim should impact the SIL claim figure. This justifies the use of ACARP as a subset of ALARP and the use of confidence building verification activities. We also suggest some heuristics for regulators and assessors and comment on the interpretation of IEC 61508. The results of the DIRC confidence work have already been integrated into the reissue of Def Stan 00-56. We intend to extend this work and to build some initial experiments we have done that elicit from experts confidence in judgments about safety integrity levels.
Purely numerical approaches to this problem, for example based on Bayesian Belief Nets (BBNs) can be notoriously difficult to justify, and even grossly misleading, as we have shown in [BW05]. Instead we are developing a Bayesian method that retains and algebraically manipulates the complete probabilistic dependability case. This is made tractable by the use of conservative approximations to the parameters of interest and to the structure itself: further work is needed to investigate and justify such conservatisms. In particular, a better understanding is needed of the psychological processes that inform expert judgment, as it seems inevitable that this will continue to be an important component of dependability cases.
The key to this work lies in the use of confidence (in a dependability claim) as a measure of efficacy of a case. Thus an important application of these ideas is to model multi-legged (i.e. diverse) cases in a probabilistic way, deriving quantitative measures of the effectiveness of diverse evidence and of diverse reasoning.We have studied such an approach aiming in the first place at a conceptual view: not producing precise numerical outputs, but instead clarity about the role of each component of a case, how critical they are to the overall confidence in a case and how effectively additional, diverse evidence mitigates uncertainty [BL03].
Not only do we want valid cases, we also want them to be convincing. Invalid yet convincing cases could lead to costly and dangerous failure; valid but unconvincing cases could lead to rejection of systems of potential social usefulness. Making a trustworthy case trusted is a social process and can involve subtle and non-obvious psychological issues. This important trade-off between trust and trustworthiness has recently been addressed in the context of E-voting, and the difficult problem of public belief in complex security arguments [RR05].
[BL03] Bloomfield, R.E and Littlewood, B., "Multi-legged Arguments: The Impact of Diversity upon Confidence in Dependability Arguments", Proceedings DSN 2003, pp. 25-34, IEEE Computer Society, ISBN 0-7695-1952-0, 2003.
[BW05] B Littlewood, D Wright “The Use of Multi-legged Arguments to Increase Confidence in Safety-Claims for Software-Based-Systems: a Study Based on a BBN Analysis of an Idealised Example”, CSR Technical Report.
[RR05] Randell, B., Ryan, P.Y.A Voting Technologies and Trust, CS-TR: 911. School of Computing Science, University of Newcastle, Jun 2005
Robin Bloomfield (City)
|Page Maintainer: email@example.com||Credits||Project Members only||Last Modified: 10 August, 2005|