http://www.dirc.org.uk/  
 
 
   
Overview
Research
 

   Themes  
   Results

Sites
People
Publications
Events
Related Projects
   
 

Full title

Diverse, multi-legged dependability cases

Keywords

Multi-legged arguments, safety cases, diversity

Summary

Study of diversity, as a means of achieving dependability, has been an important theme in DIRC [link to diversity page]. In contrast, this work addresses the ways in which diversity can be used to improve the ways in which we assess dependability. The underlying intuition here is very similar to that which drives the application of diverse redundancy in other areas: if there is a possibility of failure in one means of achieving an objective, then using two (or more) different means to that objective may significantly reduce the overall risk of failure.

Rather informally, we take a dependability case to be: an argument based on assumptions and evidence that supports a dependability claim at a particular level of confidence. For example, the claim might concern the safety of a system or subsystem – e.g. ‘the pfd (probability of failure on demand) of this system is no worse than 10-3’. It would clearly be undesirable, and may even be dangerous, to accept such a claim when it is false (e.g. the pfd is greater than 10-3). We can think of such an event as a failure of the dependability case, and we would like the probability of this to be small. Equivalently we would like to have high confidence in the correctness of the claim.

Doubt about the correctness of a dependability claim arises from several sources:

• the strength/weakness/relevance of the evidence;
• the extent of our confidence/doubt in the truth of the assumptions;
• the extent of our confidence/doubt in the correctness of the reasoning in the argument.

One way of looking at the problem is to think that cases can be faulty, just as systems can be faulty. So can we protect against the effect of case faults, using techniques similar to those that have been developed for system fault tolerance? Multi-legged cases are an attempt to do this. The idea is to support a claim with two (or more) cases, in the hope that confidence in it will thereby be higher than it would be from either of the claims alone.

Although the idea is a simple one, and has been advocated in some standards [4] and regulations [5], it is only in DIRC that rigorous studies of its efficacy have begun. The key to our approach is to model confidence probabilistically [link to confidence page], so that we have a quantitative measure of effectiveness.

As might be expected from earlier work on system diversity, issues of dependence between cases are important here. So the effectiveness of a two-legged case will depend on the effectiveness of each individual case and on the dependence between these. We have shown that, just as system failures cannot be assumed to be independent, so case failures will not be independent. For example, we would not trust claims along the lines of ‘case A gave me 90% confidence in the claim that the pfd is less than 10-4, case B also gave me 90% confidence in the claim, therefore with the two (A, B) cases I can be 99% confident in the claim’ [1, 2]

So far this work has concentrated on studying an idealized case involving two legs: a leg based on a statistical argument from operational testing, and a leg based on formal verification. In spite of the simplicity of this example, it has thrown up some surprisingly counter-intuitive results [3]. For example, adding to an existing dependability case a second, entirely supportive, second case can sometimes result in less confidence in a claim than came from the original single case: i.e. extra ‘good news’ can sometimes reduce confidence. The explanation lies in unexpected subtleties in reasoning that can sometimes arise when there is dependence between assumption ‘doubts’.

It is early days for this research, and it is turning out to be surprisingly difficult – more so than similar modeling of systems fault tolerance. At present we can show that, not surprisingly, multiple cases can bring improvements in confidence in dependability claims. But finding out how much benefit they bring can involve difficult mathematical modeling, and – worse – there are possibilities for counter-intuitive surprises.

Papers

 

R. E. Bloomfield and B. Littlewood, "Multi-legged arguments: the impact of diversity upon confidence in dependability arguments," presented at International Conference on Dependable Systems and Networks (DSN), San Francisco, 2003.

R. E. Bloomfield and B. Littlewood, "On the use of diverse arguments to increase confidence in dependability claims" in Structure for Dependability: Computer-Based Systems from an Interdisciplinary Perspective", (editors) D. Besnard, C . Gacek and C. B. Jones, pp 254-268, Springer, 2006, ISBN 1-84628-110-5

B. Littlewood and D. Wright, “The use of multi-legged arguments to increase confidence in safety claims for software-based systems: a study based on a BBN analysis of an idealized example”

Other References

[1] R. E. Bloomfield and B. Littlewood, "Multi-legged arguments: the impact of diversity upon confidence in dependability arguments," presented at International Conference on Dependable Systems and Networks (DSN), San Francisco, 2003. http://doi.ieeecomputersociety.org/10.1109/DSN.2003.1209913

[2] R. E. Bloomfield and B. Littlewood, "On the use of diverse arguments to increase confidence in dependability claims" in Structure for Dependability: Computer-Based Systems from an Interdisciplinary Perspective", (editors) D. Besnard, C . Gacek and C. B. Jones, pp 254-268, Springer, 2006, ISBN 1-84628-110-5

[3] B. Littlewood and D. Wright, “The use of multi-legged arguments to increase confidence in safety claims for software-based systems: a study based on a BBN analysis of an idealized example”

[4] MoD, "The Procurement of Safety Critical Software in Defence Equipment," Ministry of Defence Def-Stan 00-55, Issue 2, August, 1997.

[5] CAP 670 Air Traffic Services Safety Requirements, Part B, Section 3, Systems Engineering, SW 01 Regulatory Objectives for Software Safety Assurance in ATS Equipment.

Author

Bev Littlewood (City)

 

 
Page Maintainer: webmaster@dirc.org.uk Credits      Project Members only Last Modified: 6 July, 2006