The RADAR group is interested in combining data and human expertise to create intelligent solutions for high stakes decisions. RADAR work with practitioners to produce intelligent ‘unified models’, that use both data and expertise as inputs, to support expert decision making in multiple application domains, including medical, legal, systems engineering, security, risk and safety.
Previous approaches to building models
To understand RADAR’s research agenda it is helpful to consider three paradigms for model building. First there was the knowledge engineering approach, based on human intuition, perhaps supplemented by data, or naive ‘expert systems’ models that captured human expertise. These provided the ‘knowledge engineer’ with a language, such as IF-THEN-ELSE rules, to model human knowledge.
Over time a shift occured towards the second paradigm, which took advantage of advances in computer technology. Statisticians and computer scientists were at the core of model building, constructing algorithms to learn ‘rules’ from data, and use these rules for prediction and other inference tasks. However, an effect of this paradigm was to sideline the role of the expert, whose judgement was replaced by the algorithms.
For too long model builders have failed to exploit expertise, simply because human sense and experience is not amenable to machine analysis. However, a purely machine learning based approach is not sufficient for all problems. This may be because there is insufficient data given the complexity of the domain. Alternatively, we may wish to predict the result of a change, for this we need a causal model.
The unified approach
RADAR’s research agenda is focused on the view that neither the first or second paradigms are sufficient to model the complex and uncertain situations faced by practitioners and researchers in many fields, including medicine and complex systems. There is a missing ingredient in both of these paradigms and this is the idea of a ‘unified model’, which is a method and means for combining expertise (in the form of expert supplied causal conjectures or hypotheses) and data using statistical and machine learning methods to produce intelligence.
Bayesian and causal network approaches form the basis of our unified modelling approach. They are revolutionising academic research and we aim to spearhead their penetration into industry and commerce through the immersive model of research implied by the 3rd paradigm. We therefore champion an interdisciplinary approach, built around computer science, statistics & machine learning and psychology, to help deliver this unified modelling approach.
- Security: RADAR have worked with DSTL, QinetiQ and Agena to apply the unified modelling approach to the challenge of counter terrorism and systems procurement and evaluation. Both DSTL and QinetiQ evaluate the reliability of land systems, such as armoured vehicles, using a unique blend of expert judgment based on decades of engineering experience and statistical data. The aim here is to identify failure modes as early on in the procurement cycle as possible to avoid expensive testing and also to help identify the best suppliers. DSTL have used the unified modelling approach to reason about the reliability and economic efficacy of sensor networks at transport and cargo hubs.
- Systems: RADAR have created innovative extensions to their earlier highly-cited work on software (design) defect prediction using Bayesian Networks. In particular, our work has led to radical improvements in the accuracy of software defect prediction and has led to the ability to apply such predictions in arbitrary software environments. Numerous companies including Philips, Motorola, QinetiQ and Israel Aircraft Industries have used the models and tools resulting from this work. For example, Philips reported 95% accuracy in software defect prediction, giving them greater confidence in decisions for testing and release of components in the critical area of embedded systems in consumer electronics. A similar approach has been applied by Motorola on their telecoms projects with an average saving of $5m per project.
- Medicine: In 2008-2009 RADAR was funded by EPSRC to run a research cluster in the area of improved medical decision-making as part of the UK government programme for the Digital Economy; the group has close ties with medics and lawyers both within and beyond Queen Mary. This area of research is likely to be a major focus for the group in the next few years.
- Law: in 2008-2009 Professor Fenton and Professor Neil acted as expert witnesses on two major legal cases. The first involved a probabilistic risk analysis in the case of R vs Levi Bellfield at the Old Bailey (July 2007 – Feb 2008). This work focused on the uncertainty relating to the vehicle identification in the case of the Marsha McDonnell murder and the second highlighted a number of fallacies in the prosecution case and was used as the basis for the defence case. The second case involved applying Bayesian analysis/decision-making for the claimant on a medical negligence case against the NHS (July-August 2008). This involved using key elements of our unified modelling approach to capture the expert disagreements about the efficacy of the diagnostic and treatment procedures applied and assess the likely outcomes under these different hypotheses.
In addition to the above the RADAR group has applied the unified modelling approach to problems in finance, notably operational risk, insurance and model risk for high impact low
probability events, personalization, sports prediction and project management.
In 2007 the Smith Institute and the London Mathematical Society asked RADAR to produce the first in their series of Knowledge Transfer Reports on the subject of Bayesian Networks for Risk Assessment (download). RADAR is committed to active research through technology transfer and much of RADAR’s research work is done in collaboration with the spinout company Agena, as well as with partners in commerce and government.
Prof Martin Neil talks about operational risk in finance.