 |
| Research article summary (published 30 Dec 2001): |
Empirical evaluation of a hybrid intelligent monitoring system using different measures of effectiveness.
Full Abstract
The validation of a software product is a fundamental part of its development, and focuses on an analysis of whether the software correctly resolves the problems it was designed to tackle. Traditional approaches to validation are based on a comparison of results with what is called a gold standard. Nevertheless, in certain domains, it is not always easy or even possible to establish such a standard. This is the case of intelligent systems that endeavour to simulate or emulate a model of expert behaviour. This article describes the validation of the intelligent system computer-aided foetal evaluator (CAFE), developed for intelligent monitoring of the antenatal condition based on data from the non-stress test (NST), and how this validation was accomplished through a methodology designed to resolve the problem of the validation of intelligent systems.System performance was compared to that of three obstetricians using 3450 min of cardiotocographic (CTG) records corresponding to 53 different patients. From these records different parameters were extracted and interpreted, and thus, the validation was carried out on a parameter-by-parameter basis using measurement techniques such as percentage agreement, the Kappa statistic or cluster analysis. Results showed that the system's agreement with the experts is, in general, similar to agreement between the experts themselves which, in turn, permits our system to be considered at least as skillful as our experts. Throughout our article, the results obtained are commented on with a view to demonstrating how the utilisation of different measures of the level of agreement existing between system and experts can assist not only in assessing the aptness of a system, but also in highlighting its weaknesses. This kind of assessment means that the system can be fine-tuned repeatedly to the point where the expected results are obtained.
Learn Faster Today Improve your study skills
Author information
Author/s: Guijarro-Berdiņas, Bertha (B); Alonso-Betanzos, Amparo (A);
Affiliation: Laboratory for Research and Development in Artificial Intelligence (LIDIA), Department of Computer Science, University of A Coruņa, Campus de Elviņa s/n, 15071, A Coruņa, Spain. cibertha(-atsign-)udc.es
Journal and publication information
Publication Type: Evaluation Studies; Journal Article; Research Support, Non-U.S. Gov't; Validation Studies
Journal: Artificial intelligence in medicine (Artif Intell Med), published in Netherlands. (Language: eng)
Reference: 2002-Jan; vol 24 (issue 1) : pp 71-96
Dates: Created 2002/01/07; Completed 2002/03/01; Revised 2006/11/15;
PMID: 11779686, status: MEDLINE (last retrieval date: 11/6/2008)
Sourced from the National Library of Medicine. Abstract text and other information may be subject to copyright.
External Links for this article (including full text providers, if available):
Click Electronic Full-text Provider Links to see options for finding the electronic full text links to this article. Note there may be a subscription or fee required for access to the full text. See our FAQ for information on finding FREE full text articles.
This article may also be located in paper journal collections available in many libraries. Use the Journal and Publication Information above to find the full article.
MeSH headings (categories)
This article was linked to the MESH Headings shown below.
|
Related articles
This article has not been indexed for related articles as yet, however you can still use the live related article search links below.
See a large map of 100+ related articles.