Research
Medical images often represent complex and variable structures in the human body that are not easily modeled. Furthermore, they suffer from a range of imperfections due to the applied acquisition modalities. Today’s methods, directed at the automated recognition of certain structures in images, are applicable only over a limited range of standard situations and/or reach suboptimal results. To overcome this problem we developed over the past several years an image interpretation system based on the paradigm of multi-agents, an approach also taken by several other research institutes around the world. However, where other researchers applied this methodology to low-level image segmentation, agents in our system aim at the detection of objects at the human interpretation level. Agents are (segmentation) experts on a particular image object and collectively aim at a consistent image interpretation through communication, collaboration, and the resolution of conflicts. We applied this system to Intravascular Ultrasound (IVUS) images and found that image segmentation algorithms, when controlled by agents in a multi-agent setting, give more reliable results and can be applied to a larger range of problems. Moreover, reasoning over image content and segmentation results also resulted in solving problems that were not solvable otherwise.
We found that the knowledge model of the individual agent can be designed such that up to 95% of its about 500 knowledge rules is domain independent. This concerns general knowledge about image processing, conflict resolution, communication, and utilities. The remaining 5% is application dependent and describes (1) high-level common sense knowledge about the application area and (2) very specific knowledge concerning which segmentation algorithm and parameters to use under various conditions and how to interpret the consequent (lack of) results. This second kind of application dependent knowledge is very difficult to specify, requires one to foresee every possible situation an agent may encounter and does not easily transfer to another image interpretation domain. Effectively it causes a "knowledge gap" that needs to be bridged for each new application area. This hampers the general applicability of this agent-based approach, which is otherwise very promising and highly innovative. To create a truly generic system we propose to let software agents bridge this "knowledge gap" automatically by learning such knowledge from examples and user-interactions, rather than pre-programming it in an inflexible manner. Probabilistic domain models, learned from pre-segmented images, are to be used by agents to select an appropriate image processing strategy and to reflect on the consequent (lack of) results.
Further, image processing algorithms will be extended with macro-operators that can learn the appropriate relation between agent directed image processing strategies and particular image processing algorithms and parameter settings. Our approach is already unique in the way that agents have control over image processing tasks, the way agents can negotiate over the most likely image content, and because of the highly domain independent general agent image interpretation knowledge model. The proposed combination of this knowledge model with agent learning from examples and user-interactions is also unique, as is the way that agents are allowed to have explicit control over the various learning processes. This highly original combination of multi-agent image interpretation and learning techniques, and the combination of explicit and implicit knowledge, as proposed here, may very well lead to a significant breakthrough in the field of knowledge-guided image processing.
Our ultimate goal with this project is to develop a general and highly innovative adaptive learning multiagent image interpretation system, which is more flexible and easier to adapt to changes in patient context, expert preferences, or imaging devices. To test the general applicability and effectivity of the resulting system, we will apply it to both IntraVascular Ultrasound (IVUS) images and Computed Tomography Angiography (CTA) images. The IVUS image segmentation application will allow the comparison of the results and effectiveness of this new adaptive learning system with results as obtained in earlier experiments. The CTA application will serve to demonstrate the general applicability of the system.
Utilization
This project is of interest to companies that are dealing with difficult (medical) image segmentation problems, because the system as proposed here may provide those companies with an automated and superior intelligent image segmentation solution. Moreover, an adaptive learning multi-agent system will be more flexible and easier to accommodate to changes in application context, expert preferences, or imaging instrumentation, by the use of both low-level training / optimization and high-level rules. This supports the development of new applications in a shorter time. It also simplifies the maintenance and support of a commercial version of the application software.
The first application area for this project will be IVUS, which provides real-time high-resolution tomographic images of the coronary vessels. These images are difficult to interpret, however, and thus highly sensitive to intra- and inter-observer variability. Moreover, IVUS image datasets are large (500-1000 images per set), which makes manual segmentation very time consuming. Therefore an automatic segmentation method is needed. In addition, reproducible and accurate quantitative measurements of long runs of IVUS images are needed in clinical trials of new cardiovascular treatments (drugs, stents, progressive atherosclerosis, etc.) to have the outcome of these trials available sooner, at a lower cost and at the same time with higher accuracy. This will also result in smaller patient cohorts to be studied. Furthermore IVUS is an important diagnostic tool. A system as proposed here may assist in the broader acceptance of IVUS, both for quantitative research and clinical usage.
The second application area for this project is Computed Tomography Angiography (CTA). It is one of the most recent developments in the field of cardiac imaging and allows visualization of the coronary arteries and the detection of narrowings (stenoses). Since the CTA procedure is non-invasive, provides 3D information, allows short acquisition time (about 20 sec, one breath hold) and has radiation exposure close to conventional X-ray angiograms, it is to be expected that approximately many of the diagnostic invasive X-ray angiograms will be replaced by non-invasive CTA angiograms. Applications include the non-invasive coronary imaging in the pre-clinical phase, which may result in an earlier detection of the disease and earlier treatment (invasive, by drugs, or lifestyle changes, etc.), potentially preventing progression to fatal or nonfatal myocardial infarction. The size of the CTA datasets will increase with better resolution resulting in outstanding opportunities for analytical software. At present no high quality and automated quantification solutions are available for this imaging modality.
The system will be developed and evaluated in close collaboration with several end-users (cardiologists / radiologists specialized in IVUS and CTA) who are recognized experts in the field. Also, the potential direct users (candidates for commercialization of the resulting methods) have declared their support with consultancy and software and, in case of commercialization, royalty payments conform STW rules. Medis medical imaging systems bv, Leiden, is interested in further applying this new technology into existing and new products. MeVis GmbH in Bremen is interested to apply this technology in their MeVisLab medical image-processing library. We have been collaborating with MeVis now for a few years and have their MeVisLab software as a general image-processing library also available and in use at the LKEB. Bio-Imaging Technologies BV is a contract research organization that provides image analysis services throughout the world to the pharmaceutical and medical devices industry. Therefore, Bio-Imaging has expressed great interest in using novel analytical software packages for application in their trials. Toshiba Medical Systems Corporation, one of the four leading companies worldwide in the field of medical imaging and particularly in multi-slice computed tomography, is very interested to learn about the results of this research project for possible future incorporation into existing and new products. As a potential direct user, they will act as a "soundboard" in the development process throughout the project.