Project Overview
Technology is revolutionizing medicine. New scanners enable doctors to “look into the patient’s body” and study their anatomy and physiology without the need of a scalpel. At an amazing speed new scanning technologies emerge, providing an ever growing and increasingly varied look into medical conditions. Today, we cannot “only” look at the bones within a body, but we can also examine soft tissue, blood flow, activation networks in the brain, and many more aspects of anatomy and physiology. The increased amount and complexity of the acquired medical imaging data leads to new challenges in knowledge extraction and decision making.
In order to optimally exploit this new wealth of information, it is crucial that all this imaging data is successfully linked to the medical condition of the patient. In many cases, this is challenging, for example, when diagnosing early-stage cancer or mental disorders. Analogous to biomarkers, which are molecular structures that are used to identify medical conditions, imaging biomarkers are information structures in medical images that can help with diagnostics and treatment planning, formulated in terms of features that can be computed from the imaging data. Imaging biomarker discovery is a highly challenging task and traditionally only a single hypothesis (for a new biomarker) is examined at a time.
This makes it impossible to explore a large number as well as more complex imaging biomarkers across multi-aspect data. In the VIDI project, we propose to research and advance visual data science to improve imaging biomarker discovery through the visual integration of multi-aspect medical data with a new visualization-enabled hypothesis management framework.
We aim to reduce the time it takes to discover new imaging biomarkers by studying structured sets of hypotheses, to be examined at the same time, through the integration of computational approaches and interactive visual analysis techniques. Another related goal is to enable the discovery of more complex imaging biomarkers, across multiple modalities, that potentially are able to more accurately characterize diseases. This should lead to a new form of designing innovative and effective imaging protocols and to the discovery of new imaging biomarkers, improving suboptimal imaging protocols and thus also reducing scanning costs. Our project is a truly interdisciplinary research effort, bringing visualization research and imaging research together in one project, and this is perfectly suited for the novel Centre for Medical Imaging and Visualization that has been established in Bergen, Norway.
VIDI Approach
To achieve these goals we have divided the VIDI project into seven discrete workpackages (WP), which can be executed, to some extent, in parallel:
WP1: Hypothesis management
Research and design of methodologies necessary for structuring, representation, exploration, and analysis of hypothesis sets. Develop visual language for data interactions and methods for linking spatial with non-spatial data.
WP2: Data & Features
Exploration of medical image feature extraction and visualisation, definition of UX for selection and refinement of data features to extend into additional dimensions.
WP3: Hypotheses Scoring
Development of methods for interactive visual ranking and analyses of user hypotheses sets. Exploration of methods to provide user with evaluation preview of investigated hypotheses, linked to hypotheses visualisation and rankings.
WP4: Optimized Imaging
Evaluation of existing imaging protocols and development new imaging techniques. Investigation of imaging process to guard against suboptimal image acquisitions.
WP5: Integration
Integrate solutions from work packages 1-4.
WP6: Evalulation
Evaluate new hypothesis methods in context of three target applications: 1) gynecologic cancer; 2) neuroinflammation in MS; and 3) neurodegenerative disorders.
WP7: Management & Dissemination
Coordination between the involved partners, planning and reporting, and dissemination.
VIDI Project Team
PI: Helwig Hauser
Co-PIs: Stefan Bruckner and Renate Grüner, MMIV
Associated researcher: Noeska Smit
PhD students: Laura Garrison and Fourough Gharbalchi
This project is funded by the Bergen Research Foundation (BFS) and the University of Bergen.
Publications
2024
![[PDF]](https://vis.uib.no/wp-content/plugins/papercite/img/pdf.png)
[Bibtex] @article{pokojna2024language,
title={The Language of Infographics: Toward Understanding Conceptual Metaphor Use in Scientific Storytelling},
author={Pokojn{\'a}, Hana and Isenberg, Tobias and Bruckner, Stefan and Kozl{\'i}kov{\'a}, Barbora and Garrison, Laura},
journal={IEEE Transactions on Visualization and Computer Graphics},
year={2024},
month={Oct},
publisher={IEEE},
abstract={We apply an approach from cognitive linguistics by mapping Conceptual Metaphor Theory (CMT) to the visualization domain to address patterns of visual conceptual metaphors that are often used in science infographics. Metaphors play an essential part in visual communication and are frequently employed to explain complex concepts. However, their use is often based on intuition, rather than following a formal process. At present, we lack tools and language for understanding and describing metaphor use in visualization to the extent where taxonomy and grammar could guide the creation of visual components, e.g., infographics. Our classification of the visual conceptual mappings within scientific representations is based on the breakdown of visual components in existing scientific infographics. We demonstrate the development of this mapping through a detailed analysis of data collected from four domains (biomedicine, climate, space, and anthropology) that represent a diverse range of visual conceptual metaphors used in the visual communication of science. This work allows us to identify patterns of visual conceptual metaphor use within the domains, resolve ambiguities about why specific conceptual metaphors are used, and develop a better overall understanding of visual metaphor use in scientific infographics. Our analysis shows that ontological and orientational conceptual metaphors are the most widely applied to translate complex scientific concepts. To support our findings we developed a visual exploratory tool based on the collected database that places the individual infographics on a spatio-temporal scale and illustrates the breakdown of visual conceptual metaphors.},
pdf = {pdfs/garrisonVIS24.pdf},
images = {images/garrisonVIS24.png},
thumbnails = {images/garrisonVIS24thumb.png},
project = {VIDI},
git={https://osf.io/8xrjm/}
}![[DOI]](https://vis.uib.no/wp-content/plugins/papercite/img/external.png){kind=small}
[Bibtex] @inproceedings{zimanVCBM2024mobaDash,
booktitle = {Eurographics Workshop on Visual Computing for Biology and Medicine},
editor = {Garrison, Laura and Jönsson, Daniel},
title = {{The MoBa Pregnancy and Child Development Dashboard: A Design Study}},
author = {Ziman, Roxanne and Budich, Beatrice and Vaudel, Marc and Garrison, Laura},
year = {2024},
month = {September},
publisher = {The Eurographics Association},
ISSN = {2070-5786},
ISBN = {978-3-03868-244-8},
DOI = {10.2312/vcbm.20241194},
abstract = {Visual analytics dashboards enable exploration of complex medical and genetic data to uncover underlying patterns and possible relationships between conditions and outcomes. In this interdisciplinary design study, we present a characterization of the domain and expert tasks for the exploratory analysis for a rare maternal disease in the context of the longitudinal Norwegian Mother, Father, and Child (MoBa) Cohort Study. We furthermore present a novel prototype dashboard, developed through an iterative design process and using the Python-based Streamlit App [TTK18] and Vega-Altair [VGH*18] visualization library, to allow domain experts (e.g., bioinformaticians, clinicians, statisticians) to explore possible correlations between women's health during pregnancy and child development outcomes. In conclusion, we reflect on several challenges and research opportunities for not only furthering this approach, but in visualization more broadly for large, complex, and sensitive patient datasets to support clinical research.},
pdf = {pdfs/zimanVCBM24.pdf},
images = {images/zimanVCBM24.png},
thumbnails = {images/zimanVCBM24thumb.png},
project = {VIDI},
git={https://osf.io/u6kdm/}
}@MISC{zhang2024ManhattanWheel,
booktitle = {Eurographics Workshop on Visual Computing for Biology and Medicine (Posters)},
title = {{The Manhattan Wheel: A Radial Visualization Story for Genome-wide Association Study Data}},
author = {Zhang, Ke Er and Vaudel, Marc and Garrison, Laura A.},
year = {2024},
howpublished = {Poster presented at VCBM 2024.},
publisher = {The Eurographics Association},
abstract = {Genome-wide association studies (GWAS) are critical to identifying genetic variations associated with a particular trait or disease. It is important to cultivate an awareness of GWAS in the general public as members of this group are key participants of these studies. However, low genetic data literacy and trust in the sharing of genetic data pose challenges to learning and engaging with GWAS concepts. In this design study, we explore design strategies for the public communication of GWAS data. As part of this study, we present an interactive visual prototype that explores the use of narrative structure, linked visualizations through scrollytelling, and plain language to onboard and communicate genetic concepts to a GWAS-naive audience.},
pdf = {pdfs/KEZHANG_VCBM2024_Poster_ManhattanWheel.pdf},
images = {images/zhangManhattanWheel.png},
thumbnails = {images/zhangManhattanWheelthumb.png},
project = {VIDI},
git={https://github.com/amykzhang/manhattan-wheel}
}@inproceedings{correll2024bodydata,
abstract = {With changing attitudes around knowledge, medicine, art, and technology, the human body has become a source of information and, ultimately, shareable and analyzable data. Centuries of illustrations and visualizations of the body occur within particular historical, social, and political contexts. These contexts are enmeshed in different so-called data cultures: ways that data, knowledge, and information are conceptualized and collected, structured and shared. In this work, we explore how information about the body was collected as well as the circulation, impact, and persuasive force of the resulting images. We show how mindfulness of data cultural influences remain crucial for today's designers, researchers, and consumers of visualizations. We conclude with a call for the field to reflect on how visualizations are not timeless and contextless mirrors on objective data, but as much a product of our time and place as the visualizations of the past.},
author = {Correll, Michael and Garrison, Laura A.},
booktitle = {arXiv, Proc CHI24},
doi = {10.48550/arXiv.2402.05014},
publisher = {arXiv},
title = {When the Body Became Data: Historical Data Cultures and Anatomical Illustration},
year = {2024},
month = {Feb},
pdf = {pdfs/garrisonCHI24.pdf},
images = {images/garrisonCHI24.png},
thumbnails = {images/garrisonCHI24.png},
project = {VIDI}
}2023
[Bibtex] @MISC {balaka2023MoBaExplorer,
booktitle = {Eurographics Workshop on Visual Computing for Biology and Medicine (Posters)},
editor = {Garrison, Laura and Linares, Mathieu},
title = {{MoBa Explorer: Enabling the navigation of data from the Norwegian Mother, Father, and Child cohort study (MoBa)}},
author = {Balaka, Hanna and Garrison, Laura A. and Valen, Ragnhild and Vaudel, Marc},
year = {2023},
howpublished = {Poster presented at VCBM 2023.},
publisher = {The Eurographics Association},
abstract = {Studies in public health have generated large amounts of data helping researchers to better understand human diseases and improve patient care. The Norwegian Mother, Father and Child Cohort Study (MoBa) has collected information about pregnancy
and childhood to better understand this crucial time of life. However, the volume of the data and its sensitive nature make its
dissemination and examination challenging. We present a work-in-progress design study and accompanying web application,
the MoBa Explorer, which presents aggregated MoBa study data genotypes and phenotypes. Our research explores how to
serve two distinct purposes in one application: (1) allow researchers to interactively explore MoBa data to identify variables of
interest for further study and (2) provide MoBa study details to an interested general public.},
pdf = {pdfs/balaka2023MoBaExplorer.pdf},
images = {images/balaka2023MoBaExplorer.png},
thumbnails = {images/balaka2023MoBaExplorer-thumb.png},
project = {VIDI}
}@inproceedings {budich2023AIstories,
booktitle = {Eurographics Workshop on Visual Computing for Biology and Medicine},
editor = {Hansen, Christian and Procter, James and Renata G. Raidou and Jönsson, Daniel and Höllt, Thomas},
title = {{Reflections on AI-Assisted Character Design for Data-Driven Medical Stories}},
author = {Budich, Beatrice and Garrison, Laura A. and Preim, Bernhard and Meuschke, Monique},
year = {2023},
publisher = {The Eurographics Association},
ISSN = {2070-5786},
ISBN = {978-3-03868-216-5},
DOI = {10.2312/vcbm.20231216},
abstract = {Data-driven storytelling has experienced significant growth in recent years to become a common practice in various application areas, including healthcare. Within the realm of medical narratives, characters play a pivotal role in connecting audiences with data and conveying complex medical information in an engaging manner that may influence positive behavioral and lifestyle changes on the part of the viewer. However, the process of designing characters that are both informative and engaging remains a challenge. In this paper, we propose an AI-assisted pipeline for character design in the context of data-driven medical stories. Our iterative pipeline blends design sensibilities with automation to reduce the time and artistic expertise needed to develop characters reflective of the underlying data, even when that data is time-oriented as in a cohort study.},
pdf = {pdfs/budichAIstories.pdf},
images = {images/budichAIstories.png},
thumbnails = {images/budichAIstories-thumb.png},
project = {VIDI}
}@article{mittenentzwei2023heros,
journal = {Computer Graphics Forum},
title = {{Do Disease Stories need a Hero? Effects of Human Protagonists on a Narrative Visualization about Cerebral Small Vessel Disease}},
author = {Mittenentzwei, Sarah and Weiß, Veronika and Schreiber, Stefanie and Garrison, Laura A. and Bruckner, Stefan and Pfister, Malte and Preim, Bernhard and Meuschke, Monique},
year = {2023},
publisher = {The Eurographics Association and John Wiley & Sons Ltd.},
ISSN = {1467-8659},
DOI = {10.1111/cgf.14817},
abstract = {Authors use various media formats to convey disease information to a broad audience, from articles and videos to interviews or documentaries. These media often include human characters, such as patients or treating physicians, who are involved with the disease. While artistic media, such as hand-crafted illustrations and animations are used for health communication in many cases, our goal is to focus on data-driven visualizations. Over the last decade, narrative visualization has experienced increasing prominence, employing storytelling techniques to present data in an understandable way. Similar to classic storytelling formats, narrative medical visualizations may also take a human character-centered design approach. However, the impact of this form of data communication on the user is largely unexplored. This study investigates the protagonist's influence on user experience in terms of engagement, identification, self-referencing, emotional response, perceived credibility, and time spent in the story. Our experimental setup utilizes a character-driven story structure for disease stories derived from Joseph Campbell's Hero's Journey. Using this structure, we generated three conditions for a cerebral small vessel disease story that vary by their protagonist: (1) a patient, (2) a physician, and (3) a base condition with no human protagonist. These story variants formed the basis for our hypotheses on the effect of a human protagonist in disease stories, which we evaluated in an online study with 30 participants. Our findings indicate that a human protagonist exerts various influences on the story perception and that these also vary depending on the type of protagonist.},
pdf = {pdfs/garrison-diseasestories.pdf},
images = {images/garrison-diseasestories.png},
thumbnails = {images/garrison-diseasestories-thumb.png},
project = {VIDI}
}@incollection{garrison2023narrativemedvisbook,
title = {Current Approaches in Narrative Medical Visualization},
author = {Garrison, Laura Ann and Meuschke, Monique and Preim, Bernhard and Bruckner, Stefan},
year = 2023,
booktitle = {Approaches for Science Illustration and Communication},
publisher = {Springer},
address = {Gewerbestrasse 11, 6330 Cham, Switzerland},
pages = {},
note = {in publication},
editor = {Mark Roughley},
chapter = 4,
pdf = {pdfs/garrison2023narrativemedvisbook.pdf},
images = {images/garrison2023narrativemedvisbook.png},
thumbnails = {images/garrison2023narrativemedvisbook-thumb.png},
project = {VIDI}
}![[VID]](https://vis.uib.no/wp-content/papercite-data/images/video.png)
![[YT]](https://vis.uib.no/wp-content/papercite-data/images/youtube.png)