Based on ultrasound signals and their correlations with previously acquired MRI, a machine‐learning algorithm creates synthetic MR images at frame rates up to 100 per second.


Frank Preiswerk1, Matthew Toews2, Cheng-Chieh Cheng1, Jr-yuan George Chiou1, Chang-Sheng Mei3, Lena F. Schaefer1, W. Scott Hoge1, Benjamin M. Schwartz4, Lawrence P. Panych1, and Bruno Madore1

Estimated cost



Stable release, Apache License v2.0

Proposed solution generates synthetic MRI based on previously acquired MRI images, ultrasound signals and learned correlations. The method has been testes with volunteers out of the scanner room. These out‐of‐bore synthetic images might help guide therapies that could not be performed within the confines of an MRI scanner, or toward registering images subsequently acquired from different modalities and scanners.

The present work employed simple and relatively low‐cost ultrasound hardware. The single‐element transducer was small enough to easily fit below or within the openings of a multi‐element MR receiver coil, it did not need to be located or tilted in any specific way. This contrasts with other existing setups that combine ultrasound and MR acquisitions, based on full‐size imaging transducers, typically handheld or affixed to a holder.

A main realization at the basis of the present work is that, although a small transducer is insufficient to generate two‐dimensional (2D) or three‐dimensional (3D) spatially resolved images, it may not need to because MRI is spatially resolved and correlations exist between the two signal types.

The ultra sound probe employed here contacts only a small area of a subject’s abdomen, out of the way of any interventionalist and any potential sterile area. The main advantages of employing ultrasound signals instead of navigator echoes are:

  • ultrasound signal acquisitions happen in parallel and simultaneously with the MRI scan and do not reduce the time available for MR image data acquisition, unlike most navigated schemes;
  • ultrasound signals are available outside and inside the MR bore, leading to the intriguing possibility of synthesizing real‐time MR images from patients who are not even in the scanner.

A Bayesian algorithm trained to detect periods of unusual activity (eg, coughing) handles the flow of hybrid data. The system enables MRI contrast to be achieved, even for subjects outside the MR bore, and thus may have interesting applications for the field of image‐guided therapy.



1Department of Radiology, Brigham and Women’s Hospital, Harvard Medical School, Boston, Massachusetts, USA;

2The Laboratory for Imagery, Vision and Artificial Intelligence, École de Technologie Supérieure, Montréal, QC, Canada;

3Department of Physics, Soochow University, Taipei, Taiwan;

4Google Inc, New York, New York, USA.