Challenge on Ultrasound Beamforming with Deep Learning : 2020 IUS

The Challenge on Ultrasound Beamforming with Deep Learning (CUBDL) was offered as a component of the 2020 IEEE International Ultrasonics Symposium. The community was challenged to obtain the best image quality under the fastest possible frame rates after a single plane wave transmission. We informed participants that ground truth images of the same structures obtained after multiple plane wave transmissions and delay-and-sum beamforming would be used for comparison to the submitted network results. In addition, evaluation would be performed by the challenge organizers on a closed test set consisting of channel data that was crowd sourced from multiple ultrasound groups around the world.

Four participants accepted our challenge and delivered networks that achieved the stated goals. We asked the first authors of each submission to self-identify their level of experience with beamforming and deep learning prior to participation in the challenge on a scale of 1 to 5, with 1 being novice and 5 being expert. We received responses ranging from 1 to 5.

The challenge winners were Sven Rothlübbers from Fraunhofer MEVIS, whose network had the least complexity, and Sobhan Goudarzi from Concordia University, whose network produced the best image quality. Both networks received equivalent scores, based on the guidelines and ranking structure outlined prior to participation. The runners up were Yaning Wang and Zehua Li, both from Johns Hopkins University. We congratulate the challenge winners and commend all contestants for their participation, particularly when considering the additional challenges introduced by our global pandemic and the wide range of prior experience.

In addition to comparing submissions with the same code and datasets; the tools, resources, and tasks developed and designed for the challenge are recommended as a benchmark standard for both beginners and experts in this research area going forward. More details about CUBDL-related resources are available on the challenge website: https://cubdl.jhu.edu/resources/ .