 |
    |
ToF Meets RGB: Novel Multi-Sensor Super-Resolution for Hybrid 3-D Endoscopy
Thomas Köhler1, 2, Sven Haase1, Sebastian Bauer1, Jakob Wasza1, Thomas Kilgus3, Lena Maier-Hein3, Hubertus Feußner4, and Joachim Hornegger1, 2
1Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany
thomas.koehler@fau.de
sven.haase@fau.de
2Erlangen Graduate School in Advanced Optical Technologies (SAOT), Germany
3Div. Medical and Biological Informatics Junior Group: Computer-Assisted Interventions, German Cancer Research Center (DKFZ) Heidelberg, Germany
4Minimally Invasive Therapy and Intervention, Technical University of Munich, Germany
Abstract. 3-D endoscopy is an evolving field of research with the intention to improve safety and efficiency of minimally invasive surgeries. Time-of-Flight (ToF) imaging allows to acquire range data in real-time and has been engineered into a 3-D endoscope in combination with an RGB sensor (640×480 px) as a hybrid imaging system, recently. However, the ToF sensor suffers from a low spatial resolution (64×48 px) and a poor signal-to-noise ratio. In this paper, we propose a novel multi-frame super-resolution framework to improve range images in a ToF/RGB multi-sensor setup. Our approach exploits high-resolution RGB data to estimate subpixel motion used as a cue for range super-resolution. The underlying non-parametric motion model based on optical flow makes the method applicable to endoscopic scenes with arbitrary endoscope movements. The proposed method was evaluated on synthetic and real images. Our approach improves the peak-signal-to-noise ratio by 1.6 dB and structural similarity by 0.02 compared to single-sensor super-resolution.
LNCS 8149, p. 139 ff.
lncs@springer.com
© Springer-Verlag Berlin Heidelberg 2013