Normalisierung und Unmixing von hyperspektralen Aufnahmen für Hirntumorresektion unter Verwendung von Deep Learning

Identifying glioma margins is difficult. Advances in hyperspectral imaging (HSI) for fluorescence-guided surgery (FGS) have enhanced tumor detection based on their light emission spectra. However, computations are very sensitive to autofluorescence from endogenous fluorophores, and artifacts from the optical and topographic properties of the tissue. To overcome these challenges, measured spectra are normalized to account for artifacts and spectrally unmixed to isolate the protoporphyrin IX (PpIX) signal, which indicates malignant tissue. Existing methods are simplistic and based on phantoms and are unable to account for nonlinear effects such as multiple scattering, the dual photostates of PpIX, or the inhomogeneous optical properties of the tissue. These may include wavelength-dependent variations in absorption and scattering, which are also unmodeled. We propose a deep learning (DL)-based pipeline encompassing normalization and unmixing, which can fully account for the nonlinear effects and produce more accurate distributions of fluorophore abundances.

The architecture consists of a fully connected multi-layer perceptron to perform the normalization, followed by an encoder network for unmixing. A decoder network or a linear combination of known fluorophore spectra, weighted by the encoder outputs, can be used to compute a reconstruction loss for unsupervised learning on human data with no ground-truth PpIX abundance. A second encoder network with shared weights into which the pure fluorophore spectra are input ensures statistical independence of the computed abundances. This unsupervised approach was compared to the standard classical method and a fully supervised 1D convolutional neural network with two residual blocks and three fully connected layers.

On phantom and pig brain data with known PpIX concentration, the supervised model achieved coefficients of determination (R2) between the known and computed PpIX concentrations of 0.997 and 0.970, respectively. In contrast, the classical approach achieves only 0.93 and 0.82. The unsupervised approach's R2 values were 0.98 and 0.91, respectively. On human data, the unsupervised model gives qualitatively more realistic results, better removing bright spots of specular reflectance and greatly reducing the variance of the computed PpIX abundance over biopsies that should be relatively homogeneous.

These results show promise for using deep learning to improve the precision and accuracy of HSI FGS.