Abstract:
The rapid advance in high throughput digital scanners generates high resolution digital slides that contain rich information about morphological and functional characteristics of biological systems. This provides tremendous potential for understanding diseases and supporting diagnosis by automated histopathology image analysis. Meanwhile, 3D digital pathology is made possible through slicing tissues into serial thin sections and holds a significant potential to enhance digital pathology by reconstructing more accurate spatial structures. However, pathology image analysis still presents major challenges such as clumped objects with irregular shapes, complex local histology structure change across adjacent slides, and significant dissimilar structural appearance due to different staining methods.
In this dissertation work, we first present a deep learning-based region-boundary integration network for delineating overlapped steatosis droplets of liver biopsies. We propose an integrated approach with a region-based module to segment the foreground steatosis droplet, a boundary module to learn the perceptual boundary features for each overlapped steatosis region, and integration of the two modules to train a third deep neural network for dividing overlapped steatosis droplets. The resulting steatosis measures both at the pixel level and object level present strong correlation with pathologist annotations, radiology readouts and clinical data.
The detection and evaluation of viable tumor regions in hepatocellular carcinoma present an important clinical significance for assessing chemoradiotherapy response. We present a multi-resolution convolutional autoencoder based model HistoCAE for viable tumor segmentation with a customized reconstruction loss function, followed by a classification module to classify each image patch. The resulting patch-based prediction results are spatially combined to generate the final segmentation results. Our proposed model presents superior performance to other benchmark models with extensive experiments.
To register 2D serial sections from multiple stains (e.g., H&E and IHC), we propose a novel translation-based registration network CycGANRegNet using deep learning for serial whole slide images, which requires no prior deformation field information for training. We first generate synthetic IHC slides from H&E slides through a robust image synthesis algorithm. The synthetic IHC images and the real IHC images are then registered through a Fully Convolutional Network with multi-scale deformable vector fields and a joint loss optimization for enhancing image alignment. We perform the registration at original image resolution with a patch-wide approach, thus tissue details at the highest resolution are retained in the results. The performance of our method is extensively compared with several state-of-the-art pathology image registration methods where CycGANRegNet outperforms both the state-of-the-art conventional and deep learning-based methods.
Finally, we develop HistoRegNet, an end-to-end unsupervised patch-based deep learning registration model to spatially align IHC histopathology images. The model consists of an affine and a deformable module that learns the Displacement Vector Field by both affine and deformable transformation optimization. The learned DVF is provided to a spatial transformer network that generates registered images. Experimental results demonstrate the superior performance of our proposed model to other methods, suggesting its promising potential for IHC histopathology image registration.
