A generative vision model for stain imputation in multiplex immunofluorescence imaging

<p dir="ltr">Multiplex immunofluorescence (mIF) imaging plays a crucial role in studying multiple biomarkers and their interactions within the tumour microenvironment. However, acquiring mIF images that include all desired biomarkers is challenging, due to the need for specialised equipment and costly reagents, increasing technical complexity, expense, and time requirements. Stain imputation offers a promising solution by synthesising target biomarker images using generative models, thereby eliminating the need for additional staining procedures. Recent advancements in medical image generation have predominantly concentrated on radiological imaging, whereas investigations into stain imputation within histopathology images, particularly in modalities such as immunohistochemistry and mIF remain limited. Furthermore, existing deep learning-based stain imputation methods typically rely on fixed inputs or produce fixed outputs, limiting their flexibility in generating multiple biomarker images from diverse input biomarker images. Accordingly, we propose Stain Imputation in Multiplex Immunofluorescence Imaging (SIMIF), a generative adversarial network (GAN) based stain imputation framework for mIF imaging. SIMIF incorporates a random channel-wise masking (RCWM) block, enabling the model to accommodate flexible inputs. To further enhance flexibility and reliability, we propose Adaptive Stain Imputation with Multi-Input and Multi-Output (AdSI-MIMO), which features a multi-branch deep learning architecture capable of generating multiple outputs and incorporates an adaptive progressive masking strategy to accommodate the varying numbers of input biomarkers. Our study significantly improves the quality of generated activation biomarker images, and overcomes a key limitation of existing methods by eliminating the need for separate models per biomarker. We evaluated both models on two datasets, including a local dataset comprising mIF images from 257 melanoma patients and a public dataset of 55 urothelial carcinoma samples. SIMIF significantly outperformed CD8 and PD-L1 imputation performance when using limited biomarker images as input, demonstrating the e!ectiveness of the RCWM block in handling flexible input biomarkers. Specifically, SIMIF achieved a Pearson-r of 0.160 for CD8 and 0.563 for PD-L1 using DAPI and AF inputs, respectively. This effectively mitigates the mode collapse seen in MAXIM, where Pearson-r were near zero. Additionally, AdSI-MIMO achieved substantial improvements over state-of-the-art (SOTA) methods in imputing key T-cell and activation biomarkers such as CD8, PD-L1, and Ki67. Specifically, across both public and local datasets, AdSI-MIMO achieved a 18.4% improvement in Pearson Correlation Coefficient (Pearson-r) for CD8 imputation and a 48.1% improvement for PD-L1 imputation under various input biomarker configurations on the external test set.</p>