EXTENDED ABSTRACT: In materials science, the shortage of labeled data presents a persistent challenge, particularly for the accurate semantic segmentation of metallographic microscopy images. This study addresses the issue through two complementary strategies: synthetic data generation and self-supervised learning (SSL). Accurate segmentation of microstructures, such as those found in carbon steel, is crucial but hindered by the limited availability of labeled data. To overcome this limitation, we employ Momentum Contrast (MoCo) for SSL to pretrain a ResNet model on unlabeled metallographic datasets. This pretrained model serves as the backbone for a U-Net architecture, which is subsequently fine-tuned on labeled images, resulting in enhanced segmentation accuracy compared to models initialized with ImageNet weights. In parallel, we propose a novel approach for synthetic microstructure generation using diffusion transformers (DiTs). By employing parameter-efficient fine-tuning (PEFT), we fine-tune specific components of the pre-trained DiTs, such as bias terms, class embeddings, and layer normalization, while keeping other parameters fixed. This approach not only reduces training time and storage requirements by 16.33% and 16.73%, respectively, but also improves image quality and diversity. Additionally, our overlap crop method with a 0.5 overlap ratio yields a 25.89% improvement in Fréchet Inception Distance (FID) over random crop techniques, demonstrating the efficacy of our synthetic data generation. These high-quality images were evaluated against microstructural descriptors, outperforming established benchmarks.
Keywords: Synthetic Microstructure; Self supervised Learning; semantic segmentation