Hybrid Image Fusion of Multispectral and SAR Data Using Contourlet Transform and Attention-Based CNN for Cloud and Shadow Problems in Southern Coast of Peninsular Malaysia Syaifulnizam Abd Manaf, Norwati Mustapha, Nor Azura Husin, Raihani Mohamed, Siti Nur Aliaa Roslan
Faculty of Computer Science and Information Technology
Universiti Putra Malaysia
43400 UPM Serdang
Selangor
Malaysia
Abstract
Cloud and shadow contamination remain persistent challenges in optical remote sensing, severely limiting the usability of satellite imagery for time sensitive and large scale applications. This study proposes a hybrid deep learning based image fusion framework to enhance the quality of multispectral MS imagery affected by atmospheric disturbances by integrating complementary synthetic aperture radar SAR data. The fusion architecture leverages the multi resolution directional representation of the Contourlet Transform with the adaptive feature extraction capability of an attention based convolutional neural network CNN. This combination is designed to enhance spatial detail retention and spectral consistency, particularly under cloudy and low illumination conditions. Fusion quality is quantitatively assessed using eight widely adopted performance metrics: Correlation Coefficient CC, Universal Image Quality Index UIQI, Relative Bias Bias, Entropy ENT, Root Mean Square Error RMSE, Erreur Relative Globale Adimensionnelle de Synthese ERGAS, Structural Similarity Index SSIM, and Difference in Variance DIV. The proposed hybrid method is benchmarked against several traditional fusion techniques including Brovey Transform, Gram Schmidt Fusion, Intensity Hue Saturation IHS Transform, Principal Component Analysis PCA, Nearest Neighbor Diffusion NND, and Curvelet Transform Fusion. Experiments conducted on real satellite datasets focusing on the Pontian district along the southern coast of Peninsular Malaysia demonstrate that the proposed Contourlet plus Attention CNN model delivers superior performance in preserving spatial and spectral features while significantly reducing cloud and shadow effects. These findings underscore the potential of hybrid deep learning models for robust multi source image fusion in operational remote sensing, particularly in cloud prone coastal scenario.
