Convolutional neural networks (CNNs) have demonstrated a great potential in fault detection for a wide type of multilevel inverters. Despite the remarkable performance of CNNs, their interpretability remains a challenge. This is due to networks, which have complicated black boxes behaviors. Consequently, they present a substantial challenge for widespread adoption of different models in practical applications. Moreover, relying solely on accuracy is insufficient, especially in critical applications where maintaining trust and robustness is vital for protecting a system against potential damage. Therefore, this study implements a visual explanation method called gradient weighted class activation map (Grad-CAM) for fault detection of multilevel inverter. The Grad-CAM method can identify the model’s important features and interpret the detection of fault types. The proposed method was validated by both simulation and experimental results for three-level neutral-point clamped inverters, demonstrating that a reliable CNN achieved high classification accuracy and effectively identified fault types.