Abnormal discharge is a dangerous power transformer fault, which can lead to serious safety hazards if not detected in time. A method for identifying abnormal discharge in transformer is proposed by collecting the voiceprint signal in the transformer box through the contact voice pickup. And a feature extraction method and a deep neural network structure are proposed to achieve efficient identification of abnormal transformer discharge. Firstly, a two-dimensional voiceprint feature extraction method combining Mel frequency and key frequency is designed. Then, a hybrid two-dimensional feature recognition model based on convolutional neural network and Transformer network is used to accurately identify the abnormal discharge voiceprint signal while ensuring the speed. Finally, according to the experimental analysis of the discharge data collected from the 110 kV three-phase three-winding transformer in the no-load voltage regulation process, the recognition speed of the proposed method increases by 0.19 seconds per sample, and the accuracy increases by 4.5% compared with ResNet50.