The potential safety risks and economic losses caused by open-neutral and open-phase faults in low-voltage distribution networks have been longstanding challenges for power grid companies. With the popularization of intelligent detection equipment in power grids, fault detection can now be performed using voltage and sequence current data collected by smart meters on the low-voltage side. This paper first established a hybrid model, TNN-BL, based on transformer neural network (TNN) and bi-directional long short-term memory (Bi-LSTM). Secondly, by selecting appropriate loss functions and regularization functions, the model was refined to further improve its detection performance. Finally, the model performance was validated using a dataset from the China Southern Power Grid. Experimental results showed that the proposed method had a more effective feature extraction capability, higher detection accuracy and stronger robustness compared to other fault detection methods.