In order to realize the development requirements of miniaturization and lightweight of electric locomotives, the overall volume and weight of transformer are reduced by increasing the working frequency of transformer. As the key component of high-frequency transformer, the iron core can accurately calculate the core loss, which plays an important role in the design and optimization of medium and high-frequency transformer. Aiming at the characteristics of high-frequency transformer cores that usually work under non-sinusoidal excitation such as square wave or pulse width modulation (PWM) wave, the traditional Steinmetz loss calculation model is optimized and improved. The Steinmetz improved loss model considering the change rate of magnetic density and the influence of waveform coefficient on the loss is given. At the same time, in order to improve the generality of the loss model at different characteristic frequencies, the nonlinearity of the loss model coefficients is further studied, and the nonlinear relation function between the model coefficients and the frequency is given. Finally, the finite element calculation results are compared with the physical measurement results, which proves the accuracy of the loss model for the calculation of nanocrystalline core under square wave excitation.