In order to optimize the performance of sensorless control system of permanent magnet synchronous motor (PMSM) based on sliding mode observer (SMO),the mathematical model and control algorithm of the corresponding module in the traditional control system were analyzed, and the optimal SMO based sensorless control model of PMSM was established and simulated. By analyzing the traditional SMO, the sliding mode gain was optimized in real time according to the different operating conditions of the motor on the premise that the SMO satisfied the Lyapunov stability condition.The performance of SMO was improved. In order to improve the quality of the output signal of the phase locked loop (PLL), a low pass filter was added at the input of the PLL, and the position deviation caused by low pass filter was compensated. The correctness and feasibility of the above optimization strategy was verified by simulation. The results showed that the optimized model could obtain better observation signal than the traditional observation system under the condition of ensuring the dynamic performance of the system.