In view of the serious heating problem of drive motor for vacuum pump in extreme vacuum environment, the cooling system of the drive motor for vacuum pump is designed and analyzed. A 2.9 kW shielded asynchronous motor is taken as the research object. The three-dimensional model of the motor with spiral waterway is established by finite element software, and the loss is calculated. According to the velocity and pressure distribution of spiral waterway, the rationality of waterway size design is verified, and the temperature distribution of each part of the motor is simulated and analyzed. According to the temperature changes of the motor under double rated load and cooling water fault conditions, the safe operation time of the motor is analyzed. Finally, the temperature rise test of the prototype is carried out, and the test results and simulation results are compared and analyzed. It is verified that the cooling system with spiral channel in the shell has a certain cooling effect on the drive motor for the vacuum dry pump, which provides a scientific basis for the research of the drive motor for vacuum pump in the extreme vacuum environment in the future.