In order to improve the lateral tracking control performance of low speed autonomous vehicle under different conditions, a joint controller based on pure pursuit (PP) control and model predictive control (MPC) is designed. When the deviation of position and direction are large, it is switched to PP control. When the deviation of position and direction are small and the MPC constraint is satisfied, it is switched to MPC. At the same time, a mode switching controller is designed to ensure smooth switching between the two kinds of control systems. The simulation analysis and real vehicle test are carried out for the joint controller. The test results show that, compared with MPC, the speediness and stability of the tracking of the joint controller are optimized obviously and compared with PP control, the accuracy of the tracking is also improved obviously. The addition of mode switching controller improves the robustness of the switching process.