With the development of deep learning and natural language processing technology, pre-training model has become an effective way to solve problems such as relational classification and text classification. However, in practical applications, the pre-trained model still faces the challenge of low quality and quantity of knowledge information required for complex tasks, and the fusion of knowledge graph into the pre-trained model can enhance its performance. This paper analyzes and summarizes the literature about knowledge graph fusion pre-training model in recent years. Firstly, it briefly introduces the reasons, advantages and difficulties of introducing knowledge graph into pre-training model. Secondly, two kinds of methods of implicit combination and explicit combination are discussed in detail, and the characteristics, advantages and disadvantages of representative models are compared and summarized. Finally, the challenges and future research trends of pre-training models with fusion knowledge graph are discussed.