The thermal fracture of rock under cryogenic impact is caused by the instantaneous huge thermal stress caused by rapid temperature change. The law of heat transfer inside rock determines the magnitude and action form of thermal stress. In order to study the heat transfer law of rocks under cryogenic impact, the internal temperature fields of granite, sandstone, and limestone under cryogenic impact in liquid nitrogen environment at different times were measured, and the influence effect of rock characteristics on heat transfer law was discussed. The results show that: 1) The temperature change process of rock during cryogenic impact can be divided into three stages: slow cooling, rapid cooling, and temperature flattening; 2) The evolution law of temperature and temperature gradient of different types of rock is roughly the same, but its distribution is different owing to different rock characteristics; 3) The heat transfer and deformation of rock under cryogenic impact are affected by the mineral composition and structural characteristics of the rock itself, and they together determine the final degree of rock fracture.