The terahertz image produced by terahertz imaging system has the characteristics of single hue and sample, low resolution, sharpness, and contrast, and small amount of data. In order to solve these problems, more terahertz image data sets were generated through the DC-GAN network, and the ESRGAN network was used to accomplish terahertz image super-reconstruction and linear transform threshold processing, which makes the image details and texture features clearer, and filters out a lot of background noise to enhance the image contrast. At the same time, the non-maximum suppression algorithm in the network was improved for the situation where there are many overlapping detection targets(knives, mobile phones) in the terahertz security image. The Sigmoid weighting method was introduced to avoid the detection frame with a large overlap with the target being directly deleted. By reducing its confidence, the problem of missed detection due to target stacking in the terahertz security inspection image was solved. The experimental results verify that the improved Faster R-CNN network has higher accuracy for detecting suspicious objects in THz security image.