Compared with the current gesture recognition system based on radio frequency identification technology, the single-tag non-contact gesture recognition system based on convolutional neural network proposed in this paper can maximize user experience. Without the need for the user to carry any equipment, a single tag and single antenna are used to achieve precise gesture recognition. First, the tag phase signal affected by multipath effect is read by adding interference artificially; Second, the tag phase signal that accords with the characteristics of time series is filtered, and the Dynamic Time Wrapping (DTW) algorithm is selected to match with the coarse-grained gesture recognition of prior fingerprint database; Finally, the tag phase signal is used to generate the feature image by Markov Transition Field (MTF), and then IM-AlexNet model is used for in-depth training and experimental evaluation of the image. The training parameters of the improved model are reduced by 7% compared with those of the original model, and the accuracy rate reaches 96.76%. Experimental results show that taking the advantage of multipath effect, fine-grained real-time gesture recognition can be achieved in the case of an experimental deployment that only uses a single tag and a single antenna. The system is easy to operate, simple to deploy, expandable in a large range, and has high robustness.