Aiming at the problem of the ambiguity and specificity of the tumor information on the non-contrast image, this paper proposes the use of multi-modality non-contrast MRI image information to complete the segmentation and detection of liver tumors. First, the multi-scale feature extraction block(MsFEB) extracts deep semantic features for tumors of different modalities. The network further fuses the features to obtain a multi-modal fusion feature with characterization ability. Second, the segmentation path uses the fusion feature to restore the tumor layer by layer to obtain the segmentation result; the detection path uses the fusion feature to complete the tumor b-box and classification. Finally, the network completes the training of segmentation and detection at the same time under the constraint of the joint multi-task loss function. The test results on 250 clinical MRI images of liver without contrast agent show that the segmentation dice coefficient reaches(81.98±1.07)%, the pixel accuracy reaches(93.72±0.97)%, the intersection ratio between the b-box of the detection and the gold standard reaches(80.19±1.46)%, and the classification accuracy rate reaches(90.36±0.61)%, which demonstrate that the the proposed method can segment and detect liver tumors in MRI images without contrast agent at the same time.