Fully convolutional networks for semantic segmentation provide pixel-level detection of strip steel surface defects, which plays a crucial role in product quality control of strip steel. However, most of these models suffer from the loss of boundary information, and their performance is often heavily dependent on a large number of labeled samples, which limits the application of the approach. Thereby, a multiscale and boundary-aware network for segmentation of strip steel surface defects on small datasets was proposed in this work. The network consists of two cascaded encoder-decoder subnets. The first subnet employs an encoder built with one-shot aggregation modules and a feature pyramid attention module to extract hierarchical and multiscale features and reduce the dependence of performance on training dataset size. Then, a decoder consisting of global attention up-sample modules exploits high-level feature map to guide low-level features recovering the lost spatial information, and generates preliminary prediction results. Finally, the second subnet further refines the prediction results from the first subnet. Experiments on NEU-Seg defect dataset demonstrate the feasibility and effectiveness of this method for automatic extraction of surface defects such as inclusion, patch, and scratches.