In order to promote energy conservation and facilitate the utilization of solid wastes, red mud, coal metakaolin, and alkaline activators were used as main raw materials to prepare red mud-coal metakaolin based geopolymer (RCG). The effects of glass fiber, basalt fiber, polypropylene fiber, and polyvinyl alcohol fiber on the compressive strength, peak strain, elastic modulus, compressive toughness, energy absorption, flexural strength, and microstructure of RCG were investigated. The toughening mechanism of four kinds of fiber on RCG was expounded, and the results indicate that incorporation of fibers can enhance the mechanical properties of RCG and improve its toughness and ductility. Among them, basalt fibers and polyvinyl alcohol fibers have similar effects on strength enhancement, but polyvinyl alcohol fibers can effectively change the brittle failure modes of RCG and enhance the compression toughness and energy absorption capacity of RCG. Through microscopic morphology analyses, polyvinyl alcohol fiber has high moisture regain, and the bonding effect with the matrix was found to outperform other fibers, so the physical and mechanical properties of RCG are improved better.