The referee was alerted that Caddell was still making his way off the pitch when the penalty was struck, so he ordered a re-take.
Crusaders protests were waved away before Gormley converted at the second attempt and he doubled the lead with another penalty before the break.
Gormley fired home to complete his treble and Stephen Garrett added a fourth before Gavin Whyte pulled one back from a penalty.
Premiership title challengers Coleraine eased to a 4-1 victory over Championship leaders and neighbours Institute at the Showgrounds.
Jamie McGonigle grabbed a double with Eoin Bradley also on target along with a Dean Curry won goal.
Curtis Allen had an afternoon to remember with the Glentoran forward scoring all four goals in the 4-0 victory at Ballyclare Comrades.
Glenavon also enjoyed a comfortable win, overcoming Dungannon Swifts 3-0 at Mourneview Park.
