Vusumuzi Vilakazi scored for Arrows, while Thabo Nthethe netted for Phunya Sele Sele.
Celtic the better side in the opening stages and had two goals disallowed in the opening 13 minutes of the game, on both occasions striker Lebohang Kukame was adjudged to have impeded Arrows keeper Nkosana Maseko.
Norman Smith almost put Arrows ahead on the half hour, but his lob was superbly tipped over by Celtic keeper Shuaib Walters, who had to scurry back after finding himself way off his line.
The first half ended goalless, but Celtic continued pushing forward in the second half and striker Mark Amofah saw his header from a corner came back off the crossbar.
Arrows took the lead in the 58th minute when Vilakazi slotted home from close range after receiving a good pass by Smith.
But the lead lasted just eight minutes as the hosts drew level through Nthethe’s header from a corner by substitute Grant Smit.
Smith then had an effort blocked by Gary Goldstone in the 84th minute, while Amofah should have sealed the match for Celtic three minutes from the end when he found himself unmarked inside the area, but the Ghanaian fired straight into the arms of Maseko.