United took the lead within the first minute as Fortune Makaringe played Bandile Shandu through on goal, and the latter fired past the on-rushing Patrick Tignyemb.
In the ninth minute, Tshegofatso Mabaso saw his long range shot fly over the bar, as the home team searched for a quick response.
Yannick Zakri fashioned space for himself in the 18-yard area as his devilish shot flew across the upright after 18 minutes.
With 30 minutes gone, some great interplay from Ndumiso Mabena and Menzi Masuku led to Mabaso on goal but his volley crashed onto the crossbar.
Steve Komphela's men went into halftime in danger of losing their unbeaten streak as Shandu first minute goal gave United the advantage.
Five minutes into the second Komphela threw in Tebogo Potsane in place of Menzi Masuku in search of an equaliser.
The second half was a less open affair as neither team could muster clear chances, with Celtic dominating possession.
The Team of Choice resorted to a more defensive approach in the second half, spending much of the time in their own half.
As Celtic pushed for an equaliser, defender Lorenzo Gordinho saw his header fly wide in injury time.
In the end, Fadlu David’s men managed to defend their first-half lead to hand Celtic their first loss of the season.
Celtic unused subs: Dambe, Baloyi, De Jongh, Nyundu.
Maritzburg unused subs: Mpandle, Allie, Thlone, Masegela.