Keagan Dolly broke the deadlock for Chiefs early in the second-half and Linda Mntambo thought he had at least earned Pirates a point with the equaliser in the last three minutes.
However, Dolly netted the winner for Amakhosi from the penalty spot in stoppage time, and Baxter was pleased to see his side get back late in the game.
"Yeah thoughts immediately after you win the derby, it's a little bit scrambled but happy, proud of the lads for getting back right at the death," Baxter told SuperSport TV.
"I thought we were quiet comfortable at one stage and Pirates came back into it, credit to them. The goal it is what it but we lose the ball, we get done on a transition.
"I thought first-half it was an even game but second-half I thought we were more comfortable. But the derbies are the derbies, things like miskicks or bad tackle can make you lose it or win it. So delighted with the win."
Baxter went on to explain why he withheld his substitutions until late in the game.
"Because I thought, it started right, it sounds strange but it started lashing them, this is why I didn't make changes, if you put players on the weather conditions have certainly changed, it takes them time to get into the game," he said.
"I didn't think we could afford that against a Pirates team that was pushing on at the time. I wanted to put them, I didn't want the tired legs to collapse completely, I wanted to get them but I waited for the weather but then we concede a goal, we've got a scramble, go for it.
"I'm learning that if we play the way we can play, then we can beat anybody in the league but at the same time, we can be our own worst enemy and we can be picked up by anybody in the league and that is a characteristic that we have to change.
"We have to change, we have to be more consistent and we have to make sure that when when we have the possibility to put the game to bed, we put it to bed and then we don't give away silly chances. That's what we've known for a while, we're working very hard to put it right," he added.
