Mourinho replaces Louis van Gaal as manager after the Dutchman failed to lead the Red Devils into the Premier League top four.
Despite winning the FA Cup, Van Gaal was sacked at the end of the season amid much speculation linking Mourinho with the job.
Now the Portuguese coach is in place at Old Trafford, former United boss Ferguson is expecting big things next campaign.
"I think there'll be a massive improvement," Ferguson told Sky Sports.
"I think Jose was wrong to go back to Chelsea. It's always difficult when you go back somewhere for a second time.
"He's won the European Cup twice with different clubs and won titles with all the clubs he's been at. You can't ignore his record and I think Manchester United is the right kind of club for him."
Meanwhile, Mourinho has denied that he was responsible for United legend Ryan Giggs leaving the club. The long-serving Giggs left United at the weekend after almost 29 years as player and coach.
The Welshman had spent two years as assistant boss to Van Gaal, who was sacked in May, but opted not to continue under Mourinho.
Mourinho dismissed suggestions he had been reluctant to work with Giggs, insisting the 42-year-old had decided to leave.
He said: "It is not my responsibility that Ryan is not in the club. The job Ryan wanted is the job the club decided to give me. It is not my fault. Ryan wanted to be Man United manager.
"When you say did I offer him a job, could he be my assistant - he could be what he wanted in the club, but he made his decision. Ryan wants to be a manager."