The 30-year-old, who made his international debut in 2009 under Fabio Capello played 61 times for his country, scoring one goal.
Milner was part of the England team at last month's European Championships in France but only had three minutes of playing time.
The former Manchester City, Aston Villa, Newcastle and Leeds man has been a regular in England squads because of his ability to play in multiple positions.
Milner said in a statement released by the FA: "I would like to thank Sam for taking the time to come and see me in person and have the conversation.
"I think the decision we have reached is all about common sense. I wish Sam, the players, staff and supporters success in the future."
Allardyce said: "James can be proud of his seven-year career as a senior England player and has never let his country down when called upon.
"We wish him well with Liverpool this season and beyond."