The ex-Manchester City, Inter Milan and Juventus midfielder was in his third season as head coach of New York City.
The 41-year-old had been tipped to replace Arsene Wenger at Arsenal before the appointment of Unai Emery.
“Leaving New York is an incredibly difficult decision for me and my family and one that has not been taken lightly,” said Vieira.
Nice finished eighth in Ligue 1 last season, missing out on a Europa League place by one point.
Vieira takes over from Lucien Favre, who was appointed Borussia Dortmund coach in May.
Assistant coach Christian Lattanzio, performance coach Kristian Wilson and physical performance coach Matt Cook have also left New York to join Vieira at Nice.
World Cup winner Vieira retired at Manchester City aged 35 in 2011 and was named a football development executive with the Premier League club.
In 2015, Vieira left his role as head coach of Manchester City’s Under-21s after signing a three-year contract with New York City.
The Major League Soccer side are part of the City Football Group founded by Manchester City owner Sheikh Mansour.
When Wenger announced his departure after 22 years in charge, Vieira was linked with the Arsenal role along with the likes of Luis Enrique, Carlo Ancelotti, Joachim Low, Brendan Rodgers and Leonardo Jardim.
At the time, Vieira said he was “flattered” but “happy” as New York coach.
“I am always flattered to hear my name linked with different football clubs. That is good for your ego but at the same time, I am happy here,” Vieira told New York radio station WNYE in April.
“We will see what will happen in the next couple of years.
“I spent nine years at Arsenal which makes the club really special for me. But that is not enough to coach the team.”
Can a successful player become a successful manager?
During his playing career, Vieira won four FA Cups and eight league titles.
He won six major honours at Arsenal between 1996-2005 before helping Juventus finish top of Serie A in 2006, although they were later stripped of the title.
However, he then helped Inter Milan win four straight Italian titles from 2007-2010 before joining Manchester City in January 2010, helping them win the 2011 FA Cup.
His 107 caps for France include winning the 1998 World Cup and Euro 2000.
Vieira began his coaching career in impressive fashion, guiding New York to back-to-back play-off appearances in his first two seasons.
He leaves with New York in second place in the MLS’ Eastern Conference after winning eight of their 15 games.