The school issued a statement announcing Davis' dismissal nine days before the start of preseason practice. Chancellor Holden Thorp said the decision was not prompted by any changes in the ongoing NCAA investigation but said he "lost confidence in our ability to come through this without harming the way people think of this institution."
"Our academic integrity is paramount, and we must work diligently to protect it," Thorp said in a statement. "The only way to move forward and put this behind us is to make a change."
The school has scheduled a news conference for Thursday morning in Chapel Hill to discuss the change.
Davis compiled a 28-23 record in four seasons with the Tar Heels after taking over for John Bunting. His program looked ready to contend for an Atlantic Coast Conference championship and a BCS berth last year before NCAA investigators arrived on campus last July.
In all, 14 players missed at least one game and seven were forced to sit the entire season. Last month, the NCAA sent a notice of allegations to the school outlining numerous potential major violations.
That included unethical conduct of former associate head coach John Blake, whom the NCAA accused of trying to steer players to late NFL agent Gary Wichard.
But Thorp and athletic director Dick Baddour had remained publicly supportive of Davis over the past year, including at a joint appearance with the school's board of trustees in November.
Davis' dismissal comes just two days after he fielded questions about the aftermath of the investigation from reporters at the Atlantic Coast Conference's preseason media days in Pinehurst.
Davis said he found the university's public support "reassuring" and said he had never considered quitting. He also talked about how the coaching staff and university had taken steps to correct past mistakes.
"Anything we can do to make sure this doesn't happen again, that's part of my responsibility," Davis said. "I regret greatly that these things have transpired and these things have happened. I don't take them lightly. This is a very, very serious issue. It's caused a tremendous amount of embarrassment and a tremendous amount of hard times for Carolina alums and fans. But we're going to get through this. And because of it, we're going to come out of it, and we're going to be better than we were before."