O'Neill had a 48-65 record, including 7-10 this season after going 6-26 last year, setting a school record for losses.
Veteran assistant Bob Cantu will serve as interim coach while USC searches for a successor. Cantu took over for O'Neill during a Pac-10 tournament semifinal game against Arizona in 2010-11 when the coach was suspended after getting into a verbal confrontation with a Wildcats booster. He has remained on the staff through four coaching changes and is the longest tenured assistant in the Pac-12.
Athletic director Pat Haden said it became evident to him that the program needed new leadership.
"Despite a nice road win in our last game, I felt it was best to make a change now, with most of the Pac-12 season still ahead of us, in order to re-energize our team," he said.
The Trojans are 2-2 in the Pac-12 after a road split last weekend. They lost at Colorado before winning at Utah to snap a 14-game road skid. USC has lost nine of its last 13 games, which included a five-game skid.
"I enjoyed my four years at USC," O'Neill said in a statement. "It is a special place. I enjoyed the people I worked with, the players I coached and our fans. I thank USC for allowing me to guide their program. I couldn't be more proud to work anywhere."
O'Neill was hired by Haden's predecessor, Mike Garrett, who said at the time, "Hopefully he's here forever." O'Neill took the USC job after spending a season as an assistant with the Memphis Grizzlies.
Last season, the Trojans were hard-hit by injury, leaving O'Neill with just six scholarship players who managed to keep the team in nearly every game.
Upon arriving at USC in June 2009, O'Neill had to deal with the fallout from an ongoing NCAA investigation and school-imposed sanctions involving star O.J. Mayo, who played one season under Tim Floyd before leaving for the NBA. The next season, O'Neill lost three starters and four of his top six players from the rotation. In O'Neill's second season, he guided the Trojans to a 19-15 record and a NCAA tournament berth.
Then came last season, when the Trojans were plagued by injuries, most notably to point guard Jio Fontan, who tore his ACL after scoring 57 points in two exhibition games during an exhibition tour in Brazil. Forward Aaron Fuller (shoulder) and big man Dewayne Dedmon (torn MCL) also went down with season-ending injuries at different points.
"Last year was very difficult because of our injuries and the fact that when I got here there was no freshmen or sophomores, so we had two blank recruiting classes," O'Neill said last fall. "The only way you can put it back together to get where we have the bodies we have now is with junior college guys, transfers and blend in a few freshmen. Now our goal is to get mostly freshmen going forward, which we're doing. We have four guys committed right now and we're hoping to get one more, and we're starting to build what I think is a really solid program."
O'Neill had a full complement of 13 scholarship players who were healthy this season.
"It was hard for me to evaluate him as a head coach until this year when he had enough players and veterans to compete," Haden said.
O'Neill, who turns 56 on Jan. 24, was hired by coach Lute Olson as an assistant at Arizona before the 2007-08 season, and became interim coach when Olson took a medical leave of absence.
O'Neill led the Wildcats to a 19-15 record and the school's 24th consecutive NCAA tournament appearance, at the time the nation's longest active streak.
He was designated Olson's permanent successor, but when the Hall of Famer returned that spring he announced O'Neill would no longer be part of the program.
O'Neill had previous head coaching jobs at Marquette, Tennessee and Northwestern. He has a 219-245 record in the college ranks.