The Sun claims that the Aussie singer, who has been hotly tipped to replace Kelly Rowland on the ITV1 series, pulled out of talks with XF bosses after they failed to meet her rumoured £5million demands - leaving the star to withdraw from the show after 48 hours of 'tense' negotiations.
Friends of the Put The Needle On It hitmaker have also revealed that Minogue is still angry by Simon Cowell's revelations that they had a 'secret affair', which was exposed in Tom Bower's tell-all book about the music mogul.
A source explained: "It was so upsetting for her to be described as one of Simon’s ‘toys’ and for him to talk about their fling in such a humiliating way.
“Huge sums have been put on the table and it’s not convinced her. It wasn’t about money. She could easily have got more than a million. They desperately wanted her.”
Despite these reports, XF insiders claim they pulled the plug on Minogue, with sources suggesting the 40-year old wanted five times what was offered to her.
They said: "We cannot be held to ransom. We are putting contingency plans in place.”
Dannii was forced to leave The X Factor last year after it clashed with Australia's Got Talent, despite winning the 2010 series with Essex crooner, Matt Cardle.
Louis Walsh and Gary Barlow are already confirmed to return to the contest, while SiCo has hinted at Tulisa's comeback after naming her head-judge on Twitter following her win with girl group Little Mix in 2011.
With Dannii's deal falling through so close to the auditions, The Sun says guest judges such as US superstar Katy Perry will be drafted in to help until bosses find a permanent panelist.
Watch MTV News on the hour every hour on MTV - Sky Channel 126 and Virgin Channel 134