Rumours are now circulating over who will replace Abdul and the ex-Pussycat Dolls star, with a source telling The Mirror: "Simon wants nothing but the best for The X Factor - and Mariah is the best,"
However, Carey's hubby Nick Cannon has shot down the reports, telling reporters that being a judge on the show would be a "step down" for his wife.
Cannon claimed: "Why would my wife want to be on a show like that? For somebody who has a legendary career, I don't know why they would."
The Sun reports that calls are being made for sacked judgeCheryl Cole to make a comeback, while UK XF panellistKelly Rowland's name has also been thrown in the mix following reports that she was "unhappy" on the British series.
Young Money rapper Nicki and Katy Perry are also in the frame, with allegations suggesting that music mogul L.A. Reid's job on the panel is safe.
US celebrity blogger Perez Hiltonhas also reached out to SiCo in a bid to secure his seat on the show, stating in an open letter that he would the perfect choice.
Hilton explained: "Most importantly I am a huge music lover and I bring with my a lot of experience and expertise in the world of music. I have been a huge champion and early supporter of such artists as Lady GaGa, Katy Perry, Adele andAmy Winehouse.
"I'm just going to me and I'm so fabulous - in case you couldn't tell - and fun and self-deprecating! I'm going to keep it real, I'm not going to try to be like anybody else. I'm going to be me and be honest. That's not necessarily being mean or necessarily loud, it's just being the viewer and the expert."
He added: "I was watching X Factor so much this first season and I was shocked at how often the panel and judges' mentors were just whitewashing things and feeding the contestants and the audience a bunch of BS. People can smell baloney a mile away."
Perez went on to describe himself as "professional and easy to work with", stating he would give 100% of his salary to charity.
The 33-year old concluded: "I will do whatever it takes! I'm Perez Hilton and I'm your next X Factor judge."