According to The Sun, SiCo - who is busy shooting The X Factor USA in Los Angeles - is said to have rang his UK team, telling them: "Sort it out. This is not good enough."
The music mogul is also apparently planning to speak to his replacement, Gary Barlow, to tell the Take That frontman to "raise your game", as well as addressing the issue of "meddling" judge, Kelly Rowland.
An insider told the tabloid: "Simon is furious. This would never have happened on his watch and he wants the team to know this and sort it out."
"He knows there is great talent on the show, but he thinks the attention to detail has been shocking. The acts are not being showcased at their best."
They continued: "Say what you like about Simon, but he spends ages dithering over the tiny details to make sure everything is perfect. There's no way he would have sanctioned some of those song outfits or choices."
"The figures are hugely disappointing for The X Factor because the whole point of the live shows is that they are supposed to build momentum."
The source added: "Last year there were nearly 14million watching the second live show. Where have all those viewers gone? It's no wonder Cowell has gone ballistic. Losing viewers is the last thing he wanted to happen."
The insider revealed: "Kelly is trying to turn this into The Kelly Show with her meddling — and it's really putting people's noses out of joint. The contestants love it as they're in awe of her but the judges hate it."
"Tulisa has properly fallen out with her after Kelly went to speak to her about the styling and songs and vocals for Rhythmix."
They explained: "Tulisa just ignored her and was fuming that she was poking her nose in. Kelly walked off muttering something about, 'Tulisa not wanting her advice, even though she has sold millions of records and was in the biggest girl group ever'.
Watch MTV News on the hour every hour on MTV - Sky Channel 126 and Virgin Channel 134