Over the weekend, The Sunday Mirror had claimed Arthur and Flack were spotted on a date - with the Impossible singer said to be "head over heels" with the 32-year old.
A source told the tabloid: "James built up a reputation as a ladies' man over the series and made no secret of the fact he was enjoying all the female attention he was getting.
"But while he has been playing the field he has always had feelings for Caroline. She made it clear nothing could happen while the show was still on, but as the weeks went by they got closer and closer.
"At the wrap party on Tuesday – which was held at the Hippodrome Casino in London – they spent almost the entire evening together and there was a lot of flirting. They left separately but arranged to meet again."
And they are claimed to have shared a more intimate evening on Thursday at the Groucho Club, where the source says they "spent four hours chatting and drinking" and "looked like they were really close.
"Again, they went home separately but they had a great evening and really enjoyed each other's company," the source continued. "There could definitely be a future between them."
Despite the rumours, Caroline's spokesperson has denied that she's dating James, stating: "It’s definitely not true and it won’t be changing anytime soon."
Flack has previously admitted that she finds Arthur "very charming", with James also telling The Metro: "We get on well and we do a little bit of harmless flirting. I just did that to her on the show (Xtra Factor) to deflect from the fact that I’m rubbish at interviews.
"Things like this have to be developed. If something develops in the future then whatever, but if not I’m happy being friends with Caroline."
Arthur has since been spotted holding hands with former Made In Chelsea star Kimberley Garner, where they were seen leaving Hakkasan restaurant in Mayfair together last night (16th December).