By James Hibberd
Updated January 10, 2013 at 04:39 PM EST
Britney X Factor

Oops, Factor is doing it again!

The Fox reality show is shedding another judge. Britney Spears is stepping down from her post after a single season, a source close to the show confirms. Reports suggest she’s quitting because she wasn’t going to be asked back anyway (which is sort of like saying you broke up with somebody who stopped returning your texts). She’ll follow departing judge L.A. Reid, who quit the show last month. TMZ, presumably quoting Team Spears, broke the story with the standard “wants to focus on her music” explanation.

Factor‘s ratings dropped significantly for season two despite Spears’ high-profile hire. Though critics spread the blame for the performance woes across the show’s various talent and production elements, Spears was considered a particularly weak part of the lineup — a famous and expensive face that didn’t have all that much to say (she was paid a reported $15 million).

“The producers hadn’t planned for Britney to return this year,” a source close to the show says. “She did her best last season, but they have been planning a new judging line up over the holiday period and she was never going to be part of those discussions. She did her best and they had fun last year, but it’s no surprise to anyone that they were all going to be parting company. It wasn’t even really a conversation.”

Judge and executive producer Simon Cowell will, of course, return next season. With Spears and Reid out, that just leaves Demi Lovato. Like Spears, the young pop singer joined the team for season two, but its unclear if she’ll return. Plus, there’s new co-hosts Mario Lopez and Khloe Kardashain, both of whom received a fair amount of critical slagging, to consider. Last year, first season judges Nicole Scherzinger and Paula Abdul got the boot. Though high judging table turnover was also a staple of the original UK edition of Cowell’s X Factor, one has to wonder if this much instability will make it tougher for to attract top celebrity judging talent to the show for season three.

Fox hasn’t commented yet on this story and spun the matter rather differently on Tuesday. The network’s entertainment chairman Kevin Reilly defended Spears’ performance to critics at the network’s press tour event in Pasadena. “I think Britney did a really good job,” Reilly said. “People remain fascinated with her and always will be. She’s just an interesting figure who’s lived a unique life, and I thought she brought some of that. And people got to know her a little bit. Maybe some people were waiting for more drastic displays of some nature that never came. But I thought The X Factor was excellent this year. Last year, the ratings were a little bit higher, but I think it was a better show this year.” Reilly even went so far as to agree that the network would be “on board” with bringing her back for season three.

Fox’s reality guru Mike Darnell noted the show’s second season was plagued by scheduling hiccups — including NBC airing The Voice against Factor‘s premiere — but likewise defended the judges, saying they “worked.” The executive admitted co-hosts Lopez and Kardashain, however, could have been better. “I think the hosts were okay,” Darnell said. “I think Mario knows how to be a host. I think Khloe was new. And I thought for someone new she did okay. She never hosted before.”