By Jeff Labrecque
October 04, 2014 at 07:07 PM EDT

Ben Affleck’s publicity tour to promote Gone Girl took a detour on Friday night, when the outspoken liberal engaged in a heated debate with author Sam Harris and HBO’s Real Time host Bill Maher over their criticism of Islam. “They’ll criticize Christians … but when you want to talk about the treatment of women and homosexuals and free-thinkers and public intellectuals in the muslim world, I would argue that liberals have failed us,” said Harris. “We have been sold this meme of Islamophobia, where criticism of the religion gets conflated with bigotry towards muslims as people. It’s intellectually ridiculous.”

Affleck, who frequently expressed impatience and outrage at Harris’ more measured explanations, was offended by the message. “[Your point of view] is gross, it’s racist,” the actor said. “It’s like saying, ‘Oh, you shifty Jew!’”

“We have to be able to criticize bad ideas, and Islam at this moment is the motherlode of bad ideas,” said Harris, who’s written several books about world religions.

“It’s the only religion that acts like the mafia—that will f–king kill you if you say the wrong thing, draw the wrong picture, or write the wrong book,” Maher finally said.

“What is your solution? Just condemn Islam?” fired back Affleck. “We’ve killed more Muslims than they’ve killed us, by an awful lot. We’ve invaded more Muslim countries than they’ve invaded ours, by an awful lot. Yet somehow we’re exempted from these things because they’re not really a reflection of what we believe in. We did it by accident, that’s why we invaded Iraq.”

The show’s other panelists, Republican Michael Steele and journalist Nicholas Kristof, mostly stayed out of it, though Kristof calmly backed Affleck to a certain extent, questioning whether Harris’ point of view lacked the necessary amount of nuance.