It’s that time again. Over the next few weeks, broadcast insiders will watch dozens of TV pilots in contention for fall. Executives will debate their merits in meetings, publicists will chat about their favorites over lunch and agents will trade hallway gossip about which titles are hot. Some everyday folks will see the pilots, too — roughly 300 civilians wrangled into carefully monitored focus groups.

But like nearly every year since the dawn of TV, this evaluation process will happen in private. Networks will spend millions creating entertainment programs written by top Hollywood writers starring brand-name celebrities that the public will never see.

Amazon, however, did something interesting last week. The company threw its fledgling TV studio’s pilots online. Like: “Here! Tell us what you think.” Amazon customers watched the 14 titles and left ratings and comments, as if the pilots were toasters or headphones. The company has said they plan use this feedback to help them decide which of the projects will get full series orders.

Imagine that. A content producer letting the public help decide which pilots become shows. You know, those same shows that eventually sink or swim due to what the public thinks of them.

So here’s the question: Should broadcast networks put their pilots online for judgement too? Why should the Big 5 networks (nowadays, more like the Mid-Sized 5) wait until fall — after they’ve already shot eight-or-so episodes and spent millions in marketing — to learn if any of their shows are popular? Just throw all the pilots into the online Thunderdome! Given broadcast’s recent track record, could the Internet really do worse?

You might think Amazon’s strategy would spark a range of opinions from broadcast insiders. It doesn’t. Every person in my admittedly modestly sized survey group more-or-less hated the idea of opening the pilot season sausage factory to the masses. They make good points, too. Here are their objections, along with some counter-arguments:

1. Internet and TV have different audiences: If Twitter equaled Nielsen ratings, The CW’s Vampire Diaries would have more viewers than CBS’ Two and a Half Men. According to Nielsen’s research, the vast majority of TV viewing is still on a traditional set. Having pilots judged by online viewers would give networks a skewed sense of what might work in the fall — the entire broadcast schedule might be nothing but sci-fi shows, tween-lit adaptions and whatever Joss Whedon wants to do (including ABC’s S.H.I.E.L.D. pilot, related photo above). Amazon’s situation is different since its entire audience is online. It’s tough to imagine that some low-buzz successes like ABC’s The Middle would have ever gotten the Internet excited. “What works in the digital format may not scale up to work in broadcast,” one major studio insider said. Remember NBC’s Quarterlife?

BUT: That’s changing. The Internet as a barometer of popular taste is becoming increasingly accurate as more people become heavy Internet users. According to Nielsen, for ages 11 and up, the younger the viewer the more more hours they likely spend watching video online or via mobile devices. Plus, heavy engagement fans who interact with content are desired by advertisers. So the online fandom crowd may give a disproportionate sense of the popularity of certain shows, but they also represent an audience that networks disproportionately want to reach.

2. Pilots are rough drafts. Pilot titles, characters, cast members, story threads can all get tweaked between May and September. Networks want the public to first see the best possible version of a show, something that’s polished and finished and has the highest odds of drawing an audience. Would you have wanted your first experience of ABC’s Lost to be — just hypothetically — a pilot on your laptop called Stranded on Mystery Island, with David Spade playing Hurley and a dinosaur bursting out of the jungle at the end?

“If something isn’t picked up, for whatever reason, but people really liked it, that could be a problem,” one network insider said. “Or if people hated something, and we pick it up — again, for whatever reason — you’re starting off on a bad note.” There was huge online buzz, you’ll recall, for NBC’s Wonder Woman pilot (see video below), but the narrative turned sour once the network decided to not pick up the show. Noted a major network programming researcher: “Great pilots don’t always make great television series.” Conversely, if you’re a network executive, you usually don’t need millions of people to tell you a show sucks. And sometimes networks can see potential that others might miss — CBS’ The Big Bang Theory, for example, famously had a weak pilot, while the pilot for the U.S. version of NBC’s The Office wasn’t so hot either.

BUT: Many fans like being part of the process. There are fans who want to know every detail about a show or movie before it airs — on-set photos, working titles, casting changes, the works. They follow the process on entertainment sites, and that interactive engagement generates valuable buzz for a project. Others just want an unspoiled idealized finished product (just ask J.J. Abrams). Putting pilots online wouldn’t force anybody to watch them — the networks shouldn’t even promote such an endeavor — but it would give fans who like engaging with content a chance to weigh in. Just like how producers sometimes cast actors after seeing their names suggested online for a specific part, sometimes harnessing the Internet’s collective brainpower is helpful.

3. Pilots should debut along with the rest of the episodes. A TV season is about momentum and establishing a pattern of viewership. Putting pilots online months before the rest of the show breaks the pattern. Not to mention, “You want to craft some excitement around the premiere, make it an event,” one broadcaster said.

BUT: It might not matter. Fox previewing the pilot for Glee in May didn’t hurt the show when it officially debuted in the fall. TV reporters see all the pilots in May, it sure doesn’t diminish our excitement for shows that we like. If viewers enjoy a show, they’ll wait for more.

4. The current process works. Pilots may not be watched by thousands before executives judge them, but they’re watched by many people who are experts at evaluating content and knowing what needs fixing. The online hordes are amateurs (no offense). Networks have actually experimented with screening pilots on cable about a decade ago and found such airings unhelpful (though that was before the Internet really became the powerful feedback machine it is today). “I feel pretty good about our process,” one network insider said. “It’s not perfect, but I think we know the strengths and weaknesses.” Not to mention, pilots aren’t just products — they’re art, dammit. Isn’t the Internet nothing more than another giant committee trying to mess with a TV writer’s vision? Do we really need millions of cooks in the kitchen?

BUT: Except when it doesn’t. This point is tough to argue since dead pilots rarely ever get second chances or see the light of day. But when ABC ordered Ugly Betty, the network was so tepid about the project after its pilot process that it was slotted for a Friday night death slot until critics gushing about the show over the summer convinced executives air it on Thursdays where it became a modest hit. While NBC touted Crystal the Monkey on the short-lived Animal Practice last year as the highest-testing character on the network’s new slate (we’re not sure why they would say that either). NBC also developed The Walking Dead (though never made a pilot), then let the project go to AMC, where it’s now thumping broadcast in the ratings. If NBC had shot a high-quality Walking Dead pilot and put it online, would the feedback have convinced the network it had a hit? Definitely — though a focus group might have said the same thing (it’s tough to imagine a focus group would love Crystal the Monkey AND Rick shooting a child zombie in her bunny slippers … but okay).

You’ve read the arguments. Now vote! Should broadcast nets put their pilots online?

As promised, here’s the opening credits of NBC’s Wonder Woman pilot: