By Dana Rose Falcone
Updated April 22, 2015 at 08:50 PM EDT
Leigh Vogel/Getty Images

After 10 physicians submitted a letter to Columbia University’s dean of medicine recommending Dr. Mehmet Oz’s removal from the school’s faculty last week, the well-known doctor is fighting back.

The group of physicians, many of whom are linked to the genetically modified food industry, focused the letter on Dr. Oz’s “baseless and relentless opposition to the genetic engineering of food crops.” Dr. Henry I. Miller of Stanford University’s Hoover Institution spearheaded the note. “I do not claim that GMO foods are dangerous,” Dr. Oz responded in a statement last week, “but believe that they should be labeled like they are in most countries around the world.”

On Thursday’s episode of his daytime talk show, Dr. Oz will address his critics, who wrote that the Oprah-approved doctor is guilty of “outrageous conflicts of interest or flawed judgments about what constitutes appropriate medical treatments.” In a preview clip released, Dr. Oz declares, “I know I’ve irritated some potential allies in our quest to make America healthy. No matter our disagreements, freedom of speech is the most fundamental right we have as Americans. And these 10 doctors are trying to silence that right.

“So I vow to you right here and right now: we will not be silenced, we will not give in.”

Dr. Oz, who serves as the vice chairman of the university’s surgery department, will also appear on the Today show on Friday to further address his position.