Credit: CBS

Since he debuted Late Night with David Letterman in February 1982 on NBC, David Letterman has introduced the world to countless new artists and sounds—and played host to some of late-night’s most memorable musical performances. Just a few of the icons who started out on his stage: R.E.M., Arcade Fire, Counting Crows, and dozens more.

As the host of the show, Letterman has also bid farewell to numerous greats over the years: Warren Zevon made his last public performance on Late Night, while Sammy Davis Jr. and Lou Reed delivered some of their final television performances in front of their friend.

As Letterman prepares to bid good night one last time on Wednesday, we’re looking back at his best musical performances. Here are EW‘s 17 favorites. (As for Letterman’s personal favorites, all we can guarantee is that drums are probably on the list.)

Whitney Houston, “Saving All My Love for You,” 1985

Damn, Whitney: we miss you, girl!