By Andrea Towers
Updated April 30, 2015 at 07:09 PM EDT

The latest bit of goodness from the Avengers: Age of Ultron press tour focuses on the decade-long friendship between Chris Evans and Scarlett Johansson. The duo have known each other years, having starred together in five different movies, including The Perfect Score and The Nanny Diaries.

But just how well does Johansson know her co-star? While promoting Age of Ultron on The Ellen DeGeneres Show, Evans crashed the actress’ interview, surprising her before joining DeGeneres and Johansson for a round of trivia called “Avenger? I Hardly Know Her.”

Johansson, who has previously admitted to the fact that she hates being pranked, was genuinely shocked by her friend’s appearance—and the reaction was pretty perfect. (The trivia, not so much. Come on, Evans… you should know go-to karaoke songs!)

Avengers: Age of Ultron

  • Movie
  • PG-13
  • 150 minutes
  • Joss Whedon