Credit: Brad Barket/Getty Images

It’s hard to imagine the film world without Sir Ian McKellen behind the roles of two of the most important pop culture characters of our time: Lord of the Rings‘ Gandalf and X-Men‘s Magneto. But as the actor recently shared in PEOPLE, both of those roles came very close to not happening.

According to McKellen, he was initially offered a part in Mission: Impossible II, opposite Tom Cruise. Because he wasn’t given the whole script, however, he found it difficult to sign on for the role. “I couldn’t judge from reading just those scenes what the script was like,” he revealed. “So I said no. And my agent said, ‘You can’t say no to working with Tom Cruise!’ and I said, ‘I think I will.'”

Turns out fate works in mysterious ways — the day after turning down the film, McKellen was offered both Gandalf and Magneto by Lord of the Rings director Peter Jackson and X-Men director Bryan Singer, respectively. And when McKellen became concerned that he’d have to bow out of Lord of the Rings because X-Men production ran behind schedule, Singer fortunately realized how important that other role was. “Well, you must do Gandalf. I’ll make sure you get out in time,” Singer said, according to McKellen. Jackson even promised to “keep it free” for him.

Meanwhile, as he tells it, Mission: Impossible II was “put off, put off, put off” — “I had decided to do that, I wouldn’t have been in X-Men and I wouldn’t have been in Lord of the Rings,” McKellen told PEOPLE.

“You can’t prepare for luck,” he added. “I’ve learned that you’ve got to be ready for the luck when it arrives, but you’re going to need the luck.”