By Elizabeth Leonard
Updated June 24, 2016 at 02:56 AM EDT
Jason LaVeris/FilmMagic

When Matthew McConaughey became an overnight superstar with the 1996 hit A Time to Kill, he tried to keep his private life out of the public eye.

His mother, not so much.

In fact, Kay, a former schoolteacher, so proudly embraced her son’s newfound fame that, well, she needed to share. A lot.

“K-Mac [his nickname for his mother] was enjoying this. I said, ‘Just do me a favor and if anybody wants to do an interview, just check with me first.’ She’s like, ‘I got it. I got it,’ ” the Free State of Jones star tells PEOPLE and EW editorial director in the latest episode of The Jess Cagle Interview.

“Eight days later I get a phone call. ‘Matthew. Turn on Hard Copy. Your mom’s on the show. She’s got a cameraman walking through your bedroom. She just said that you slept with this girl in this bed.'”

Sure enough, the actor turned on the TV and found his mom giving a tour. “I call and I go, ‘Mom, what are you doing?’ She was like, ‘But he was such a nice guy’,” McConaughey, 46, says.

“Now I can laugh about it, [but] back then I was like, ‘Hang on, let me figure this out first.'”

No hard feelings, of course. Like the good son that he is, McConaughey still calls K-Mac every Sunday.

Check out the full interview between McConaughey and Cagle above.