“After being married for a few weeks, they both realized that they actually felt like husband and wife without having a religious ceremony,” a source tells PEOPLE about Bieber, 24, and Baldwin, 22, who secretly wed in September, just two months after getting engaged.
“They want to openly celebrate their marriage,” the source says. “This is why Hailey decided to change her name. She loves being married to Justin. They are still planning a wedding and a religious ceremony.”
The model recently changed her Instagram handle to “Hailey Bieber” and even showed off a new diamond necklace of her husband’s surname. And days before changing her name on social media, she also wore a custom Levi’s trucker jacket with her new married name splashed across the back at the opening of the brand’s new Times Square Flagship store in N.Y.C.
The pop star confirmed their marital status the same day, referring to his bride by her new title for the first time publicly. “My wife is awesome,” he captioned a photo of the pair smiling and holding hands.
RELATED VIDEO: Inside Justin Bieber and Hailey Baldwin’s secret ceremony: ‘They didn’t want anyone to know’
In addition, the Biebers also celebrated their first Thanksgiving together as a married duo, as well as her Nov. 22 birthday in his native country.
“They had a small birthday celebration for Hailey with his family and spent the rest of the weekend relaxing in Canada,” the source says. “Justin loves being in Canada because it’s very peaceful. He can take Hailey out and it’s not a mess with photographers. They are much more left alone. They seem very happy together.”
Bieber’s mom, Pattie Mallette, also shared happy family photos and videos from their celebrations, including footage of her son smashing a huge sheet cake into his wife’s face.
During the holiday weekend, the pop star reflected on how much it meant to him to be celebrating his first Thanksgiving as a married man.
“First Thanksgiving as a married man, first time hosting Thanksgiving. First time both sides of the family has come together. Relationships are hard and love isn’t always easy but thank you Jesus for showing me how,” Bieber captioned a photo of a snowy forest on Instagram. “Every day is a learning process, trying to be more like him, patient, kind, selfless, boy I got a long way! But gods grace is sufficient!”
On Saturday, the pair stepped out together at a hockey game in Toronto, where they weren’t shy about packing on the PDA. After being spotted by photographers, Bieber, who was standing nearby his wife, crossed over to her and planted a sweet smooch on her lips.