Overnight, Kasumigaseki Country Club (pictured), the proposed venue of the 2020 Olympic golf competition, postponed its vote regarding the club’s membership policies towards women. Now, according to a Golf.com report, the failure to repeal the discriminatory practice could lead the International Golf Federation to find a new home for the Summer Games.

“The IGF has clearly stated to both Tokyo 2020 and Kasumigaseki CC our requirements that the golf competition be delivered according to the Olympic Charter,” IGF vice president Ty Votaw told the website. “If the club does not change its rules, then we cannot support holding the events at this venue.”

Though it does have female members, women are currently not granted full rights at Kasumigaseki, and can’t play on Sundays. According to Reuters, a unanimous vote of the club’s 15-member board was required to change the rule.

“That this situation has developed is a nuisance for us, it’s really perplexing,” Kasumigaseki board chairman Kiichi Kimura told reporters after the vote.

Kasumigaseki’s membership policies, according to Golf.com, were unknown to the IGF. The issue came to light thanks to Yuriko Koike, who recently became the first female governor of Tokyo.

“I feel very uncomfortable about women not being able to become a regular member in this era,” Koike said in January.