WASHINGTON (AP) — Rep. Matt Gaetz, facing accusations of a sexual relationship with an underage girl, should at a minimum be removed from the House Judiciary Committee if the claims are true, House Speaker Nancy Pelosi said Thursday.

Pelosi, D-Calif., also said the House Ethics Committee should consider the allegations against the Florida Republican.

Gaetz, 38, who has been one of former President Donald Trump’s closest allies since coming to Congress in 2017, said anew Thursday that the accusations are wrong.

Asked directly if Gaetz should resign from Congress or lose his committee seat, Pelosi didn’t specifically address the more severe step.

“If in fact these allegations are true, of course being removed from the Judiciary Committee is the least that could be done,” Pelosi told reporters. She added, “From what we’ve heard so far, this would be a matter for the Ethics Committee.”

The Ethics panel, made up of five members from each party, can recommend punishments ranging from a reprimand, or formal rebuke, to expulsion, which is rare partly because lawmakers often resign first. The full House would have to approve such actions, with expulsion requiring a two-thirds majority.

Pelosi’s comments fell short of the swift removal from the Judiciary Committee sought by at least one Democratic lawmaker. But the remarks still left the political future clouded for Gaetz, who is under federal investigation over accusations that he had a sexual relationship with a 17-year-old girl and paid her to travel with him, two people familiar with the matter have told The Associated Press.

Pelosi’s remarks came about two months after the House, on a mostly party-line vote, took the unprecedented step of removing Rep. Marjorie Taylor Greene, R-Ga., from both her committees. That was in reaction to her history of spreading threatening and false conspiracy theories on social media prior to taking office.

The Justice Department has also been examining whether Gaetz has had relationships with other underage girls, the people said. Investigators are trying to determine whether Gaetz violated federal sex trafficking laws, said the people, who were not authorized to discuss the matter publicly and spoke on condition of anonymity.

The Judiciary Committee oversees the Justice Department. On Wednesday, Rep. Ted Lieu, D-Calif., tweeted that Gaetz should not be “sitting on the Congressional Committee that has oversight over the Department that is investigating him.”

Gaetz has so far received little vocal support from his fellow Republicans.

House Minority Leader Kevin McCarthy, R-Calif., said Wednesday that the accusations were “serious” and that if proved, Gaetz would be removed from committees. He’s also on the Armed Services Committee.

“The allegations against me are FALSE,” Gaetz tweeted Thursday. He also repeated his assertion that the charges are related to an alleged attempt to extort his family.