Members of the House Oversight Committee had requested a "transcribed interview" with the former UK prince in connection with his "long-standing friendship" with the pedophile financier, who took his life in a New York prison in 2019 while awaiting trial on sex trafficking and conspiracy charges.
But after saying they had not heard from him, Democrats Robert Garcia and Suhas Subramanyam accused Andrew of hiding.
"Andrew Mountbatten-Windsor's silence in the face of the Oversight Democrat's demand for testimony speaks volumes," the committee's statement read.
"The documents we've reviewed, along with public records and Virginia Roberts Giuffre's testimony, raise serious questions he must answer, yet he continues to hide.
"Our work will move forward with or without him, and we will hold anyone who was involved in these crimes accountable, no matter their wealth, status, or political party. We will get justice for the survivors."
Andrew, who was stripped of his prince and Duke of York titles earlier this month, denies any wrongdoing.
He had previously agreed to stop using his titles but had expected to remain a prince and retain his dukedom, ahead of the publication of the memoirs of the late Giuffre, who had accused him of sexually assaulting her when she was a teenager. He denies the accusation.
Damaging newspaper allegations also include that he had tried to get Britain's Metropolitan Police to dig up dirt for a smear campaign against Giuffre.
The force previously said it was looking into the reports after The Mail On Sunday newspaper claimed he had passed Giuffre's date of birth and social security number to his taxpayer-funded bodyguard in 2011 and asked him to investigate.
It comes as The Sun reported Andrew's ex-wife Sarah Ferguson is considering offers for a television interview, with US channels reportedly said to be willing to pay six-figure sums.
1800 RESPECT (1800 737 732)
National Sexual Abuse and Redress Support Service 1800 211 028
Lifeline 13 11 14
beyondblue 1300 22 4636
23°C