The Home Secretary Suella Braverman is facing a potentially major court battle over the safety of women and children at the Manston centre in Kent.
Details of the first legal case were released by lawyers as she toured the overrun facility on Thursday.
The action, which is at its earliest stages, alleges ministers left a woman in "egregiously defective conditions".
If the home secretary does not address the allegations, judges will be asked to hear the case in full.
The Home Office said it would respond to the claim but would not comment publicly on legally proceedings.
In the complaint, lawyers tell the Home Office the woman bringing the case comes from outside of Europe and she was taken to the Manston facility last month.
Manston was set up as a short-term holding facility, with no overnight accommodation, meaning officials cannot normally detain anyone there beyond 24 hours
Instead, the woman was held for three weeks. She's now been released after the Home Office permitted her to go and stay with a family member.
The legal action against Ms Braverman, also involving the charity Detention Action, alleges there is evidence of:
* Routine and prolonged unlawful detention in the facility
* A failure to implement "essential safeguarding measures" for children
* Adult men sleeping alongside women and children they are not related to
* Migrants exposed to infectious diseases amid poor sanitation
* No means for the detainees to get legal advice about why they are being held
Detention Action said its case was focused on ending "serious threats" to the safety of children.
"We have taken this action out of serious concern for the welfare of thousands of people, including children, still being detained at Manston for periods far beyond legal limits," said Jamie Wilson, deputy director of the charity.
"We are calling on the home secretary to declare that anyone held at Manston for more than 24 hours is being detained unlawfully.
"We are also asking that the home secretary allow access to the facility for organisations qualified to provide support in immigration detention settings."
A judgement against the government could lead to thousands of compensation claims for unlawful detention.
On Thursday, Ms Braverman toured the migrant facilities in Kent - although officials told the media there would be no opportunity for journalists to ask her about what she had seen.
The visits came as she has faced increasing pressure over how she has handled the numbers of people arriving in Kent.
Albania's Prime Minister Edi Rama has accused the UK of scapegoating people from his country to hide what he said had been policy failures and a government acting "like a madhouse".
The chairs of four parliamentary committees have also demanded that Ms Braverman explains how her department is going to fix the "dire" conditions at Manston.
A Home Office spokesperson said it would be responding to the legal claim - but would not comment publicly about legal proceedings.
They added: "Manston remains resourced and equipped to process migrants securely and we will provide alternative accommodation as soon as possible."
Watch in full: UK using migrants as scapegoats - Albanian PM Edi Rama