Brooke Mueller could lose her custody rights of her children with Charlie Sheen

The troubled socialite - who has twin sons Max and Bob, four, from her past relationship with the 'Anger Management' star - has worried the actor by moving out of her all-female sober living facility in San Juan Capistrano.

Now social workers are reportedly pushing for her "parental right [to] be terminated" with news she now isn't subject to random drug tests.

An insider told RadarOnline: "Brooke says she is still getting outpatient treatment at a clinic in San Jan Capistrano, but she isn't being randomly drug tested.

"Social workers are preparing a report for the judge in child dependency court and are going to recommend that Brooke's parental right be terminated, immediately."

Brooke already lost custody temporarily of Max and Bob - who are currently in the care of Charlie's other ex-wife Denise Richards - when she entered rehab and the Los Angeles County of Department of Children and Family Services are pushing for this to be permanent.

The source added: "She clearly isn't taking getting sober seriously. She claims she had to move out of the sober living house because Brooke is a celebrity and a public person and couldn't get privacy.

"This flies in the face of reason and the Los Angeles County of Department of Children and Family Services feel it's in the best interests of Bob and Max, to have Brooke's parental rights revoked."