The late federal prosecutor Jeremiah O’Sullivan was the federal agent who allegedly gave the notorious Irish American gangster James “Whitey” Bulger immunity to commit his reign of terror, his lawyers revealed this week.

According to the Boston Herald, attorney J.W. Carney, Jr made the allegation in a court filing on Wednesday in which he again called for a US District Court judge to recuse himself from presiding over the case.

Carney said the judge has an apparent conflict of interest as a former prosecutor who worked at the same time as O’Sullivan, a former US attorney who died in 2009 at age 66.

Carney added that he may call US District Court Judge Richard G. Stearns and other former prosecutors as witnesses to testify about 'the leeway that the leadership within the US attorney’s office gave Bulger,' and their failure for years to charge him with any crimes, which he said would speak to the immunity agreement that Bulger claims he had.

Judge Stearns was a former federal prosecutor and chief of the criminal division during part of Bulger’s alleged heyday in the 1970s and 1980s. But the judge was not part of the New England Organized Crime Strike Force that had an apparent relationship with Bulger at the time and he has maintained he did not know that Bulger was the target of any investigation.

Carney countered that there was no line dividing work between the Strike Force and the US attorney’s office and so prosecutors from both units shared and were aware of investigations.

'I have no doubt whatsoever about my ability to remain impartial at all times while presiding over the case,' the judge said in his ruling, maintaining that he had no knowledge 'of any case or investigation' in which Bulger was 'a subject or a target.'

But Carney said Bulger’s reputation was well known particularly among leaders in the US attorney’s office.

He also said that Bulger, now 83, will testify to support his claim and will provide 'a detailed account of his receipt of immunity by O’Sullivan,' who was a member of the strike force and at one point its chief.

Brian T. Kelly, one of the prosecutors in the case, wrote a letter to Carney on Friday in which he said the government has provided defense counsel with ample materials pertaining to O’Sullivan, calling it typical procedure in the case.

'The First Circuit has already held that O’Sullivan was unaware of any promise of immunity,' Kelly said. 'O’Sullivan himself testified under oath before Congress that he never extended immunity to either James Bulger or Stephen Flemmi.'

Bulger was one of America’s Most Wanted criminals until his arrest in June 2011 after 16 years as a fugitive. He is accused in a federal racketeering indictment of participating in 19 murders.