Smokey Robinson is speaking out after he was accused in a civil lawsuit of sexual assault by four former employees.

In a phone call with the Daily Mail on Wednesday, May 7, the Motown star, 85, briefly addressed the allegations put forth against him in a complaint filed in Los Angeles on Tuesday, May 6.

“I am appalled,” Robinson said, adding, “I can’t speak about this right now.”

An attorney for the star did not immediately respond to PEOPLE’s request for comment.

The former Miracles frontman was accused of sexual battery, assault, false imprisonment, gender violence and creating a hostile work environment by four Jane Does who worked for the star and his wife Frances over a period of nearly 20 years.

The complaint claimed Robinson “repeatedly forced his fingers and penis into their vaginas, causing severe and excruciating pain” without consent, and also alleged that he would use “physical barriers, force, threats of force” and more, such as locking his bedroom door, to prevent the women from leaving during the alleged assaults.

Frances, whom Robinson married in 2002, is also named in the lawsuit, as the plaintiffs allege that she had “full knowledge of his prior acts of sexual misconduct” and “failed to take the appropriate corrective action” to prevent his “deviant misconduct,” even after settling cases with other women who experienced similar alleged assaults by Robinson.

The plaintiffs — who all worked as housekeepers — also accuse the couple of failing to pay minimum wage and overtime.

Jane Doe 1 began working for Robinson in January 2023, and claims she was assaulted by the star at least seven times before she resigned in February 2024, according to the complaint. She and the three other plaintiffs say they did not report his alleged assaults to authorities for fear of losing their “livelihood, familial reprisal, public embarrassment, shame and humiliation,” as well as “being threatened and intimidated” by Robinson’s celebrity status.

Jane Doe 2 worked for Robinson from May 2014 to February 2020, and claims she was assaulted by the star at least 23 times. Per the complaint, Robinson allegedly “raped” her and also forced her to “perform oral sex on his erect penis.”

Jane Doe 3 says in the complaint that Robinson “raped” her “at least 20 times” during her tenure as a housekeeper, from February 2012 to April 2024.

Per the complaint, Jane Doe 4, meanwhile, worked for Robinson from October 2006 to April 2024, when she resigned due to the alleged sexual assaults and sexual harassment. Jane Doe 4, who also worked as Frances’s personal assistant, cook and hairdresser, says in the complaint that Robinson first “forced her into his bedroom and raped her” in 2007, and that such assaults “continued” before she quit.

The plaintiffs are seeking damages of at least $50 million.