Former vice president and frontrunner for the Democratic party’s presidential nomination, Joe Biden, faced pushback for comments he made at a fundraiser in New York Tuesday night — remarking on his ability to work with senators who had opposed desegregation and espoused white supremacy.

“I was in a caucus with James O. Eastland,” Biden said. “He never called me ‘boy,’ he always called me ‘son.’”

Eastland was an avowed white supremacist senator from Mississippi, who served alongside Biden in the U.S. Senate. Eastland consistently voted against civil rights legislation and said the Brown v. Board of Education of Topeka decision desegregating public schools “destroyed” the U.S. Constitution.

“At least there was some civility,” Biden said.

The former vice president continued reminiscing about how he worked with white supremacist senators.

“We didn’t agree on much of anything. We got things done. We got it finished, but today you look at the other side, and you’re the enemy,” Biden said.

Sen. Cory Booker, D-N.J., one of Biden’s rivals for the Democratic nomination, issued a statement denouncing Biden’s comments and calling on him to apologize.

“You don’t joke about calling black men ‘boys.’ Men like James O. Eastland used words like that, and the racist policies that accompanied them, to perpetuate white supremacy and strip black Americans of our very humanity,” Booker said in the statement.

The statement goes on to call Biden’s relationships with “proud segregationists … not the model for how we make America a safer and more inclusive place for black people, and for everyone.”

Booker was joined by the other African American senator seeking the Democratic nomination, Kamala Harris of California, in criticizing Biden’s remarks.

“If those men had their way, I wouldn’t be in the United States Senate and on this elevator right now,” Harris said to ABC News when asked about the comments Biden made.

Meanwhile, New York Mayor Bill de Blasio, another of Biden’s primary rivals, tweeted “It’s past time for apologies or evolution from [Biden]. He repeatedly demonstrates that he is out of step with the values of the modern Democratic Party.”

“Senior Advisor” to the Biden campaign, Symone Sanders, tweeted a response to the controversy.

“[Biden] did not praise a segregationist. That is a disingenuous take. He basically said sometimes in Congress, one has to work with terrible or down right racist folks to get things done. And then went on to say when you can’t work with them, work around them,” Sanders said.

Despite the criticism from his rivals, Biden maintains a wide lead over the Democratic primary field and an even wider lead among nonwhite voters.

A Monmouth poll released Wednesday found Biden leading his closest rival Sen. Elizabeth Warren, D-Mass., by a 32-15 margin among all Democratic voters and a 33-10 margin among nonwhite voters.