July 13, 1863 the New York City Draft Riots began
Massive New York City protests decrying the first-ever wartime draft lottery led to a bloody riot as a mob of 50,000 burned buildings (including the Colored Orphan Asylum on Fifth Avenue), stores and draft offices, and attacked police. Some clubbed, lynched, and shot large numbers of blacks, whom they blamed for the war.

By the time troops returning from Gettysburg finally restored order, 1,200 had died over five days.
New Yorkers, spurred on by the Democratic leadership of Tammany Hall and tired of the seemingly endless war, had been angered by Pres. Abraham Lincoln’s recent call for 300,000 more troops. They especially resented the legal provision allowing a cash payment ($300 commutation fee) as a way for those with the means to avoid military service in the Union Army. White workers compared their value unfavorably to that of southern slaves, stating that “[we] are sold for $300 [the price of exemption from war service] whilst they pay $1000 for negroes.”

Please visit this link for more on the 1863 Draft Riots

The above pic is an illustration of the burning of the Colored Orphan Asylum from The Illustrated London News August 15, 1863.