According to The Associated Press, the fire was reported around 5 a.m. ET at the now-vacant Concord Chemical plant in Camden, across the Delaware River from Philadelphia.
Hazmat crews were sent to the scene to investigate whether any chemicals remained on the site.
WPVI reported that nearby homes were being evacuated.
The Associated Press quoted a Camden County Department of Public Safety official as saying that part of the building had collapsed.
According to The Weather Channel, winds were "calm" in the area on Sunday morning.
The incident came just days after flames leveled an abandoned tire business and most of the two surrounding city blocks in Camden.
Three firefighters and one civilian were hospitalized for smoke inhalation after Thursday's blaze. That fire damaged at least 23 buildings.