Each pillar supports a part of the building that collapses when that pillar is destroyed. A first-floor pillar can’t be destroyed before a second.
Wait a little while after the whole mansion is destroyed, and with a mighty rumble, it rises from the ground, good as new!