Otto, as I'm sure you know, most NYC passenger terminals and stations were hugely out-of-scale in relation to the available business in the mid-1950s.
Worst of all, in NY State, NYC's properties were taxed on the basis of replacement costs for all in-place improvements, including structures, so - in effect - NYC was paying a premium in property taxes that reflected cumulative facility investments of prior generations. Over the years, a very few initiatives have brought limited relief, but on balance the problem remains to the present.
I have heard the same folklore accounts of structure immolation. I know that it was not unusual to offer a structure to a local fire department, for use as a training exercise, which reduced the cost of demolition.
By the 1980s, when I was a Division Superintendent / GM, the fire training exercise had become a hard sell, but tax (in NY especially) and property management issues remained. We tried, with some success, to give them away (Chatham). Others were more of a problem because they were too close to active trackage but they were donated and moved (Chester, MA). We offered others that found no takers (Huntingdon, PQ) and - with regrets - these were the "unsaved".
The NY tax effects cannot be understated. In the 1990s we had a certain branch line in Upstate NY where the property taxes exceeded the GROSS revenue (on-branch + off-branch) of the line, resulting in a decision to sell the line for a token amount (we did not favor the conventional abandonment/salvage approach on active branches, as such rail was rarely suitable or needed for our reuse).