It was on the ground (except for some bridges) from Spuyten Duyvil to about 120th St. where it entered a tunnel under Riverside Park. The tunnel continued to 72nd St and the tracks emerged at the beginning of 72nd St. yard (or 60th St yard depending on which end you reckon from!). The tunnel from 120th to 72nd Street was built in about 1936 when the late famous --- or infamous, depending on your opinion of him --- Robert Moses extended Riverside Park down to the Hudson Riverfront and built the Henry Hudson Parkway . Prior to that Riverside Park was nothing like it is today. It was a narrow strip of greenway on the west side of Riverside Drive, and the RR was all exposed.
I have that from my dad who, after he graduated college in 1932, lived the rest of the 1930s at West 98th St. and Riverside Drive. He personally witnessed the construction!
When my wife and I lived on the "upper west side" for a while she belonged to a garden club at 91st Street that was literally right above the RR tunnel (Trivia: the garden is featured in the closing romantic scene of the Tom Hanks/Meg Ryan RomCom "You've Got Mail")