South Amboy (Brown's Yard) sits along the Amboy Secondary, which runs northeast (railroad direction east) and southwest (railroad direction west). There are two ways out of Brown's Yard:
1. If the SA-31 is going to Red Bank / Lakewood, they will exit the yard by going Railroad East. It will then need permission to travel down New Jersey Transit's North Jersey Coast Line from South Amboy through Matawan, Hazlet, Holmdel, and Middletown. Just south of the NJT station in Red Bank is the switch to enter Conrail's Southern Secondary. This line extends through Shrewsbury, Eatontown, Tinton Falls, Howell, Farmingdale, Lakewood, and Lakehurst.
2. If the SA-31 is going to Freehold, it will go railroad Railroad West through Spotswood, Helmetta, and Jamesburg. At Jamesburg there is a fancy bit of trackage called a "wye" (pronounced like the letter "Y") that allows the train to enter the Freehold Secondary. The train will then proceed through Monroe, Manalapan, Englishtown, Freehold Borough, and Freehold Township.
Back in the olden days, this line through Freehold was called the "Freehold & Jamesburg Agricultural Railroad". It continued through Howell, Farmingdale, and Wall, and connect to the New York & Long Branch Railroad (NY&LB - now the North Jersey Coast Line) at Sea Girt.
As you may have read on this board, there is a stretch of track between Freehold and Farmingdale that is out of service (meaning that it is unsuitable for trains to travel over it). East of Farmingdale, the track has been torn up, and part of it was made into a trail in Allaire State Park. If you look at a satellite photograph, you can still make out the old right-of-way.
As a bonus explainer: there was another line that ran from Freehold Borough up through Marlboro to Matawan, where it would connect to the NY&LB just west of the current NJT station. The rails have been torn up from this line, and it is now part of the Henry Hudson Trail.
Be careful when exploring these areas. Remember that the right of way (and most of the land adjacent to it) is private property. Not only do you need to worry about trains, but you also need to be mindful of unstable turf, and of wildlife (from ticks and chiggers to foxes or feral dogs). You may also encounter other people, and they may not be as friendly or interested in the railroad as you are. Don't travel far from the road alone. And if you're anywhere near Earle, be careful not to stray onto federal property.
If I've made any factual errors, please correct me.