Believe it or not, you can still occasionally spot a train chugging down the center of First Avenue here in Sunset Park (video). I've even seen a locomotive have to stop and wait in the road because someone left a car parked on the track, presumably not considering that the rails might still be active. All the engineer could do was honk his horn until the driver came and moved the car.
This line is operated by New York New Jersey Rail (NYNJ), the last railroad still floating freight cars to and from New Jersey across New York Harbor (video*). It's a small-time operation, but NYNJ's car floats do provide a convenient alternative to the roundabout rail-only route known as the Selkirk Hurdle, so named because a freight train bound from, say, New Jersey to Brooklyn must travel all the way up to Selkirk — some 140 miles north of here — in order to cross the Hudson River by rail. There used to be a rail bridge over the Hudson at Poughkeepsie, much closer to NYC, but it was shut down after a fire in 1974. (It has since reopened to foot traffic and is now the longest pedestrian bridge in the world!)
In 2012, NYNJ shifted the Brooklyn terminus of its car float operation from the Bush Terminal rail yard (a few blocks north of here, i.e. behind me as I was taking this photo at about 54th Street) to the 65th Street Yard (about half a mile south of here). As the 65th Street Yard is directly connected to the Bay Ridge Branch freight line, freight cars floated across the harbor no longer have to move down First Avenue to get to the Bay Ridge Branch. But the First Avenue track still sees some use: it provides a connection between the 65th Street Yard and the recently rehabilitated South Brooklyn Marine Terminal, where rail service to the new Sims Municipal Recycling Facility began in February, and it also allows new and rebuilt subway cars that have been floated over from New Jersey to be hauled up First Avenue (photo) toward a rail connection** with the subway system around 38th Street and 4th Avenue. (Similarly, old subway cars can be sent down the line and barged across the harbor for scrapping or rebuilding.)
* The video, from 2012, is already a bit out of date. It was shot before the more modern landing site at the 65th Street Yard in Brooklyn opened. Additionally, the old Brooklyn pontoon bridge shown in the video has been relocated to the New Jersey landing, which was badly damaged by Hurricane Sandy.
** This is one of only two connections between the subway system and the national rail network. The other, as we've learned, is near the Linden Shop in eastern Brooklyn.