A plaque on the wall proclaims that this structure, built in 1833, is the oldest Catholic church building in Manhattan. That's partially true. As we learned yesterday, the Church of the Transfiguration was built in 1801, making it older, but it did not become a Catholic church until 1853, 20 years after St. Joseph's was built. Additionally, St. Peter's was the first Catholic church built in Manhattan, but the original structure was replaced with a new church in 1838.
So, to be precise, St. Joseph's is Manhattan's oldest surviving Catholic church building that was built as a Catholic church.
UPDATE: I should also mention that St. Patrick's Old Cathedral was completed in 1815, well before St. Joseph's. Perhaps it is excluded from the rankings because of the damage it sustained in an 1866 fire (although the extent of the destruction varies greatly from one account to another; a recent piece in the NY Times simply says the building needed to be restored, but an 1884 Times article reports that it had to be rebuilt).