So what in the world are all these gravestones doing out here? It turns out that this peninsula I've been scrambling around, with the residential developments built on top of it, is a former landfill that was in operation from about 1963 to 1977. (Compare aerial images from 1951 and 2012 to see the extent of the landfill.) All sorts of things were dumped here over the years, including lots of stones from cemeteries.
Because we treat death and the burial process with such sanctity, it's quite shocking to find these stones haphazardly strewn about the waterfront, and one's immediate sense is that there must be something scandalous going on. But, of course, sometimes gravestones do need to be replaced, and the old ones have to be discarded somewhere. It's just that, in this case, they ended up on the periphery of a landfill that wasn't fully capped, and hence are still exposed and visible decades later. (I should note, however, that they're not exactly in public view. They can only be reached by those who decide for some reason to climb around a steep, rocky stretch of shoreline.)