Sperry Breakwater Lighthouse as seen in a 1901-1907 era postcard. The light was built in 1899, then torn down in 1933 and replaced by a skeleton tower.
The harbor is protected from Long Island Sound by a peninsula from its western side, once known as "Little Necke" but now called Lighthouse Point, because of the lighthouse that was constructed on its tip in 1805. The original lighthouse was replaced in 1845 by the current structure, called the Five Mile Point Lighthouse. This lighthouse was replaced for navigation in 1877 by the offshore Southwest Ledge Light. Sperry Lighthouse (1899-1933) also served the harbor.
The harbor is circumnavigable via the partially-completed "Harborside Greenway" bicycle and pedestrian trail, which is part of the East Coast Greenway system.