The first fortifications were begun by the French in 1759 during the Seven Years' War to resist the advance of the British, who captured and destroyed the fort the following year. In 1775 the island was occupied by American troops under generals Richard Montgomery and Philip Schuyler. After the Americans evacuated the island in 1776, the British built stronger fortifications. During the War of 1812, the British established a shipyard and the island became a naval base.
Between 1819 and 1829 a new fort was constructed; the complex was named for Charles Lennox, Duke of Richmond. The fort was garrisoned for many years, but the British finally abandoned it in 1870. Among the massive stone buildings are the officers' quarters, guardhouse, powder magazine, barracks and commissary. Narrated tours are available. Picnicking is permitted.