The oldest building on the island is this tabby (a mix of sand, water, shells and limestone made of burnt shells, typically collected from Native American shell middens) house, a remnant of Catherine "Caty" Greene Miller's original Dungeness plantation. It was then -- early 19th century -- the gardener's house.
Though raised as a Quaker and self-educated in warfare, Nathaneal Greene was one of the most successful generals on the colonial side. After the debacles of Howe, Lincoln and Gates, Greene, executing what has been termed "one of the masterful military achievements of all time," consolidated the colonial forces in the South, destroyed all but 10% of the British forces in two major engagements and was largely responsible for forcing Cornwallis to retreat to Yorktown while isolating the remaining British units to a few coastal towns in the South, while he controlled the countryside and the rest.
Greene had borrowed heavily to raise and equip his army. To repay him, the government gave him Cumberland Island, Mulberry Grove in mainland Georgia and another plantation called Boone's Barony in coastal South Carolina.
Nathaneal died quite young of sunstroke at Mulberry Grove in 1786. (He is buried here: http://tinyurl.com/q8xzaa9) Caty who had had to learn to be independent during the war took on the management of Mulberry Grove and after marrying one tutor -- Phineas Miller -- encouraged another, a young Yale graduate named Eli Whitney to use his spare time at Mulberry Grove to invent the cotton gin. The Millers lost Mulberry Grove as a result of their investments in the cotton gin company which suffered as many early Patent Office filings did from piracy of the publicly available designs.
They moved to Cumberland Island and built a home here calling it Dungeness for the view of the wetlands beyond.