Andrew Pickens Calhoun was born on October 15th, 1811, and was the eldest child of John C. and Floride Calhoun. He was named for Revolutionary War hero General Andrew Pickens, his maternal uncle by marriage. A.P.attended Yale University and South Carolina College. He married twice: first, his cousin, Eugenia Chappell, daughter of Joe Chappell and Sophia Maria Green Chappell; after Eugenia's death, Andrew married Margaret Green. Margaret and A.P. had eight children together and after A.P.'s death in 1865, Margaret and the children continued to live at Fort Hill for a short time.

After his marriage to Margaret in 1836, A.P. began purchasing land in Marengo County, Alabama. Personal correspondence indicates that the land would become a cotton plantation that would be managed by A.P. The enslaved persons and equipment needed for the operations of the plantation would be provided by his father, John C. Calhoun. This prompted what would become an ongoing exchange of enslaved persons between Fort Hill and the Alabama Plantation. In Alabama, the enslaved African Americans were expected to clear the land and plant and harvest the cotton purchased by A.P. with money given to him by his new brother-in-law, Thomas Green Clemson. He continued to increase his land in Alabama by expanding his Tulip Hill plantation to include the neighboring Cuba plantation. Even after moving back to Fort Hill with his family in 1854, he continued to expand his property by purchasing land from neighbors. The labor of the plantation was performed by the enslaved persons A.P. was continuously provided from the enslaved populations at Fort Hill and John C. Calhoun's goldmine in Dahlonega, G.A. In 1840, there were eighty-six enslaved persons recorded as being on the plantation at Marengo County, and by 1850 that population had grown to 116. After owning Tulip Hill plantation for twenty years, A.P. sold the land and the enslaved population to Goetlib Breightling.

On March 19th, 1865, Floride Clemson wrote in her diary that her uncle, A.P., had passed from congestion of the heart that past Thursday. The enslaved community at Fort Hill were listed as "property" on the estate inventory performed after his death. It listed a total of 139 enslaved persons as living at Fort Hill and provided some details including their names and ages. 127 of those enslaved at Fort Hill at this time were listed as field laborers and twelve were classified as enslaved domestic workers or skilled laborers. In 1865, Nicholas was the Blacksmith, Ted was the carpenter, Isaac was the gardener, Christy was the cook, Billy was the miller, and Priest was the coachman. Documents such as wills and estate inventories may be hard to view as they list African Americans as property, but they can offer insight into the lives of those who were enslaved and therefore blocked from being able to tell their story for themselves.

By September of 1865, after the end of the Civil War, there were fifteen freed persons of color at Fort Hill; their names were not recorded. The earliest known Freedman's contract was signed between Harriet, Jim, Peter, Abb, and A.P.'s son, John C. Calhoun, II. Starting in 1867, freed persons of color, including children, signed contracts with Thomas Clemson.

“The African American Experience at Fort Hill | Clemson University, South Carolina,” accessed May 21, 2020, https://www.clemson.edu/about/history/properties/fort-hill/african-americans1/.

“Andrew Pickens Calhoun (1811-1865) | WikiTree FREE Family Tree,” accessed June 17, 2020, https://www.wikitree.com/wiki/Calhoun-126.

“Educational Resources | Clemson University, South Carolina,” accessed May 21, 2020, https://www.clemson.edu/about/history/properties/ed.html.

“Historic Properties | Clemson University, South Carolina,” accessed May 21, 2020, https://www.clemson.edu/about/history/properties/.