John Thomas Mercer was born February 7, 1830 in Taliaferro County, Georgia to Dr. Leonidas Bennington Mercer and Lovicia Janes Mercer. John received an appointment to West Point in 1850, and graduated in 1854. Upon graduation, he was commissioned a Lieutenant in the United States Army in the First Dragoons, and was sent out to the American West.

First Lieutenant Mercer served honorably in the U. S. Army in California until his home state of Georgia seceded from the Union. Lt. Mercer resigned his commission and reported to Montgomery, Alabama to tender his services to the Confederate Army. He was subsequently given command of the 21st Georgia State Infantry with the rank of Colonel. Colonel Mercer led his men in battles at Winchester, Virginia, Cross Keys, Malvern Hill, and was mentioned for gallantry at the Battle of Chancellorsville and Gettysburg.

In early 1864, the 21st Georgia, led by Colonel Mercer was assigned to Dole's Brigade, was sent to New Bern, then on to Plymouth, North Carolina in April of 1864, to engage the enemy, and take back control of the town on the Roanoke River, from Union forces who captured Plymouth in 1862.

On April 18, 1864, Colonel Mercer led an assault on Fort Sanderson, near Welch's Creek Swamp, and received a fatal gunshot wound in the head. Moments later, Fort Sanderson was taken by the Confederates, and Plymouth was once again, under the control of the Confederate Army. Colonel Mercer was the highest ranking officer killed in the Battle of Plymouth.

All of Colonel Mercer's family lived in Georgia, except for a distant cousin, and West Point classmate, General William Dorsey Pender, who was wounded at Gettysburg, and died days later from his wound. Colonel Mercer was taken to Goldsboro, then on to Tarboro, North Carolina, and laid to rest next to General Pender at Calvary Episcopal Church.