He was one of at least 9 children of Lightfoot Williams, formerly of Alabama, who died about 1848 in DeSoto Parish, Louisiana, and was a substantial slaveholder. Jesse was appointed Lieutenant Colonel of the 2nd Louisiana Infantry on 1 May 1861 at the unit's organization at New Orleans. He took command of the regiment after the battle of Malvern Hill (1 July) on the mortally wounding of Col Isaiah Norwood. He was promoted full Colonel 26 July 1862.
Colonel Williams was seriously wounded in the chest on the morning of 17 September 1862, in combat near where General Starke was killed during the Antietam Campaign
After Chancellorsville, in May 1863, he was appointed to command of the Brigade, and led it at Gettysbug in July. He was killed at Spottsylvania in May 1864, when his unit was overrun.