Taylor was born 4 May 1796 in Louisville, Kentucky. He enlisted as a private to fight in the War of 1812 and rose to the rank of lieutenant winning a commission on 20 May 1813. He was discharged on 15 June 1815 when the war ended. He was reinstated in the regular army as a lieutenant of artillery on 17 May 1816 and was promoted to captain on 6 July 1825. In 1829 Taylor was reassigned to the commissary department. He was brevetted colonel on 30 May 1848 for his performance in supply during the Mexican War, then was promoted to full colonel on 29 September 1861.
As assistant commissary general Taylor was responsible for supplying Union forces in both theaters of the war. When Colonel George Gibson, the commissary general, died a few months after the beginning of the war, Taylor became commissary general. On 9 February 1863 he was promoted to brigadier general when Congress passed an act stipulating that the commissary general hold the rank of brigadier.
Taylor died on 29 June 1864 while at his post in Washington DC. Joseph Taylor's brother was Zachary Taylor, the 12th president of the United States. Joseph's nephew was Confederate Lieutenant General Richard Taylor, Zachary's son. Joseph's niece was Sarah Knox Taylor, Zachary's daughter and the first wife of the eventual president of the Confederacy, Jefferson Davis.