Ole Herman J. Krag was a student at the Hartvig Nissens skole in Oslo, and started his Norwegian Army military career in January 1854. He became a Second Lieutenant in 1857 and a full Lieutenant in 1861. In 1866 he was ordered to the Kongsberg Våpenfabrikk, the most important Norwegian weapons factory of its day.

Ole H. J. Krag constructed his first repeating rifle in 1868, which evolved into the Krag-Petersson (aided by Axel Petersson) by 1872. This, as well as his work in the armoury, gave him a thorough understanding of what requirements the Norwegian Army had to a rifle, allowing him to create the successful Krag-Jørgensen with the help of his good friend Erik Jørgensen.

He designed a wide range of firearms during his lifetime, but only two: the Krag-Petersson and the Krag-Jørgensen – were adopted by any armed forces. Less than 1,000 Krag-Petersson rifles were made for the Royal Norwegian Navy, while several hundreds of thousands Krag-Jørgensen rifles were made for the Danish, Norwegian, and U.S. armies.

The United States military held a rifle competition in 1892, comparing 53 rifle designs including Lee, Krag, Mannlicher, Mauser, and Schmidt-Rubin. The trials were held on Governors Island in New York harbor, and the finalists were all foreign manufacturers - the Krag, the Lee, and the Mauser. The contract was awarded to the "Krag" design in August 1892. The Krag-Jørgensen bolt action rifle was designated the M1892 by the U.S. Army.

He married in 1870 with Karen Elise Theodora Collett (1844-1926). They were the parents of lawyer Herman Anton Johan Krag (1871–1931), and were the grandparents of Norwegian architect Herman Krag (1920-1982).

In 1880 he was named director of the armoury. After the Krag-Jørgensen was accepted as the main rifle of the Norwegian Army, he was made a Lieutenant Colonel in 1894.

He retired in 1902. He is honored in Kongsberg with a street named Krags gate.

Ole H. J. Krag died in Paris in December 1916 and was buried in Oslo during January 1917.

Ole Herman Johannes Krag