After graduating from Harvard in 1927, Rorimer went to work at the Metropolitan Museum of Art in New York. He helped develop the museum’s medieval art collection, including the famous Cloisters, which he became curator of in 1938. In 1943, Rorimer joined the army infantry as a private and soon was transferred to the Monuments, Fine Arts, and Archives section (MFAA), which rescued historical and cultural artifacts across Europe. He rose to head the MFAA of the Seventh US Army, Western Military District. Rorimer most notably was involved with the seizing of art at Neuschwanstein and the Heilbronn mines, as well as the art collections of Göring, Goebbels, and Alfred Rosenberg. After the war, Rorimer returned to the Metropolitan Museum of Art, eventually becoming its director in 1955; he served in that position until his death in 1966.