He played college football at Texas A&M University where he led the Aggies in rushing with 371 yards in 1961. He played on the 1963 College All Star Football Team and in the 1963 Challenge Bowl and is a member of the Texas A&M Hall of Fame. He was selected in the seventh round of the1963 NFL Draft by the Philadelphia Eagles and was named All Rookie. Caffey was acquired by the Green Bay Packers in the famous 'Jim Ringo' trade where he became a starting linebacker and would become an All-Pro Player. With 10.0 speed in the 100-yard-dash he was one of the fastest linebackers in the league at that time. He played on three of Vince Lombardi's championship teams at Green Bay which include the 1965 World Championship, Super Bowl I, where he led with 7 tackles, and Super Bowl II. Caffey was named AP and UPI All Pro in 1966, played in the 1966 Pro Bowl, and played in the legendary 'Ice Bowl' in 1967.
He is a member of the Green Bay Packer Hall of Fame and was selected to the All Time Packer Team, the Texas High School All Super Bowl Team, and was nominated for ESPN's All Time Super Bowl Team. In 2006, the Green Bay Packers' linebacking corps of Ray Nitschke, Dave Robinson, and Lee Roy Caffey was named one of the NFL's Top 10 Greatest Linebacking Trios of All Time. Caffey later played for the Dallas Cowboys' first Super Bowl championship team (Super Bowl VI) under Tom Landry where he earned his third career Super Bowl ring and fourth championship ring.
Lee Roy Caffey died of colon cancer at MD Anderson Cancer Hospital in Houston, Texas in 1994.