Charlie Keller, who for much of 10 American League baseball seasons from 1939 through 1948 formed with Joe DiMaggio and Tommy Henrich one of the finest outfields in New York Yankee history, died of cancer yesterday at his home in Frederick, Md. He was 73 years old.

A splendid all-round athlete at the University of Maryland, where he earned a degree in agricultural economics, Keller joined the Yankees in 1939 and quickly became the starting right fielder. As a rookie, the muscular left-handed power hitter batted .334 with 11 home runs and 83 runs batted in. He topped a splendid season by crashing three homers and batting .438 as the Yankees swept four games from the Cincinnati Reds in the World Series.

Keller, who served during 1944 and much of 1945 in the Merchant Marine, was a regular performer for the Yankees for only six seasons. He played part time from 1947 through 1949 when he was troubled by a ruptured disk in his back.

Hit 189 Home Runs

But as a regular - mostly in left field with Henrich in right and DiMaggio patrolling center field - he was a feared batsman with great power, and was more than a competent fielder. Counting two seasons he played in Detroit, where he served mostly as a pinch-hitter in 1950 and 1951, Keller's career batting average was .286 with 189 home runs and 760 runs batted in.

In four World Series, he batted .306 with 5 homers and 18 runs batted in.

At 5 feet 10 inches and 185 pounds, he was a compact man with great strength. ''Keller wasn't scouted,'' said Lefty Gomez, the antic Yankee left-handed pitcher. ''He was trapped.''

A Quiet, Private Man

Because of his strength, he was dubbed King Kong Keller, a nickname he never liked and seldom answered to. He was a quiet, private man who was reluctant to make public appearances and often felt uncomfortable in the limelight. ''I don't want people to treat me any different just because I used to play baseball,'' he said after his retirement in 1952.

Although the most money he earned in one season was $27,500, Keller managed to save enough from his baseball earnings to go into the breeding of standardbred horses - pacers and trotters - near his hometown of Middletown, Md. He eventually founded Yankeeland Farm in Frederick, Md., and had a successful career as a breeder. Many of the foals reflected his baseball heritage: Fresh Yankee, Handsome Yankee, Yankee Slugger, Gay Yankee. ''You've got to call them something, I suppose,'' he said.

He benefited by owning syndicated shares of several stallions, which entitled him to free stud fees. Many of the foals went on to have successful racing careers. But if they did not, it bothered him.

''If someone buys a horse from me and it doesn't turn out right, I feel bad,'' he once said.

He is survived by his wife, Martha; two sons, Charles 3d and Donald, and a daughter, Jean Bittle, all of Frederick.

Charles E Keller