Many art world insiders viewed the competitions with distrust. “Some people were enthusiastic about it, but quite a few were standoffish,” Stanton says. “They didn’t want to have to compete, because it might damage their own reputations.” The fact that the events had been initiated by art outsiders, rather than artists, musicians or writers—and the fact that all entries had to be sport-themed—also led many of the most prominent potential entrants to decide the competitions were not worth their time.

Still, local audiences enjoyed the artworks—during the 1932 Games, nearly 400,000 people visited the Los Angeles Museum of History, Science and Art to see the works entered—and some big names did enter the competitions. John Russell Pope, the architect of the Jefferson Memorial, won a silver at the 1932 Los Angeles Games for his design of the Payne Whitney Gymnasium, constructed at Yale University. Italian sculptor Rembrandt Bugatti, American illustrator Percy Crosby, Irish author Oliver St. John Gogarty and Dutch painter Isaac Israëls were other prominent entrants.

In 1940 and 1944, the Olympics were put on hold as nearly all participating countries became embroiled in the violence and destruction of World War II. When they returned, the art competitions faced a bigger problem: the new IOC president’s obsession with absolute amateurism. “American Avery Brundage became the president of the IOC, and he was a rigid supporter of amateur athletics,” Stanton says. “He wanted the Olympics to be completely pure, not to be swayed by the weight of money.” Because artists inherently rely on selling their work for their livelihood—and because winning an Olympic medal could theoretically serve as a sort of advertisement for the quality of an artist’s work—Brundage took aim at the art competitions, insisting they represented an unwelcome incursion of professionalism. Although Brundage himself had once entered a piece of literature in the 1932 Games’ competitions and earned an honorable mention, he stridently led a campaign against the arts following the 1948 Games.

After heated debate, it was eventually decided that the art competitions would be scrapped. They were replaced by a noncompetitive exhibition to occur during the Games, which eventually became known as the Cultural Olympiad. John Copley of Britain won one of the final medals awarded, a silver in 1948 for his engraving, Polo Players. He was 73 years old at the time, and would be the oldest medalist in Olympic history if his victory still counted. The 151 medals that had been awarded were officially stricken from the Olympic record, though, and currently do not count toward countries’ current medal counts.

Still, half a century later, the concept behind the art competitions lingers. Starting in 2004, the IOC has held an official Sport and Art Contest leading up to each summer Games. For the 2012 contest, entrants sent sculptures and graphic works on the theme of “Sport and the Olympic values of excellence, friendship and respect.” Though no medals are at stake, winners will receive cash prizes, and the best works will be selected and displayed in London during the Games. Somewhere, the Baron Pierre de Coubertin might be smiling.