Spirits of the City: The Bizarre Lives Of A Pacific Heights Mansion
By Gary Kamiya & Paul Madonna

The colossal 30-room Arizona sandstone edifice (each of its four floors is 3,500 square feet), with its mighty twin curved towers, classical pediment and facade, and vaguely medieval aura, is unique to the neighborhood, but not the City. Other Richardsonian Romanesque buildings in San Francisco include the Sacred Heart Church at 554 Fillmore Street, the Hills Brothers Coffee building on the waterfront, and the edifice that most resembles the Whittier Mansion, the sandstone Sharon Building next to the Golden Gate Park Playground.
The Whittier Mansion’s architecture is unusual, but its history is downright bizarre. It may have housed the strangest strange-bedfellows collection of inhabitants of any home in the City.
The mansion’s original owner, William Franklin Whittier, was the founder of the Whittier, Fuller & Company paint manufacturers, which later became Fuller O’Brien Paints. Whittier had a ne’er-do-well son named Billy, whose dissipated life ended when he was just 52. According to Dolores Riccio in her 1989 book, Haunted Houses USA, many people have reported seeing a ghost in the building; she speculates that the alleged ghost is Billy Whittier, wandering unhappily around in his old family mansion.
In April 1941, the Whittier family sold it for $44,000 to the Nazi government, which turned it into the German Consulate. The German consul was Fritz Wiedemann, who had been Adolf Hitler’s commanding officer in World War I and later became his aide-de-camp and one of his most trusted intimates. But after Hitler discovered that Wiedemann was having an affair with a Hungarian princess whom the führer had been using for secret diplomatic missions — and was also infatuated with — he shipped Wiedemann to San Francisco. There Wiedemann, a handsome and well-dressed man who was said to “exude eroticism,” lived a playboy lifestyle, making regular appearances in Herb Caen’s gossip column until he was forced to leave before the Japanese attacked Pearl Harbor.
The Whittier Mansion was held by the Office of the Alien Property Custodian until 1947, when it was sold again. After being visited by alleged ghosts and housing hedonistic Nazis, it then became unexpectedly haunted by highbrows. From 1952 to 1955 it housed philosopher, educator, author and Great Books editor Mortimer Adler’s Institute for Philosophical Research, a unique think tank where Adler and 20 (!) fellows did the research that led to the two-volume work The Idea of Freedom. The mansion’s intellectual tradition continued in 1956, when it became the headquarters of the California Historical Society, which also gave tours of the building. It was sold in 1993 and is now a private residence, bringing an end, at least for now, to one of the strangest histories of any building in San Francisco.