Hell of a story. I don't remember the restaurant, but the plaza behind the Charles Hotel has a fancy gym that I used to use (with student discount). It's an odd little pocket, even within the overall prestige of Harvard Square, it feels very Old Money in there.