hole-in-the-wall (n.)

"small and unpretentious place," 1816, perhaps recalling the hole in the wall that was a public house name in England from at least 1690s. "Generally it is believed to refer to some snug corner, perhaps near the town walls," but the common story was that it referred to "the hole made in the wall of the debtors' or other prisons, through which the poor prisoners received the money, broken meat, or other donations of the charitably inclined" [Jacob Larwood and John Camden Hotten, "The History of Signboards: From the Earliest Times to the Present Day," 1867]. Mid-19c. it was the name of the private liquor bar attached to the U.S. Congress.