button-hole (n.)

1560s, "hole or loop in which a button is caught," from button (n.) + hole (n.). The verb, also buttonhole, meaning "to detain (someone) in conversation against his will" (1862) was earlier button-hold (1834), from button-holder (1806, in this sense). The image is of holding someone by the coat-button so as to detain him.