c. 1600, in music, "involving tones foreign to the normal tonality of the scale, not diatonic," from Latin chromaticus, from Greek khrōmatikos "relating to color, suited for color" (also used in reference to music), from khrōma (genitive khrōmatos) "color, complexion, character" (but chiefly used metaphorically of embellishments in music), originally "skin, surface" (see chroma).
Greek also used khrōma for certain modifications of the usual diatonic music scale. The reason the Greeks used this word in music is not now entirely clear. Perhaps the connection is the extended sense of khrōma, "ornaments, make-up, embellishments," via the notion of "characteristic" of a musical scale or speech.
In English, the musical sense of "progressing by half-tones, involving the sharps and flats of the staff" is by 1881. Meaning "of or pertaining to color" is from 1829.