Built from out of and pocket, the place where personal money was traditionally carried. The financial use is recorded from 1679, and the adjective for a direct cash expense from 1885. The sense of being wildly inappropriate comes from mid-20th-century African American English.