"something resembling the handle of a pan," 1851, from pan, n. + handle, n.. Especially in reference to geography, originally American English, from 1856, in reference to Virginia (now West Virginia; Florida, Texas, Idaho, Oklahoma, and Alaska also have them). Meaning "an act of begging" is attested from 1849, perhaps from notion of arm stuck out like a panhandle, or of one who handles a (beggar's) pan.

"to beg," 1888, from panhandle, n. in the begging sense. Related: Panhandled; panhandling.