"whittle away, waste bit by bit, spend on trifles," 1728, probably from noun fritter "fragment or shred" (though this is recorded later), perhaps an alteration of 16c. fitters "fragments or pieces," which is perhaps ultimately from Old French fraiture "a breaking," from Latin fractura [OED]. Or perhaps from a Germanic *fet-source (compare Middle High German vetze "clothes, rags," Old English fetel "girdle").

"fried batter cake," served hot and sometimes sweetened or seasoned or with other food in it, late 14c., from Old French friture "fritter, pancake, something fried" (12c.), from Late Latin frictura "a frying," from frigere "to roast, fry" (see fry, v.).