The meaning is that it is good to indulge in your favorite food, drink, or pastime – sometimes. It can be healthy even. But keep in mind the ‘little’ part.
This proverbial saying was first the title of a vaudeville song, made famous by the risqué Victorian singer Marie Lloyd, a music hall performer in England towards the end of Queen Victoria’s reign and into the early 20th century, much to the chagrin of the moralists of the day, who did everything they could to prevent her from performing.