The custom of viewing spring-time flowers (hanami) dates back to the 8th century, and originated as an aristocratic diversion, first in Nara and then Kyoto.
Initially it was ume (plum), not sakura (cherry) blossoms which provided excuse for lavish parties among the courtiers of the imperial palace. By the 17th century, common people discovered the pleasures of hanami as well.
It was then, during the Edo period, that bringing food and drink and sharing them under the trees in bloom became a tradition, eagerly upheld today.
And -- yes, you figured it out -- it's not just sakura anpan ( cherry blossom buns, which Emperor Meji was reportedly very fond of), sakura mochi (pink, cherry flavored cakes made of rice) or sakura yu (broth-like tea) that are being enjoyed during sakura parties...