Preheat the oven to 350°F (175°C). Grease and flour two 8-inch round cake pans.
In a small bowl, mix the mashed bananas with lemon juice and set aside.
In a medium bowl, whisk together the all-purpose flour, baking soda, salt, ground cinnamon, and ground nutmeg.
In a large bowl, cream together the 1/2 cup softened butter, granulated sugar, and light brown sugar until light and fluffy, about 3-4 minutes.
Add the eggs one at a time to the butter and sugar mixture, beating well after each addition. Stir in 1 teaspoon vanilla extract.
Alternately add the flour mixture and buttermilk to the wet ingredients, beginning and ending with the flour mixture. Mix until just combined.
Fold in the mashed bananas gently until evenly incorporated.
Divide the batter evenly between the prepared cake pans and smooth the tops with a spatula.
Bake for 45 to 50 minutes, or until a toothpick inserted into the center comes out clean.
Remove the cakes from the oven and let them cool in the pans for 10 minutes. Then invert onto wire racks and cool completely.
To prepare the vanilla buttercream frosting, beat the 1 cup softened butter on medium speed until creamy, about 2 minutes.
Gradually add the powdered sugar, 1 cup at a time, mixing on low speed until combined.
Add the 1/4 cup whole milk and 2 teaspoons vanilla extract, then beat on high speed for 3 minutes until light and fluffy. If the frosting is too thick, add more milk, 1 tablespoon at a time; if too thin, add more powdered sugar.
Place one cake layer on a serving plate and spread a thick, even layer of frosting on top.
Place the second cake layer on top and frost the top and sides of the entire cake evenly.
Decorate as desired and serve.