Preheat the oven to 350°F (175°C). Grease and flour an 8-inch square baking pan.
In a medium bowl, whisk together the all-purpose flour, baking powder, baking soda, and salt. Set aside.
In a large bowl, beat the softened butter and granulated sugar together until light and fluffy, about 3-4 minutes.
Add the eggs one at a time, beating well after each addition. Mix in 1 teaspoon vanilla extract.
Stir in the mashed bananas.
Alternately add the dry ingredients and buttermilk to the banana mixture, beginning and ending with the dry ingredients. Mix until just combined.
Pour the batter into the prepared baking pan and spread evenly.
Bake for 30-35 minutes or until a toothpick inserted into the center comes out clean.
Remove from the oven and let the cake cool completely in the pan on a wire rack.
While the cake is cooling, prepare the vanilla pudding according to package instructions by whisking the pudding mix with 2 cups cold whole milk until thickened. Refrigerate until ready to use.
In a large mixing bowl, beat the heavy whipping cream, powdered sugar, and 1 teaspoon vanilla extract with an electric mixer on high speed until stiff peaks form. Refrigerate until assembly.
Once the cake is cooled, cut it into 1-inch cubes.
In a large clear trifle bowl or individual serving glasses, layer one-third of the banana cake cubes evenly on the bottom.
Top the cake layer with one-third of the prepared vanilla pudding, spreading evenly.
Add a layer of banana slices evenly over the pudding.
Add one-third of the whipped cream over the bananas.
Repeat the layering process two more times: cake cubes, pudding, banana slices, and whipped cream.
Finish with a decorative swirl of whipped cream on top.
Refrigerate the trifle for at least 2 hours before serving to allow flavors to meld.