Preheat the oven to 350°F (175°C). Grease and flour an 8-inch round cake pan.
In a medium bowl, whisk together the flour, baking powder, and salt. Set aside.
In a large bowl, beat the softened butter and granulated sugar together with an electric mixer until light and fluffy, about 3 minutes.
Add the eggs one at a time, beating well after each addition.
Mix in the lemon zest, lemon juice, and vanilla extract until combined.
Gradually add the dry ingredients to the wet ingredients, alternating with the milk, beginning and ending with the dry ingredients. Mix until just combined.
Pour the batter into the prepared cake pan and smooth the top with a spatula.
Bake for 35 to 40 minutes or until a toothpick inserted into the center comes out clean.
Allow the cake to cool completely in the pan on a wire rack.
Once cooled, cut the cake into 1-inch cubes.
In a large bowl, whip the heavy cream, powdered sugar, and vanilla extract with an electric mixer until stiff peaks form.
In a large trifle bowl or individual serving glasses, layer one-third of the lemon cake cubes.
Spread one-third of the lemon curd evenly over the cake cubes.
Add a layer of one-third of the whipped cream over the lemon curd.
Sprinkle one-third of the fresh blueberries and raspberries over the whipped cream.
Repeat these layers two more times, ending with berries on top.
Cover and refrigerate the trifle for at least 1 hour before serving to allow flavors to meld.