Preheat the oven to 325°F (163°C). Grease a loaf or bundt pan well and set aside.
In a large bowl, beat the butter and sugar together for 3 to 5 minutes until the mixture becomes light in color and fluffy in texture. This step helps create a soft cake.
Add the eggs one at a time, mixing well after each addition. Make sure the batter stays smooth before moving to the next step.
In a separate bowl, whisk together the flour, baking powder, and salt until evenly combined.
Add the dry ingredients into the wet mixture in small parts, alternating with the milk. Mix gently after each addition. Do not overmix the batter.
Stir in the key lime juice, lime zest, and vanilla extract. Mix just until everything is combined and smooth.
Pour the batter into the prepared pan and spread evenly. Lightly tap the pan to remove air bubbles.
Bake for 60 to 70 minutes, or until a toothpick inserted in the center comes out clean or with a few crumbs.
Allow the cake to cool in the pan for 10 to 15 minutes, then remove and let it cool completely on a wire rack.
In a small bowl, mix powdered sugar with key lime juice until smooth. Drizzle the glaze over the cooled cake and let it set before slicing.