Preheat the oven to 350°F (175°C). Grease a 9×5 loaf pan with butter or cooking spray.
Make the batter: In a large bowl, cream together butter and sugar until light and fluffy. Add eggs and vanilla, mixing until combined. In a separate bowl, whisk together flour, baking powder, and salt. Gradually add the dry ingredients to the wet ingredients, alternating with the milk. Mix until just combined.
Prepare the cinnamon swirl: In a small bowl, mix together granulated sugar and cinnamon.
Layer the cake: Pour ⅓ of the batter into the prepared pan. Sprinkle half of the cinnamon-sugar mixture over the batter. Add another third of the batter, followed by another layer of cinnamon-sugar. Top with the remaining batter, then sprinkle with brown sugar.
Bake for 50-55 minutes, or until a toothpick inserted into the center comes out clean. Let the cake cool in the pan for 15 minutes before transferring to a wire rack.
Make the glaze: In a small bowl, whisk together powdered sugar, milk, and vanilla until smooth. Drizzle over the cooled cake.