Preheat the oven to 350 degrees. Line an 8x4 pan with parchment paper and set it aside.
Peel and core the apples, then cut them into cubes. Mix them with the granulated sugar and cinnamon. Set them aside.
In another small bowl, mix together the flour, baking powder, baking soda, salt, cinnamon, and ginger. Set this aside as well.
In a medium-size bowl cream together the softened butter and brown sugar for 1-2 minutes until it is light in color and creamy. Add in the eggs and vanilla and this time only mix to combine.
To this add the dry ingredients. Mix this in, but only mix to combine. Do not overdo it or the bread will be tough. Last is the milk, mix to combine.
Pour half the batter into a parchment-lined 8x4 loaf pan. It will be thick, so spread it gently with a spoon or spatula. Top this with a little over half the apple mixture. Press the apples into the batter and swirl slightly.
Repeat with the remaining batter then apple mixture to finish the loaf. Move it to the preheated over and bake for 60-75 minutes, until a toothpick inserted comes out mostly clean.
Let it cool in the pan for about 5 minutes, then carefully lift it out with the parchment paper. Place on a cooling rack and let it cool for at least 10 minutes before cutting into it.