Preheat oven to 350 degrees F. Wrap outside of 10-inch by 3-inch springform pan with foil to prevent leaking.
Prepare Plum Topping: Melt butter in springform pan in oven, about 4 minutes; remove from oven. With fork, stir brown sugar into butter until blended, spreading to coat bottom of pan. Arrange plum wedges in concentric circles on top of brown-sugar mixture, overlapping wedges slightly; set aside.
Prepare Almond Cake: On waxed paper, combine flour, baking powder, and salt.
In large bowl, with mixer at low speed, beat sugar, almond paste, and butter 2 minutes or until blended, scraping side of bowl often with rubber spatula.
Increase speed to medium; beat 2 minutes longer or until light and fluffy, occasionally scraping bowl. Add eggs, 1 at a time, and vanilla, beating just until blended. Reduce speed to low; add flour mixture alternately with milk, beginning and ending with flour mixture. Beat just until mixed, occasionally scraping bowl.
Pour batter over plums in pan, carefully spreading with rubber spatula. Bake 60 to 70 minutes or until toothpick inserted in center comes out clean, covering pan loosely with foil after 50 minutes if top is browning too quickly.
Cool cake in pan on wire rack 10 minutes. Loosen edge of cake from side of pan. Invert cake onto serving plate; remove side and bottom of pan. Cool cake 15 minutes to serve warm or cool completely.