Combine butter and peanut butter in a large bowl and use an electric mixer to beat until creamy and well-combined.
Gradually add powdered sugar until completely combined. Be sure to scrape the sides and bottom of the bowl so all ingredients are well-combined. Stir in vanilla extract.
With mixer on low-speed, add milk and stir until well-combined. Gradually increase speed to high and beat for 30 seconds. Spread or pipe frosting onto completely cooled cupcakes or cake.