I am not an expert by any means but I believe it has to b the original with your effects in order to work then put the image you want to b masked on top the create a clipping mask on that image and it should b good to go. Also the image you posted looks like you hv the shadow attached to it, the mask is just the overlay and the shadow ie the body of the icon falls behind it. So basically have your icon body on the bottom with a black inside where the icon falls, or white if you want the more transparent look. Then your mask, then your icon. Hope I was clear enough.
I think Maxner has it right. The ApIconShadow has transparent pixels outside the rounded edges. Transparent pixels on top of you image make anything underneath it transparent. Or so my limited experience has told me.