The tracker node will set your alpha to full white.
There. I said it.
I’ve spent a lot of time moving tracker nodes and then inverting roto alphas because they no longer work. It may not be a bug, but it certainly is not intuitive. Once you’ve heard this and know to look out for it things will start to make more sense.
The best way to deal with it is probably to shuffle black into the alpha after the transform.