Moze with swordplay.

I noticed something weird with how the swordplay stacks work on moze. I dont seem to lose stacks even when I use ult on a different enemy while in stealth. Did moze not lose stack or an ult + followup instantly gives him 5 stacks?