Stagger meter went backwards

Was battling Darak, used my rune to do some stagger damage, then hit him with my cold enchanted sword. The stagger meter went the wrong way. Hit him with a rune again and stagger meter went up. Then some normal hits with a combo, stagger meter went down. I know the stagger meter goes down if I’m idle for too long but this wasn’t it. It just went the wrong way. The problem went away after phase 1. Phase 2 Darak stagger meter worked as expected.