Whispers in general are finnicky because it feels like you have to stand near pixel perfect on top of them to be able to interact with them.
I just had a similar issue with small chests, one in the nameless pass prison before the round cell with the round push around door opener (best naming ever). left of that, under the ledge, wasn’t reachable.