If you've take a couple of rest days off it can be a "false positive" and actually be indicating your recovery, otherwise big swings up or down are indicators that you need a rest.
This is explained in some depth in the book if you're interested in the science behind the reasoning. (Unfortunately I lost mine after my last move so I can't look it up)
Higher HRV isn't better. Normal HRV (for you) is better. Higher means parasympathetic dominant (long-term strategies). Lower means sympathetic dominant (short-term strategies).
According to the book, higher HRV in response to training means you've shifted into long-term adaptations to stress. A low HRV in contrast is associated with short-term adaptations (such as increased hormonal output). Any of it has to be interpreted in context.