…or only when they play Let’s Do That Again, which seems to be the only place that allowance is stated.

See this answer in a video!
Little Answers

A: Anyone may look through the discard pile at any time.

Here is our logic: If you could only look through the discard pile if you had played the card, then you’d never be able to check to see whether there was anything worth playing in there, so you might totally waste what is a potentially great card.

So, let’s say you’re only allowed to look if you’re planning to play it. Then it would only allowed if you had the card in hand, then you’d have to prove it to everyone, by showing it, potentially, which isn’t terribly fair.

In short, it’s just as the card says: “Anyone may look through the discard pile at any time,” and “anyone” means “anyone,” not just the person with the card, and “any time” means “any time,” not just when you play it.