When is it time to refret?

Hi there!

I'm getting some buzz on some frets, and the action on my T-40 couldn't be better adjusted.
I think the frets it has are far from the jumbo sized we see on some instruments. In fact, they are kind of thiner. How do I know is it time for a refret job?

Thanks.