Supported too, by the clear evidence that at time of LoTR, it was only the 'older'
Elves that had much real power left. There's someplace it's even mentioned their power was waning as their time in
Middle-earth came to an end.
The same can't be said for
men, as at the end of LoTR, it was the "age of men."
That said, it's clear the men dispersed abroad were but shadows of the great men of the earlier age, which is also spoken to. In part I believe, because in one place it's said their power grew less as they were overtaken by greed and lust in their pursuit of the immortaility of the Elves.
Evil too,
Sauron and
Saruman, were but shadows of the evil that was
Melkor, and probably even
Ungoliant. We can recall Sauron originally...