I actually like all of his albums and found none of them bad, but I do like some more than others.
I'd say the weakest link, IMO, is maybe Encore. I still remember when I was 13 and bought Encore (after listening to his 3 previous albums). I put it in the CD player and first few tracks, I was like what the hell! And I didn't even get to Pee Wee Herman section of the album yet. I did find several tracks in there I did like, and it was one of my most listened to albums when I was 14.
But even then, Encore isn't a bad album. It's actually pretty good. But just not up to par with his others.
I think it's relative though. The bigger you are, the more expectations you have. When you're listed among some of the best, having a good album isn't enough.