In terms of the GMs...., just like HCs, QBs, etc., right or wrong, most are judged long term on SBs, playoffs, and to a lesser extent, division titles. Like I've said, I like Ballard. He's not perfect, and I don't worship, but he's done far more good than bad IMO. But that simply needs to translate soon. The buck stops with him, and Irsay as an owner.
In terms of fans.... takes all kinds. If you only had sunshine types, the board would be a pretty boring place if there was even a board. Even some of the most winningest teams (like the Tide) who should pump more sunshine than anything else, don't... Again, takes all kinds.