Yes, the Colts were a much better franchise with Peyton Manning at the helm. The team had bottomed out by the time Andrew arrived, Mathis and Wayne were pretty much the only relevant players from that era. Ergo, they both stepped into bad teams.
Division is a fair point but Andrew is 6-2 in his career against that division, while Stafford is 3-5 against South teams. The article you are quoting your arguments from doesn't take into account running stats or supporting casts. Again, give Andrew Luck a defense that good and you would feel that team should be Super Bowl caliber, the Lions lost in the Wild Card.