It's a valid point, but the Colts under Ballard have done this a few times before this.
The Top 30 visits have been reported before as a way for teams to get clarity on a player if they have concerns or questions.
Kwity (and Dayo?) has worked with him and I'm guessing the Colts must've liked what they heard about him. If anything it just makes me more confident when the Colts, who are so concerned about character, draft him without meeting him.