I'm really bad at visualizing people, too. I know of writers who go looking for pictures of people that match the appearance of the character in their head, but most of the time my characters don't even have an appearance to match to. Your method of having a file of people sounds like a good idea.