Whether you're too short or not tall enough is really not of any concern to any college coach (well, I hope their not thinking that way)- it's going to be how well you function (your mechanics, speed, movement, control and more).
Will a taller pitcher have an easier time- maybe. Would you have to put more effort in to gain a stride length like a taller pitcher- probably. Could you generate better arm speed then taller players- absolutely.
It's what you do with your body, how you treat it and how well you use it.
You will notice that all advice given by the posters centers on what you can "do" and not much on the height factor. Here are two articles I have on my ChicagoNow blog that will help clarify much of what posters are talking about. I hope you enjoy them, especially the video at the end of Part II, they will help with understanding:
the average is prob 5'8
if ur in college, im gonna be honest, i think that is realli short!!!!, but if ur good at it, i would stick 2 pitching!!! i am in 6th grade, and i am 5'7. i hope 2 be 5'10 or 5'11 so i can succeed in college and possibly pro!! :)