Well Usually most bugs aren't harmful to the aquarium. If you are still shaky about keeping them in your tank you could remove them with a fine net or something of the sort. However cichlids wouldn't mind eating a bug or two I know people that feed cichlids grasshoppers and other insects deliberately.
I have these too. I know exactly what you are talking about. They are almost too small to see, like fleas or something. Never did any harm to my tank....I think they made me itch a few times after cleaning the tank though.