I think you're right about the relationship between geography and demography, but the character descriptions in the book are consistent with the casting. Egwene and Nynaeve have darker complexions than the boys.

It's not clear how that would be maintained in a small insular community like the Two Rivers, although neither Rand nor Nynaeve were born there. Also, the people of that region are descendants of a great cosmopolitan nation that fell hundreds of years earlier, so maybe they started with a very high degree of genetic diversity.