Its mainly the quality of the eggs, they are supposed to gradually decline in quality at aged 35 through to early 40's. I asked my FS about this as I wondered how much infertility in late 30's was things like implantation problems and she said its usually the eggs. To illustrate she said if women use donor eggs from age 35-50 the pregnancy rates are similar to younger women, if they use their own eggs pregnancy rates decline each year.
So the quality of the eggs obviously makes it harder to get pregnant in the first place, higher MC rates and then higher incidences of chromosomal problems.
When you are in your early twenties you might pop out say (random number ahead!) 8 good eggs every year, when in your late 30's it might only be 1 or 2, so thats why it often takes longer. However some women get pregnant quickly regardless of age so its all just general statistics. Being on the younger side of the over 35 group Im sure you will be fine, good luck!