As soon as you have re-assembled your entire party post Absorb-Gate, head into the Fabre Mansion to hear that Suzanne as fallen ill and the Mushroom Road to be first mentioned. The cut-off point of this sidequest is upon entering Eldrant, so you have a pretty big reign on when you do the quest.

C-A

To expand on CatMuto's answer:

The time limit to TURN IN the Mushroom to Suzanne expires once you enter Eldrant. You can still access Mushroom Road and find/pick the mushroom, which you have to do for Collector's credit anyway, and there is NO time limit for that.

HOWEVER, if you do the sidequest before entering Eldrant... you can play as Asch again. Keep in mind there are a few skits you can only see while he's in party (such as his cooking skit for mastering a recipe).