You should be able to take the FE exam once you are done with undergraduate college. Many people (including myself) take the FE exam after getting their bachelors engineering degree.
In general, you will need about 4-5 years of work experience to be eligible/qualified to take the PE exam. Getting a Master's degree in the relevant engineering topic can substitute for 1 year of work experience (at least that's how it was where I applied). I think some states are starting to allow you to take the PE exam before you get all the required years of work experience, but will simply not officially give you the PE title until you get your required years of work experience.
