The problem is that slow roll inflation requires a large amount of fine tuning. According to standard FRW cosmology, bubble nucleation requires negative curvature. The so called inflaton potential that gives rise to this would need to be extremely flat if you want gravitationally bound structures to form in the universe. If there were lumpy on the order of one part in $10^5$, there would be too much negative curvature, things would move apart faster than the escape velocity, and there would be no structure formation.