I agree with the broader point but my guess is that if you're rich you can afford to lose investments more than if you're not.
The problem is that if you are poor, you cannot afford to NOT get a chance out of poverty thru these investments either. Keeping "poor" people out of the ability to invest because they could lose so much also keeps them out of the ability to get out of poverty.