Brains are incredible, I bet it might work even with an adult, it would just take some getting used to.

Another thought I had (more of a weird experiment than a game) would be to feed the view from a 3d camera into a VR headset such that you can see all of it at once. Obviously it would be weird and everyone would look funny, but I bet if you stayed in long enough your brain would get used to it and it would start to look/feel normal