The constraint system uses the left and right eye locations relative to the left and right image plates to compute correct view frustums