US Patent No. 10,509,467

DETERMINING FIXATION OF A USER'S EYES FROM IMAGES OF PORTIONS OF THE USER'S FACE ENCLOSED BY A HEAD MOUNTED DISPLAY


Patent No. 10,509,467
Issue Date December 17, 2019
Title Determining Fixation Of A User's Eyes From Images Of Portions Of The User's Face Enclosed By A Head Mounted Display
Inventorship Shih-En Wei, Pittsburgh, PA (US)
Jason Saragih, Pittsburgh, PA (US)
Hernan Badino, Pittsburgh, PA (US)
Alexander Trenor Hypes, Pittsburgh, PA (US)
Mohsen Shahmohammadi, Pittsburgh, PA (US)
Dawei Wang, Pittsburgh, PA (US)
Michal Perdoch, Pittsburgh, PA (US)
Assignee Facebook Technologies, LLC, Menlo Park, CA (US)

Claim of US Patent No. 10,509,467

1. A head mounted display (HMD) comprising:a rigid body having a front side, a left side, a right side, a top side, and a bottom side and including a display element configured to display content to a user wearing the HMD and an optics block configured to direct light from the display element to an exit pupil and an additional exit pupil of the HMD;
a left image capture device coupled to an interior surface of the HMD proximate to the left side of the front rigid body and configured to capture images of a portion of the user's face enclosed by the rigid body, the portion of the user's face including a left eye of the user;
a left illumination source positioned proximate to the left image capture device and configured to emit light illuminating the portion of the user's face;
an right image capture device coupled to an interior surface of the HMD proximate to the right side of the front rigid body and proximate to the bottom side of the rigid body and configured to capture images of an additional portion of the user's face enclosed by the rigid body, the additional portion of the user's face including a right eye of the user;
a right illumination source positioned proximate to the right image capture device and configured to emit light illuminating the additional portion of the user's face; and
a controller coupled to the left image capture device and to the right image capture device, the controller configured to:
obtain the images of the portion of the user's face including the left eye of the user from the left image capture device;
obtain the images of the additional portion of the user's face including the right eye of the user from the right image capture device;
generate a vector indicating a fixation of the user's left eye and a fixation of the user's right eye relative to the position of the head of the user by applying a model to the images and to the additional images;
modify content presented by an electronic display included in the HMD based on the vector by increasing a resolution of a segment of content at a location of the electronic display corresponding to the fixation of the user's left eye relative to a resolution of content presented at other locations of the electronic display and increasing a resolution of an additional segment of content at a location of the electronic display corresponding to the fixation of the user's right eye relative to the resolution of content presented at other locations of the electronic display.