Seeing 3D from 2D Images - University of Florida

Seeing 3D from 2D Images - University of Florida

Seeing 3D from 2D Images William and Craig 115 - 164 How to make a 2D image appear as 3D! Output is typically 2D Images Yet we want to show a 3D world! How can we do this? We can include cues in the image that give our brain 3D information about the scene These cues are visual depth cues Visual Depth Cues Monoscopic

Depth Cues (single 2D image) Stereoscopic Depth Cues (two 2D images) Motion Depth Cues (series of 2D images) Physiological Depth Cues (body cues) Monoscopic Depth Cues Interposition An object that occludes another is closer

Shading Shape info. Shadows are included here Size Usually, the larger object is closer Linear Perspective parallel lines converge at a single point

Surface Texture Gradient more detail for closer objects Height in the visual field Higher the object is (vertically), the further it is Atmospheric effects further away objects are blurrier

Brightness further away objects are dimmer Stereoscopic Display Issues Stereopsis Stereoscopic Display Technology Computing Stereoscopic Images Stereoscopic Display and HTDs. Works for objects < 5m. Why? Stereopsis The result of the two slightly different views of the external world that our laterally-displaced eyes receive. Retinal Disparity If both eyes are fixated on a point, f1, in space, then an image of f1 if focused at corresponding points in the center of the fovea of each eye. Another point, f2, at a different spatial location would be imaged at points in each eye that may not be the same distance from the fovea. This difference in distance is the retinal disparity. f2

f1 Left Eye Right Eye Retinal disparity =

Disparity If an object is closer than the fixation point, the retinal disparity will be a negative value. This is known as crossed disparity because the two eyes must cross to fixate the closer object. If an object is farther than the fixation point, the retinal disparity will be a positive value. This is known as uncrossed disparity because the two eyes must uncross to fixate the farther object. An object located at the fixation point or whose image falls on corresponding points in the two retinae has a zero disparity.

Convergence Angles +a+c+b+d = 180a+a+c+b+d = 180c+a+c+b+d = 180b+a+c+b+d = 180d = 180 +a+c+b+d = 180c+a+c+b+d = 180d = 180 - = a+a+c+b+d = 180(-b) = 1+a+c+b+d = 1802 = Retinal Disparity f1 D1 f2 a

c 1 D2 b d i 2 Miscellaneous Eye Facts Stereoacuity

- the smallest depth that can be detected based on retinal disparity. Visual Direction - Perceived spatial location of an object relative to an observer. Horopters Corresponding points on the two retinae are defined as being the

same vertical and horizontal distance from the center of the fovea in each eye. Horopter - the locus of points in space that fall on corresponding points in the two retinae when the two eyes binocularly fixate on a given point in space (zero disparity). Points on the horopter appear at the same depth as the fixation point. f1 f2

Vieth-Mueller Circle Stereoscopic Display Stereoscopic images are easy to do badly, hard to do well, and impossible to do correctly. Stereoscopic Displays Stereoscopic display systems create a three-dimensional image (versus a perspective image) by presenting each eye with a slightly different view of a scene.

Time-parallel Time-multiplexed Time Parallel Stereoscopic Display Two Screens Each eye sees a different screen Optical system directs each eye to the correct view. HMD stereo is done this way. Single Screen Two different images projected on the same screen

Images are polarized at right angles to each other. User wears polarized glasses (passive glasses). Passive Polarized Projection Issues Linear Polarization Ghosting increases when you tilt head Reduces brightness of image by about Potential Problems with Multiple Screens (next slide) Circular

Polarization Reduces ghosting but also reduces brightness and crispness of image even more Problem with Linear Polarization With linear polarization, the separation of the left and right eye images is dependent on the orientation of the

glasses with respect to the projected image. The floor image cannot be aligned with both the side screens and the front screens at the same time. Time Multiplexed Display Left and right-eye views of an image are computed and alternately displayed on the screen. A shuttering system occludes the right eye when the left-eye image is being displayed and occludes the left-eye when the right-eye image is being

displayed. Stereographics Shutter Glasses Screen Parallax Display Screen Pleft Left eye position Right eye position Pright P

P Object with positive parallax Pright Pleft Object with negative parallax The screen parallax is the distance between the projected location of P on the screen, Pleft, seen by the left eye and the projected location, Pright, seen by the right eye (different from retinal disparity). Screen Parallax (cont.) p = i(D-d)/D

where p is the amount of screen parallax for a point, f1, when projected onto a plane a distance d from the plane containing two eyepoints. i is the interocular distance between eyepoints and D is the distance from f1 to the nearest point on the plane containing the two eyepoints d is the distance from the eyepoint to the nearest point on the screen f1 D

Projection Plane p i Left eyepoint Right eyepoint d Screen Parallax 5.00 Screen Parallax

-5.00 0 50 100 150 200 250 300 350 -15.00 -25.00

-35.00 -45.00 -55.00 -65.00 Distance from Eye Zero parallax at screen, max positive parallax is i, negative parallax is equal to I halfway between eye and screen Stereoscopic Voxels Left Eye Point 1 2

3 1 2 3 1 1 2 1 1 Right Eye Point

4 5 5 4 5 3 2 2 B A 3

2 1 4 3 4 1 2 3 4

1 2 3 4 5 5 Screen Parallax and Convergence Angles Screen parallax

depends on closest distance to screen. f2 Different convergence angles can all have the same screen parallax. Also depends on assumed eye Projection Plane separation. f1 f3

How to create correct leftand right-eye views To specific a single view in almost all graphics software or hardware you must specify: Eyepoint Look-at Point Field-of-View or location of Projection Plane View Up Direction Basic Perspective Projection Set Up from Viewing Paramenters

Y Z X Projection Plane is orthogonal to one of the major axes (usually Z). That axis is along the vector defined by the eyepoint and the look-at point. What doesnt work Each view has a different projection plane Each view will be presented (usually) on the same plane What Does Work i

i Setting Up Projection Geometry No Look at point Eye Locations Yes Eye Locations Look at points Screen Size

The size of the window does not affect the retinal disparity for a real window. Once computed, the screen parallax is affected by the size of the display screen Visual Angle Subtended Screen parallax is measured in terms of visual angle. This is a screen independent measure. Studies have shown that the maximum angle that a non-trained person can usually fuse into a 3D image is about 1.6 degrees. This is about 1/2 the maximum amount of retinal disparity you would get for a real scene. Accommodation/ Convergence

Display Screen Position Dependence (without head-tracking) Interocular Dependance M Projection Plane Perceived Point Modeled Point F Tr ue

od ele Ey es dE ye s Obvious Things to Do Head tracking Measure Users Interocular Distance

Another Problem Many people can not fuse stereoscopic images if you compute the images with proper eye separation! Rule of Thumb: Compute with about the real eye separation. Works fine with HMDs but causes image stability problems with HTDs (why?) Two View Points with Head-Tracking True Eyes Modeled Eyes Projection Plane

Perceived Points Modeled Point Maximum Depth Plane True Eyes Projection Plane Perceived Point F Modeled Eyes Modeled Point

E Maximum Depth Plane Can we fix this? Zachary Wartell, "Stereoscopic Head-Tracked Displays: Analysis and Development of Display Algorithms," Ph.D. Dissertation, Georgia Institute of Technology, August 2001. Zachary Wartell, Larry F. Hodges, William Ribarsky. "An Analytic Comparison of Alpha-False Eye Separation, Image Scaling and Image Shifting in Stereoscopic Displays," IEEE Transactions on Visualization and Computer Graphics, April-June

2002, Volume 8, Number 2, pp. 129-143. (related tech report is GVU Tech Report 00-09 ( Abstract , PDF , Postscript .) Zachary Wartell, Larry F. Hodges, William Ribarsky. "Balancing Fusion, Image Depth, and Distortion in Stereoscopic HeadTracked Displays." SIGGRAPH 99 Conference Proceedings, Annual Conference Series. ACM SIGGRAPH, Addison Wesley, August 1999, p351-357. (Paper: Abstract , PDF , Postscript ; SIGGRAPH CD-ROM Supplement,, supplement.tar.Z ). Point of fixation 0.40 0.30 Symmetric convergence 0.25 0.20

Convergence 20 centimeters to the left of the left eye 0.15 0.10 0.05 150 140 130 120 110 100

90 80 70 60 50 40 30 20 0.00 10

separation in centimeters Change in eyepoint 0.35 Distance in centimeters from eye plane Change in eyepoint separation with change in point of fixation. Centers of rotation of the eyes are assumed to be 6.4 centimeters apart. Ghosting Affected by the amount of light transmitted by the LC shutter in its off state. Phosphor persistence

Vertical screen position of the image. Ghosting (cont.) Extinction Ratio = Image Position Top Middle Bottom Luminance of the correct eye image -----------------------------------------------------------Luminance of the opposite eye ghost image Red 61.3/1 50.8/1

White 17/1 14.4/1 41.1/1 11/1 Ghosting (cont.) Factors affecting perception of ghosting

Image brightness Contrast Horizontal parallax Textural complexity Time-parallel stereoscopic images Image quality may also be affected by Right and left-eye images do not match in color, size, vertical alignment. Distortion caused by the optical system Resolution HMDs interocular settings Computational model does not match viewing geometry.

Motion Depth Cues Parallax created by relative head position and object being viewed. Objects nearer to the eye move a greater distance Pulfrich Effect Neat trick

Different levels of illumination require additional time (your frame rates differ base of amount of light) What if we darken one image, and brighten another? ml videos/pulfrich.avi Physiological Depth Cues Accommodation focusing adjustment made by the eye to change the shape of the lens. (up to 3 m) Convergence movement of the

eyes to bring in the an object into the same location on the retina of each eye. Summary Monoscopic Interposition is strongest. Stereopsis is very strong. Relative Motion is also very strong (or stronger). Physiological is weakest (we dont even use them in VR!) Add as needed ex. shadows and cartoons

Recently Viewed Presentations

  • NYS Workers Compensation System Compliance Outreach: Measuring and

    NYS Workers Compensation System Compliance Outreach: Measuring and

    April 2015 - Advise all system participants of new process. April 2015- September 2015 - TRIAL PERIOD (6 months)- Reports generated and . distributed to all system participants. Oct 1, 2015- System goes . live. January 2016 - 2015 4....
  • A Question of Questions: Prosodic Cues to Question

    A Question of Questions: Prosodic Cues to Question

    A Question of Questions: Prosodic Cues to Question Form and Function Julia Hirschberg (Joint work with) Jennifer Venditti and Jackson Liscombe Questioning in Dialogue A fundamental activity in conversation Elicit information Elicit action But How to define a question?


    4 channel DVR is a good device for capture all video (ECOR 960-4F. Recommend output to a 30" TV/monitor. (EC, Gates, Boat Video is used) Can be purchased $320.00. Splasheye- Same but it as auto gate playback. Splasheye also has...
  • Words Their Way

    Words Their Way

    Words Their WayGreen: Syllables and AffixesSort 12. Spelling Sorts: Do not use Slide Show to sort the words.
  • Before you begin

    Before you begin

    Affordable access to high-quality education and training for small and medium diversity business leaders. Graduate-level executive-management scholarships awarded to assist diverse suppliers in their pursuit of operational excellence. A national mentoring program for diverse - owned Business Leaders
  • TASC Test Overview Data Recognition Corp. Based in

    TASC Test Overview Data Recognition Corp. Based in

    TABE Online will transition from the current McGraw-Hill platform to the DRC INSIGHT platform starting in August 2017. TABE 11&12 will be added to TABE Online (DRC INSIGHT) in 2017. TABE 11&12 paper materials will be added to our warehouse...


    Exl BM et al. Eur J Nutr 2000; 2. Exl BM et al. JPGN 2000 * Affected infants (%) Intervention group 3 months 6 months 3.9 4.3 Control group 3 months 6 months 6.2 8. After appropriate medical evaluation the...
  • Cross-validation and Model Selection

    Cross-validation and Model Selection

    CROSS-VALIDATION AND MODEL SELECTION Many Slides are from: Dr. Thomas Jensen and Prof. Olga Veksler - CS9840 - Learning and Computer Vision EXAMPLE The following table gives the size of the floor area (ha) and the price ($000), for...