Mediated reality is to "augment, diminish, or otherwise alter your visual perception of reality." Most readers will be familiar with the concept of virtual reality. Mediated reality is a similar to virtual reality in that both alter the user's percepction of their environment, typically by replacing their vision with the output from a head-mounted display. However, virtual and mediated reality differ in that virtual reality seeks to create an entirely new environment for the user, whereas mediated reality alters what you would normally see - i.e. it alters your perception of your actual environment.

Eye Tap is one device which is capable of creating a mediated reality for the user. Typically, Eye Taps resemble typical eyeglasses or sunglasses, worn by the user. Eye Tap devices have three main parts:

• a measurement system typically consisting of a camera system, or sensor array with appropriate optics;
• a diverter system, for diverting eyeward bound light into the measurement system and therefore causing the eye of the user of the device to behave, in effect, as if it were a camera;
• an aremac for reconstructing at least some of the diverted rays of eyeward bound light. Thus the aremac does the opposite of what the camera does, and is, in many ways, a camera in reverse. The etymology of the word ``aremac'' itself, arises from spelling the word ``camera'' backwards.

There are two embodiments of the aremac: (1) one in which a focuser (such as an electronically focusable lens) tracks the focus of the camera, to reconstruct rays of diverted light in the same depth plane as imaged by the camera; and (2) another in which the aremac has extended or infinite depth of focus so that the eye itself can focus on different objects in a scene viewed through the apparatus.

Focus tracking Eye Tap systems

This article describes only the focus tracking embodiment of the Eye Tap system. The aremac has focus linked to the measurement system (e.g. ``camera'') focus, so that objects seen depicted on the aremac of the device appear to be at the same distance from the user of the device as the real objects so depicted. In manual focus systems the user of the device is given a focus control that simultaneously adjusts both the aremac focus and the ``camera'' focus. In automatic focus embodiments, the camera focus also controls the aremac focus. Such a linked focus gives rise to a more natural viewfinder experience, as well as reduced eyestrain. Reduced eyestrain is important because the device is intended to be worn continually.

The operation of the depth tracking aremac is shown in the figure below.

Figure 1(a): Focus tracking aremac: with a NEARBY SUBJECT, a point P0 that would otherwise be imaged at P3 in the EYE of a user of the device is instead imaged to point P1 on the image SENSOR, because the DIVERTER diverts EYEward bound light to lens L1. When subject matter is nearby, the L1 FOCUSER moves objective lens L1. out away from the SENSOR automatically, as an automatic focus camera would. A signal from the L1 FOCUSER directs the L2 FOCUSER, by way of the FOCUS CONTROLLER, to move lens L2 outward away from the light SYNTHesizer. At the same time, an image from the SENSOR is directed through an image PROCessor, into the light SYNTHesizer. Point P2 of the display element is responsive to point P1 of the SENSOR. Likewise other points on the light SYNTHesizer are each responsive to corresponding points on the SENSOR, so that the SYNTHesizer produces a complete image for viewing through lens L2 by the EYE, after reflection off of the back side of the DIVERTER. The position of L2 is such that the EYE's own lens L3 will focus to the same distance as it would have focused in the absence of the entire device.

Figure 1(b): With DISTANT SUBJECT MATTER, rays of parallel light are diverted toward the SENSOR where lens L1 automatically retracts to focus these rays at point P1. When lens L1 retracts, so does lens L2, and the light SYNTHesizer ends up generating parallel rays of light that bounce off the backside of the DIVERTER. These parallel rays of light enter the EYE and cause its own lens L3 to relax to infinity, as it would have in the absence of the entire device.

The operation of the aremac focus and zoom tracking is shown in Fig 2.

Figure 2: Focus of right camera and both aremacs (as well as vergence) controlled by autofocus camera on left side. In a two eyed system, it is preferable that both cameras and both aremacs focus to the same distance. Therefore, one of the cameras is a focus master, and the other camera is a focus slave. Alternatively, a focus combiner is used to average the focus distance of both cameras and then make the two cameras focus at equal distance. The two aremacs, as well as the vergence of both systems also track this same depth plane as defined by camera autofocus.

The apparatus is usually concealed in dark sunglasses that obstruct vision except for what the apparatus allows to pass through.

Because the device absorbs, quantifies, processes, and reconstructs light passing through it, there are extensive applications in mediated reality. Mediated Reality differs from Virtual Reality in the sense that Mediated Reality allows the visual perception of reality to be augmented, deliberately diminished, or, more generally computationally altered. The theory and practice of mediated reality are discussed in the next feature article.

Bibliography

1

Stephen R. Ellis, Urs J. Bucher, and Brian M. Menges.
The relationship of binocular convergence and errors in judged distance to virtual objects.
Proceedings of the International Federation of Automatic Control, June 27-29 1995.

Previous Articles