VR, Tele-Perception, Tele-Existence and EEG-Feedback

introduction

Virtual Reality provides a basis for the technology which enables humans to experience events and acts in a virtual environment just as if they were in the real world. It is considered to be important developing Virtual Reality as a basic technology, because VR is expected to become a key-technology fields and applications in the 21st century.

Some applications of Virtual Reality are

Tele-existence and bilateral robotics, Space tele-robotics
Amusement and simulators, VR entertainment
Interactive CAD design and virtual prototyping
Virtual Kitchen and VSDSS
Multi-media and networked VR, Tele-conferencing
Computer Supported Cooperative Work (CSCW)
Artificial Life and cyberspaces
Virtual actors and real-time animations
VR in medicine, Laparoscopy, "Nintendo surgery"
Medical micro-robots, surgical planning,high definition medical imaging
Rehabilitation and amenity
Virtual hospitals and tele-medicine

Classification fo Virtual Reality Systems

If we regard Virtual Reality in the perspective of a human interface can identify two types of communications, these are:

A. human-machine or single user systems, where the interface should establish communication between user and, e.g., computer;

B. human-(machine)-human or multi-user, where the interface is actually intended for communication of two or more people, and the machine is regarded as a communication media between them.

We can also subclassify VR systems according to the kind of "experience" they present to the user, which are:

1. Transmitted Reality,
2. Synthetic but based on a real-world model, and
3. Pure-synthetic imaginary worlds.

It is possible to define a classification of Virtual Reality systems based on the two above mentioned criteria. Categories are then characterized by the type of communication and the kind of world they present.

Tactile, Kinesthetic and Haptic Displays

To improve the sense of reality it may be sometime necessary to give some form of physical feedback to the operator. For example it may be useful to provide tactile cues that allow the user to recognize surface texture of materials, or kinesthetic feedback that let the user feel the weight of a virtual object. Tactile and kinesthetic sensations are commonly called, haptic perception.

It is no doubt that haptic feedback will be difficult to achieve, but it may improve the degree of presence dramatically. Although research on visual and auditory displays has been much progressed, the research on haptic feedback is still in the beginning phase but receives much attention nowadays. At the moment, the only way to produce satisfactory haptic feedback seems to build some kind of exoskeleton on (parts of) the body.

History of Virtual Reality

The term "virtual reality" has been introduced by Jaron Lanier, who founded VPL Research in the US in 1989, when he introduced the DataGlove and the EyePhone on the market. Soon after that, the term has exploited by the media and game makers to create a kind of fashionable ambiance to the public.

The first attempt to create a synthesized sense of reality could be traced back to Morton Heilig, who introduced in 1960 the "Sensorama" that should have become the ultimate theater providing all the sensory signals to make cinema undistinguishable with reality. The Sensorama looked like an arcade machine, and was outfitted with handlebars, a binocular-like viewing device, a vibrating seat, and small vents that could blow air when commanded. In addition, stereophonic speakers were mounted near the ears, and close to the nose, a device for generating odors specific to the events viewed on the film.

The first Head Mounted Display (HMD) was developed in 1966 by Ivan Sutherland ho has been credited being the "father of computer graphics". In 1965 he published an article entitled "The Ultimate Display" in which he writes, "A display connected to a digital computer gives us a chance to gain familiarity with concepts not realizable in the physical world. It is a looking glass into a mathematical wonderland".

Tele-Existance

The concept of tele-existence was proposed by Susumu Tachi in 1980, now professor at the elite University of Tokyo. This technology enables a person to have a realtime sensation of being at a place other than the place where he or she actually exists. The proposed system consisted of a tele-existence master-slave system with visual and auditory sensation of presence for robotic applications. The master system measures the head and arm movements of the operator through mechanical links, and a computer controls the movement of the slave anthropomorphic robot with a stereo camera and a locomotion mechanism for movements.

EEG-Feedback

EEG-Feedback is a technique developed in the early 80'ies that uses an amplifier to measure light elecromagnetical changes at the scalp caused by nerve-cell activities in the brain. Changes in the state of mind correspond with physiological changes in brainwave frequency and energy patterns. For example in the altered state of Trance, enourmous changes in activation and dynamics in various brain-areas can be measured in a short time. When practising Yoga or meditation, wide ares of the brain get to a synchronised, calm vibration, with slight changes and amounts of slow Theta and Alpha-Waves.

BrainWave-Feedback has some therapeutical effects like stress relaxation, muscle relaxation, re-focusing at disorders or enhancement of conciousness, like as a meditation aid. An interesting effect of EEG-feedback is that through a training prcess, a person can learn to influence the own brain-activity, and to use it for selection or to navigate virtual our real vehicles.

raw EEG-Signal

Frequency Spectrum using Fast Fourier Transformation

Using modern computer systems, the measured EEG-data can be visualised or transposed into sounds in various ways. For example the Fourier-Analyses show the spectra of the mixed-up frequencies.

TBS: a interdisciplinary and international art project based on a Tele-Existence-VR-System

Seen from the artistic point of view, VR gives enormous spaces for the reflection of reality. In a virtual world or landscape one travels an inner space, compareable to a shaman's ride of spirtis. Body-movements are reduced to a minimum. Using a sensitive VR-system, an iris-reaction, finger-movement or, when using the EEG as an input-parameter, just activation of the mind is enough information for a selection or for navigating the virtual space.

The Teleroboted Brainwashing System (TBS) uses these technologies to point out the interdependance between a subject's individual existance (percepted through the process of becoming aware of one's own brain activity in the EEG-feedback) and the cultural and social reality in it's unpredictable flow, percepted through the travelling in an artificial Perceptual Landscape that consists of historical, emotional, spiritual, social and political inplants taken out of the real world. The Perceptual Landscape is a melting pot for cultural artifacts, symbols, statements and quests. The development of the Landscape is an essential phase of the TBS-project, where people from different countries and with different socialisations meet and arrange thier own significants in the created landscape that grows to a puzzle-like social sculpture. The Perceptual Landscape is a metaphor for the intercultural processes of reality creation, which gain greatest power at the level of a global conciousness. The TBS-Project investigates the dynamics of this processes and the boundaries (if there are) for their current embodyments. TBS wants to encourage the discussion about static structures in our cultural-bound systems.

There are 3 TBS-Areas that interact with each other:

1. Brainwashing Station with EEG-Feedback

The Brainwashing Area shows the individual in it's perceptual world. A person expieriences programming and meta-programming of the mind becoming aware of interpretations of his/her own brain activitiy. The WashingMachine, which immerses the person partly or totally, tells the picture of the 'world machine' that surrounds us and re-programs us daily with new emanations out of the vast spectrum of living.

The EEG-equipment transposes Alpha-Activity and Peak-to-Peak-Amplitues of the brain activity to audial and visual interpretations that are transmitted to certain monitors in the Perceptual Landscape, to a video-projector for the audience and to the individual itself, where they generate a biofeedback-loop when enabled through the person's awareness.

The Brainwashing Station is equipped with a VR-set of 3D video-glasses. The displayed video is transmitted from the WashBot driving in the Perceptual Landscape. The WashBot can be navigated from another person, the Brainwashing Operator, who embodies all the conditions of live that we can't conciously manipulate.

2. Tele-Perception Operation and Movement-Control

The Tele-Operation Area is the place where reality manipulation and tele-existance can be expierienced by the Operator. He/She is equipped with a HMD (3D-video glasses, head tracking) and a joystick. The 3D-video shows the cam-picture of the WashBot in the Perceptual Landscape. The viewing direction of the Bot's cameras corresponds whith the person's head movement. With the joystick, the Washbot can be navigated through the landscape. The Operator gets a haptic feedback of the road characteristics and surface conditions measured by the robot (climbing, turning, going down, etc.) though the hydraulically driven VR-chair. The haptic feedback dramatically increases the level of the Tele-Perception-Expierience.

The Operator is a person with great responsibility for his/herself and the Brainwashed Person, who's indoctrinated with the Operator's way travelling the Landscape. In this kind of play, real and more dramatic ways of using power and enforcement are point out and questioned.

There are meeting points for the entities 'Operator' and 'Washed': At certain places in the Perceptual Landscape, little video-monitors show the brain activity of the Washed: A place where all awareness of the TBS-System is focussed at one point - the Minds activity

3. Washbot Teleperception-Robot in Perceptual Landscape

The washbot is a mobile Robot-Platform, equipped whith stereoscopic video cameras, video-sender, compass and radio link. It gets it's navigation information from the Operation Area and drives at specified ways through the Perceptual Landscape. The Robot is used as an avatar that enables a human being to effect and expierience parts of 'the real'. It is the interface between virtual and physic space. Seen in that way, the Operator uses the WashBot like the Mind uses the Body, as an information collector and an effector in the real world.

The Perceptual Landscape

A former cooperation project between SHIFZ and NurSchrec! points out the philosophical and social implication of man/machine interdependencies in the so-called 'information-age'.

Martin Reiter (NurSchrec!):

Unser Projekt befasst sich mit dem umstand, daß seit dem eintritt in das "elektronische mittelalter" keine "verbindliche wirklichkeit" mehr existiert. alles ist wahr bzw. falsch und unzählige paralellwirklichkeiten werden notwendig, um konflikte - die dadurch entstehen, daß defacto eine verbindliche wirklichkeit existiert ( "alle leben auf dem selben planeten in letztlich 'einer' zeit") - und deren auswirkung zu verdrängen oder zumindest aus dem sinn (ausßerhalb von wahr-nehmung) zu bekommen.

Medien, unterhaltungstechnik/-industrie, cyberblabla, roboter, werkzeugmaschinen und dergleichen werden im informationszeitalter zum undurchdringlichen kokon, in dem das individuum- von verbindlicher wirk-lichkeit und verantwortung scheinbar befreit - in stasis versetzt wird. oder auch, wie im falle der automatisierung (im wirtschaftsjargon: "rationalisierung") der arbeit, überhaupt unnötig und "lästig" wird (lohnarbeitslose, die sich nicht selbst abschaffen ["töten ??!!"] und so zur "belastung" der "volkswirtschaft" werden.)

Wir setzen sich mit den noch vorhandenen schnittstellen und brüchen in dieser entwicklung auseinander. eine datenbrille und ihre verbindungen zu einem mechanischen apparat (robot) ermöglichen mensch, die "welt" (umgebung des apparates) durch die linsen und mikrophone der maschine (hier kann von einem robotavatar gesprochen werden) wahrzunehmen und den robot fernzusteuern. Über die audioverbindung kann mensch mit dem umfeld der maschine in kontakt treten. Über das netzspiel AL kann ein netzwerk von telerobotern entstehen, die den kokon der "unverbindlichen wirklichkeiten" zumindest hinterfragen und uns menschen vielleicht über die maschine stolpern lassen, die letztlich nur wir (menschine) selbst sind.