Editorials

back  To Overview of Editorials

Editorials 2010

Editorials 2009

Editorials 2008

Editorials 2007

Editorials 2006

Editorials 2005

Editorials 2004

Editorials 2003

Editorials 2002

Editorials 2001

Editorials 2000

 

From GUIs to Narrative Interfaces – From Point-and-Click to Computers Telling Stories

By Gerd Waloszek, SAP AG, Usability Engineering Center – Updated: January 22, 2004; audio version February 23, 2005

In the late seventies and early eighties, when the first graphical user interfaces appeared on the market, many people heralded these interfaces as intuitive, that is, easy to learn and use. According to them, graphical user interfaces would remove the barrier between computer systems and users and thus make computers accessible for everyone. Simple, intuitive physical interactions — Ben Shneiderman coined the term "direct manipulation" for them — replaced cryptic character-based commands, which were only known by, and understandable to, experts. The mouse took its place on the desktop and commanded the virtual desktop on the computer screen.

These days, history seems to be repeating itself: Narrative interfaces based on the story-telling paradigm set out to start a second revolution in computer interfaces in order to make computers "really" accessible for everyone. In fact, even though graphical user interface have opened computers to the masses, many people still have problems with using them. And the problems are increasing and may have a huge economic impact.

 

Graphical User Interfaces Are Reaching Their Limits

How is that possible, considering the simple actions by which graphical interfaces are controlled? First, graphical user interfaces still obey a set of conventions, which has to be acquired. Even though these conventions are simple and easy to remember, they still have to be acquired. When I unpacked my first Mac years ago and turned it on, I could not do anything with it because I did not know that I had to double-click in order to open an application. I expected to be able to use my Mac without having to read the manual, but that was not at all the case. Just like any other user of graphical interfaces, I had to learn all those conventions for clicking, double clicking, dragging, dropping, shift click, option click, and so on. As a long-time user you may have forgotten this learning effort. But you will encounter it again, whenever you help newcomers during their first steps at the computer. Once you have grasped the conventions, however, you have got the basics. Next you have to learn the menu structure, where to find which command, and how to proceed in order to achieve something. With graphical user interfaces the user is in command, not the computer — and that is where the problems start with graphical user interfaces: Users are given only very limited guidance for processing their tasks with the computer.

The massive problems currently emerging with graphical user interfaces, though, may not have been realized at first. They arise because of the huge range of possibilities graphical user interfaces offer application designers. Nowadays, you can do nearly anything on your computer: write letters, create drawings and even paintings, manipulate photos, lay out magazines, fill in forms, or create and manipulate complex 3D objects — to name just a few of the many options. Consequently, having more complex tasks means that the handling of the applications also becomes more complex, even though the elementary actions are so simple. Nowadays it is often the case that companies using computers have to invest more money in training and retraining their employees when software is introduced or updated than for the software installation itself. Thus, ease of use has become an issue of economic significance. In addition, as updates are delivered faster and faster, people can no longer keep up with all the new features, additions, and changes to the applications they have to use. Thus, software has to be intuitive and easy to use — without any fallback to manuals — in order to remain cost-effective for companies.

 

Narrative Interfaces – A Solution to the Training Dilemma?

This is where narrative user interfaces come into play. This new paradigm sets out to finally make computers usable for everyone, without recourse to manuals. The idea behind narrative interfaces is as simple as it is difficult to implement: Computers mimic the communication behavior of humans. They talk to people, listen to them, and even recognize the situational context. In addition, they offer representations of humans, such as avatars, which have realistic facial expressions and can show emotions. Finally, they offer "natural" physical devices for people to interact with the system. Thus, users interact with the computer in a "mixed reality" of virtual and physical objects, where the computer's behavior comes close to human behavior, and thus is easy for people to understand.

IZA prototype

Figure 1: The IZA prototype from the German Fraunhofer IDG is an example of a system based on a narrative user interface (from Fraunhofer IDG)

Considering these high claims, it is no wonder that narrative interfaces rely heavily on

 

Computers Telling Stories

The most important aspect of narrative interfaces is their ability to engage in "natural" conversations with users. The paradigm behind the conversational aspect of narrative interfaces is called storytelling. Authors of conversational computer programs have to take into account how humans tell stories and conduct conversations. They have to let the dialog follow a certain dramaturgy that finally leads to the fulfillment of the computer's goal. Techniques from the film industry, theater, and literature provide the necessary knowledge and have to be transferred to the design of computer interfaces.

The conversation between the computer and a user is driven in pursuit of certain goals given by the application. The computer is no longer passively waiting for user commands, as with graphical user interfaces, nor does it issue commands, as with command-driven interfaces, but behaves proactively. It also has to observe the user and his or her goals as well as the situational context when developing a strategy for pursuing its goals. This is not an easy task, as the conversation involves alternations (that is, user and computer talk alternately), may be unstructured, or even end abruptly.

This is complemented by the computer's ability to express emotions through the facial expressions of an avatar, combined with lip-synch speech and gestures.

 

Developing Narrative Applications

Authors of narrative user interfaces surely do not want to bother with the complexities of AI and graphic programming. Therefore, a development infrastructure has to be offered that is easily usable by authors of narrative applications. A simple example may illustrate this point: When Microsoft implemented its wizards (for example, the infamous Clippie), it put much investigation and effort into realizing a realistic movement behavior of the wizard. For programmers using the wizards, however, Microsoft offered a simple and fairly easy to use Visual Basic interface. Thus, the programmer who used the wizard in his or her applications did not need to bother with the movement behavior, but simply determined the messages and the global behavior of the wizard.

For narrative interfaces, Professor José L. Encarnação of the Fraunhofer Institute for Computer Graphics Research (IGD) in Darmstadt, Germany proposed that there will be a distribution of responsibilities among application authors: One author may deliver the content, another may script the conversation, and a third one may take care of the graphical representation and the actual user interface of the system.

 

Outlook

Current prototypes of narrative systems show that these interfaces are still in their early stages — there is plenty of room for improvement. Because much artificial intelligence involved, I expect that the development of foolproof narrative systems will take much longer than it took for graphical user interfaces. Perhaps we will have to wait a decade until we see such systems in our offices and homes and in widespread use.

While there are still immense problems to be overcome on the technical side, the user experience aspect can be tested today without too much effort and technical equipment. For example, to test the user interface of a narrative information booth, you could simply build a simple booth and seat a person behind it. This person may play the role of an avatar and interact with users as the application would do. The person may select the texts of the basis of given scripts and read them, identify the user's behavior, motivation, emotions, the situational context, and so on. If you want to involve more technique, a computer may help the person to select the texts; in order to avoid misreadings, a speech synthesizer might read the selected texts. Thus, researchers can already test whether their concepts apply to real-world situations, such as a visit to an information booth at a trade show. There is no need to wait for user tests until the technique works satisfactorily. On the contrary, you can shape the development of the upcoming technique on the basis of the behavior of real users now.

Let me conclude this article with the efficiency issue. When graphical user interfaces were introduced, many authors questioned the efficiency of those interfaces, especially the need to use the mouse instead of the keyboard and to switch between both input devices. In the course of the long evolution of graphical user interfaces, this issue has largely been settled. Now, with narrative user interfaces, this discussion will arise once more. One of the possible causes for the inefficiency of narrative interfaces is the linear flow of spoken text. For example, it may take quite a while for an avatar to list the possible options for a user. On the other hand, such an interface may be much more efficient in specifying items in large sets, where visual scanning would be time-consuming or even impossible. We require more experience with these new interfaces to be able to assess their strengths and weaknesses.

 

Addendum

This is only an initial introduction to narrative interfaces. A more in-depth article on this topic can be found in the Design Tidbits of the SAP Design Guild.

See also Future Scope - Tangible Information

References

 

Definitions

Avatar: In the Hindu religion, an avatar is an incarnation of a deity; hence, an embodiment or manifestation of an idea or greater reality. In 3D or virtual reality games and in some chat forums on the Web, an avatar is the visual "handle" or display appearance visitors use to represent themselves ("bodily incarnation in Cyberspace"). Generally, an avatar is a "virtual" representation of a human.

 

top top