Books & People

back Books & People Overview

Books, People


Book Reviews


Links & More

Print version Print version

Related Links

document Featured People
document People

document People – In Memoriam

Background Links

document Columns
document Books

Pioneers of the Future

By Gerd Waloszek, SAP AG, SAP User Experience – November 7, 2012; updated: July 10, 2013

Here we started to compile short presentations of people who pioneered the future of user interfaces – either by thinking ahead, writing, or trying to build real systems. This list is highly subjective and arbitrary, and we hope that it will grow over time. Please send us your suggestions for further people that we should include in the list.

The texts are mostly based on Wikipedia entries, but may also contain personal statements from the author of this page.

List of Pioneers


Pioneers of the Future


Gordon Bell: MyLifeBits (Life-Logging)

Photo of Gordon Bell

C. Gordon Bell (born in 1934) is an American computer engineer and manager. Since 1995 he is a researcher at Microsoft Research, where is the "experimental subject" of MyLifeBits, a life-logging project (for details on Bell's bio, see the references below). The MyLifeBits project was inspired by Vannevar Bush's hypothetical Memex computer system. The project includes full-text search, text and audio annotations, and hyperlinks. The project will try to collect a lifetime of storage on and about Bell. Jim Gemmell of Microsoft Research and Roger Lueder were the architects and creators of the system and its software.

The MyLifeBits project is an experiment in life-logging (not the same as life-blogging) and an attempt to fulfill Vannevar Bush's vision of an automated store of the documents, pictures (including those taken automatically) (using the SenseCam digital camera), and sounds an individual has experienced in his lifetime, to be accessed with speed and ease. For this, Bell has digitized all documents he has read or produced, CDs, emails, and so on. He continues to do so, gathering web pages browsed, phone and instant messaging conversations and the like more or less automatically. The Dutton book, Total Recall, describes the vision and implications for a personal, lifetime e-memory for recall, work, health, education, and immortality (paperback: Your Life Uploaded: The Digital Way to Better Memory, Health, and Productivity).
(From Wikipedia, adapted)



Tim Berners-Lee: The World Wide Web

Photo of Tim Berners-Lee

Timothy John "Tim" Berners-Lee (born in 1955), also known as "TimBL", is a British computer scientist and the inventor of the World Wide Web. He made a proposal for an information management system in March 1989, and on 25 December 1990, with the help of Robert Cailliau and a young student at CERN, he implemented the first successful communication between a Hypertext Transfer Protocol (HTTP) client and server via the Internet.

Berners-Lee is the director of the World Wide Web Consortium (W3C), which oversees the Web's continued development (for details on Bell's bio, see the references below).
(In part from Wikipedia, adapted)



Vannevar Bush: Memex – First Ideas for a Hypertext System

Photo of Vannevar Bush

Vannevar Bush (1890-1974) was an American engineer and science administrator known for his work on analog computing, his political role in the development of the atomic bomb, and the idea of the Memex, which was seen decades later as a pioneering concept for hypertext systems and the World Wide Web (for details on Bush's bio, see the references below).

In 1945, Bush published the essay As we may Think, in which he describes the Memex system, a visionary system augmenting human memory that can be seen as a precursor of hypertext and thus the Internet. A Memex is an electromechanical device enabling individuals to develop and read a large self-contained research library, create and follow associative trails of links and personal annotations, and recall these trails at any time to share them with other researchers. This device would closely mimic the associative processes of the human mind, but it would be gifted with permanent recollection. As Bush writes, "Thus science may implement the ways in which man produces, stores, and consults the record of the race".

The technology used would have been a combination of electromechanical controls, microfilm cameras and readers, all integrated into a large desk. Most of the microfilm library would have been contained within the desk, but the user could add or remove microfilm reels at will. The top of the desk would have slanting translucent screens on which material could be projected for convenient reading. The top of the Memex would have a transparent platen. When a longhand note, photograph, memoranda, or other things were placed on the platen, the depression of a lever would cause the item to be photographed onto the next blank space in a section of the Memex film.

The vision of the Memex predates, and is credited as the inspiration for, the first practical hypertext systems of the 1960s. The Memex proposed by Bush would create trails of links connecting sequences of microfilm frames, rather than links in the modern sense where a hyperlink connects a single word, phrase or picture within a document and a local or remote destination.
(From Wikipedia, adapted)



Bill Buxton: Gesture Interaction & Multi-Touch

Photo of Bill Buxton

Bill Buxton (born in 1949) pioneered gesturel interaction and multi-touch interfaces in the late 70s.

In 1975 Buxton started designing his own digital musical instruments. It is also the path that brought him into the field of human-computer interaction, which is his technical area of specialty. He pioneered music composition tools and gesture interaction multi-touch interfaces in the late 70s, while working in the Dynamics Graphics Project at the University of Toronto, where he still is an adjunct professor.

From 1987-89, Buxton was in Cambridge England, helping establish a new satellite of Xerox's Palo Alto Research Center (EuroPARC). From 1989-94 he split his time between Toronto, where he was Scientific Director of the Ontario Telepresence Project, and Palo Alto, California, where he was a consulting researcher at XEROX PARC. Then he was Chief Scientist at Alias Wavefront and SGI, before he joined Microsoft Research in December 2005.

Buxton's scientific contributions include applying Fitts' law to human-computer interaction and the invention and analysis of the marking menu (together with Gordon Kurtenbach). Recently, he became also known for his 2007 book Sketching User Experiences: Getting the Design Right and the Right Design.

Buxton received the SIGCHI Lifetime Achievement Award in 2008 for his many fundamental contributions to the human–computer interaction field.
(From Wikipedia and homepage, adapted)



Douglas Engelbart: NLS and the Mouse

Photo of Doug Engelbart

Douglas Carl Engelbart (1925-2013) was an American inventor, and an early computer and internet pioneer (for details on Engelbart's bio, see the references below). He is best known for his work on the challenges of human–computer interaction, particularly while at his Augmentation Research Center Lab in SRI International, resulting in the invention of the computer mouse, and the development of hypertext, networked computers, and precursors to graphical user interfaces. Engelbart designed NLS, or the "oN-Line System", a revolutionary computer collaboration system from the 1960s that was implemented by researchers at the Augmentation Research Center (ARC) at the Stanford Research Institute (SRI). The NLS system was the first to employ the practical use of hypertext links, the mouse, raster-scan video monitors, information organized by relevance, screen windowing, presentation programs, and other modern computing concepts.
(From Wikipedia, adapted)

Engelbart was still going to work at his "Bootstrap Alliance," trying to persuade people of the value of his ideas about augmenting the human intellect, and evangelizing the virtues of his "Augment System." He has always wanted to create designs that enhance human performance and is not interested in ease of use for the novice.
(From page on Designing Interactions Website, adapted)


Photo of the NLS with raster-scan video monitor, keyboard, and mouse

Figure: Photo of the NLS with raster-scan video monitor, keyboard, and mouse



Hiroshi Ishii: Tangible Bits, Media, and User Interfaces

Hiroshi Ishii (born 1956) is a Japanese computer scientist. He is a professor at the Massachusetts Institute of Technology (MIT). Ishii pioneered the Tangible User Interface in the field of Human-computer interaction.

Hiroshi Ishii founded the Tangible Media Group and started their ongoing Tangible Bits project in 1995, when he joined the MIT Media Laboratory as a professor of Media Arts and Sciences. Ishii relocated from Japan's NTT Human Interface Laboratories in Yokosuka, where he had made his mark in Human Computer Interaction (HCI) and Computer-Supported Cooperative Work (CSCW) in the early 1990s.

I attended presentations by Ishii at two CHI conferences: at CHI 1997, where he held the presentation together with his student Brygg Ulmer, and at CHI 2008. Moreover, Brygg Ulmer once gave a presentation at SAP.
(From Wikipedia, adapted)


Ishii and Ulmer at MetaDesk   Physical icon

Figures: MetaDesk and Tangible Geospaces (with physical icons)



Alan Kay: Object-Oriented Programming and Windowing Systems

Photo of Alan Kay

Alan Kay (born in 1940) is a pioneer in object-oriented programming and graphic user interfaces (GUIs), particularly window systems (for details on Kay's bio, see the references below).

In 1970, he joined Xerox Corporation's Palo Alto Research Center, PARC, and in the 1970s he was one of the key members there to develop prototypes of networked workstations using the programming language Smalltalk. Thus, he is one of the fathers of the idea of object-oriented programming and was also the architect of the modern overlapping windowing graphical user interface (GUI). These inventions were later commercialized by Apple Computer in their Lisa and Macintosh computers.

Kay also conceived the Dynabook concept, which defined the conceptual basics for laptop and tablet computers and E-books. Because the Dynabook was conceived as an educational platform, Kay is considered to be one of the first researchers into mobile learning. Many features of the Dynabook concept have indeed been adopted in the design of the One Laptop Per Child educational platform, with which Kay is actively involved.

Alan Kay once stated: "The best way to predict the future is to invent it."
(From Wikipedia, adapted)



Steve Mann: Wearable Computing and Lifeglogging

Photo of Steven Mann

Steve Mann (Steven Mann; born in 1962) is a pioneer in wearable (or bearable) computing (for the distinction between wearable and bearable computing, see Mann's article Wearable Computing in the HCI Encyclopedia). While at MIT, Mann was one of the founding members of the Wearable Computers group in the Media Lab (for details on Mann's bio, see the references below).

In the 1970s and early 1980s Mann designed and built a number of general-purpose wearable computer systems, including various kinds of sensing, biofeedback, and multimedia computers such as wearable musical instruments, audio-based computers, and seeing aids for the blind. In 1981 Mann designed and built a backpack-based general-purpose multimedia wearable computer system with a head-mounted display visible to one eye. The system provided text, graphics, audio, and video capability, and included a handheld chording keyer (for one-handed input). Because of its generality, this system fit the description of what most people would call a "computer" by today's standards. The system allowed various computer applications to be run while walking around doing other things. The computer could even be programmed while walking around. Among the applications written for this wearable computer system was an application for photographically mediated reality and "lightvector painting" ("lightvectoring") used extensively throughout the 1980s. A variety of different systems were designed and built by Mann in the 1980s, and this marked a steady evolution in wearable computing toward something resembling ordinary eyeglasses by the late 1990s.


Evolution of prosthesis

Figure: The evolution of prosthesis, demonstrated by Steve Mann (photo of Amber Case's slides at Interaction 2012)

By 1994 Mann had streaming live video from his wearable computer to and from the World Wide Web, such that viewers to his web site could see what he was seeing, as well as annotate what he was seeing. This "Wearable Wireless Webcam" was the first embodiment of live webcasting from a wireless device. In 1998 Mann made a working prototype of a wristwatch computer running GNU Linux. The wristwatch included video-conferencing capability and was demonstrated at the ISSCC 2000 conference.

From 1994-1996, Mann conducted a Wearable Wireless Webcam experiment, called lifeglogging, where he streamed live video from his wearable computer to and from the World Wide Web, on an essentially 24 hour-a-day basis. For the most part, the wearable computer streamed continuously although the computer itself was not waterproof so it needed to be set aside during showering or bathing. As a personal data capture, Wearable Wireless Webcam raised some new and interesting issues in the capture and archival of a person's entire life from their own perspective. And it also opened up some new ideas such as the roving reporter, where day-to-day living can result in serendipitous capture of newsworthy events.
(From HCI Encyclopedia, adapted)



Ted Nelson: Hypertext

Photo of Ted Nelson

Ted Nelson (born in 1937) is one of the hypertext pioneers (he coined this term). His biggest project was the Xanadu hypertext system, which has never been completed.

Nelson founded Project Xanadu in 1960 with the goal of creating a computer network with a simple user interface. The effort is documented in his 1974 book Computer Lib / Dream Machines and the 1981 Literary Machines. Much of his adult life has been devoted to working on Xanadu and advocating it. The Xanadu project itself failed to flourish, for a variety of reasons. Journalist Gary Wolf published an unflattering history on Nelson and his project in 1995, calling it "the longest-running vaporware project in the history of computing". Nelson objected to Wolf's view and threatened to sue him.

Nelson claims that some aspects of his vision are in the process of being fulfilled by Tim Berners-Lee's invention of the World Wide Web, but he dislikes the World Wide Web – regarding Berners-Lee's work as a gross over-simplification of his original vision:

  • "HTML is precisely what we were trying to PREVENT – ever-breaking links, links going outward only, quotes you can't follow to their origins, no version management, no rights management."

(From Wikipedia, adapted)



Rosalind Picard: Affective Computing

Photo of Rosalind Picard

Rosalind W. Picard (born in 1962) is credited with starting the branch of computer science known as affective computing with the publication of Affective Computing in 1997, which described the importance that recognizing human emotions has to relationships between people, and the possible effects of such recognition by robots.

Picard is a researcher in the field of affective computing and the founder and director of the Affective Computing Research Group at the MIT Media Lab. This group develops tools, techniques, and devices for sensing, interpreting, and processing emotion signals that drive state-of-the-art systems which respond intelligently to human emotional states. The key aspect that Picard focuses on in her research is not in the difference between "excited" and "calm" emotions, but in the difference between "excited-happy" and "excited-angry-or-upset", which are complicated for a computer to determine. Applications of their research include improved tutoring systems and assistive technology for use in addressing the verbal communications difficulties experienced by individuals with autism. Thus, her work in this field has led to an expansion into autism research and developing devices that could help humans recognize nuances in human emotions.

Picard also works with Sherry Turkle and Cynthia Breazeal in the fields of social robots, digital image processing, pattern recognition, and wearable computers. Picard's former students include Steve Mann, professor and researcher in wearable computers.
(From Wikipedia, adapted)



Bruce Sterling: Spimes, and Biots – Ideas for the Future of Computing

Photo of Bruce Sterling

Michael Bruce Sterling (born in 1954) is an American science fiction author who is best known for his novels and his seminal work on the Mirrorshades anthology. This work helped to define the cyberpunk genre.

Sterling has a habit of coining neologisms to describe things which he believes will be common in the future, especially items which already exist in limited numbers. For example, in 2004 he suggested a type of technological device, which he called he called "spime" and which, through pervasive RFID and GPS tracking, can track its history of use and interact with the world.

Sterling describes spimes in his 2005 book Shaping Things, where he also presents an outlook on biots, which is hard to agree with for me. I feel reminded of ideas of Hans Moravec and Raymond Kurzweil who propose that robots will take over in a few decades and that there will be no room for mankind left. My assessment is based on statements about biots, such as "You are no longer human. ... you have a wide variety of interesting, challenging problems. It's just that none of those are human ones." Or: "The human condition isn't abolished overnight, it isn't obliterated. It is composted." Even though Sterling tries to counter the fears of some of his readers here, this is a technology-centered view of society (and a direction inspired by artificial intelligence called transhumanism), not a human-centered one. In my opinion, at the end of his book Sterling becomes a science fiction author again and gives his fancy full scope into a direction that I personally reject. In it's final consequence, I can neither follow Sterling's techno-centered optimism nor his transhumanistic perspective.
(In part from Wikipedia, adapted; other parts are taken and adapted from my review of Shaping Things)



Ivan Sutherland: Sketchpad – Pioneering Graphics, GUIs, and Object-Oriented Programming

Photo of Ivan Sutherland

Ivan Sutherland (born in 1938) invented Sketchpad, an innovative program that influenced alternative forms of interaction with computers.

Sutherland invented Sketchpad (aka Robot Draftsman), a revolutionary computer program in 1963 in the course of his PhD thesis. It helped change the way people interact with computers. Sketchpad is considered to be the ancestor of modern computer-aided drafting (CAD) programs as well as a major breakthrough in the development of computer graphics in general. For example, the graphic user interface (GUI) was derived from the Sketchpad as well as modern object oriented programming. Ivan Sutherland demonstrated with it that computer graphics could be used for both artistic and technical purposes in addition to showing a novel method of human-computer interaction.

Sketchpad could accept constraints and specified relationships among segments and arcs, including the diameter of arcs. It could draw both horizontal and vertical lines and combine them into figures and shapes. Figures could be copied, moved, rotated, or resized, retaining their basic properties. Sketchpad also had the first window-drawing program and clipping algorithm, which allowed zooming (see the video with Alan Kay). Sketchpad ran on the Lincoln TX-2 computer and influenced Douglas Engelbart's oN-Line System (NLS). Sketchpad, in turn, was influenced by the conceptual Memex as envisioned by Vannevar Bush in his influential paper As We May Think.
(From Wikipedia adapted)

Photo ofr Sketchpad

Figure: Sketchpad




Kevin Warwick: Bearable Computing (Implants) and Cyborgs

Photo of Kevin Warwick

Kevin Warwick (born in 1954) is a British scientist and professor of cybernetics at the University of Reading, United Kingdom. He is known for his studies on direct interfaces between computer systems and the human nervous system, and has also done research in the field of robotics.
(From Wikipedia, adapted)

Warwick instigated a series of pioneering experiments involving the neuro-surgical implantation of a device (Utah Array/BrainGate) into the median nerves of his left arm in order to link his nervous system directly to a computer to assess the latest technology for use with the disabled. The development of the implant technology was carried out by a team of researchers headed by Dr. Mark Gasson. Warwick was successful with the first extra-sensory (ultrasonic) input for a human and with the first purely electronic communication experiment between the nervous systems of two humans. His research has been discussed by the US White House Presidential Council on BioEthics, The European Commission FTP and led to him being widely referenced and featured in academic circles as well as appearing as cover stories in several magazines.
(From Warwick's homepage, adapted)

At the CHI 2002 conference in Minneapolis, I attended a remote interview with Warwick. At that time he was already exploring electronic transplants and also planned the above-mentioned electronic communication experiment between the nervous systems of two humans (he and his wife). By the way, just recently, I learned that Steve Mann includes both implantable and portable devices like smartphones in his definition of "wearable", or as he prefers to say, "bearable" or "body-borne" computing.



Mark Weiser: The Disappearing Computer – Ubiquitous Computing

Photo of Mark Weiser

Mark Weiser (1952-1999) is widely considered to be the father of ubiquitous computing, a term he coined in 1988.

Weiser worked for a variety of computer related startups, but his seminal work was in the field of ubiquitous computing while leading the computer science laboratory at PARC, which he joined in 1987. Weiser's ideas were significantly influenced by his father's reading of Michael Polanyi's The Tacit Dimension. He became head of the computer science laboratory at PARC in 1988 and chief technology officer in 1996, authoring more than eighty technical publications. His best known article is from 1991 and is entitled The Computer for the 21st Century.

Weiser described ubiquitous computing as follows:

  • "Ubiquitous computing names the third wave in computing, just now beginning. First were mainframes, each shared by lots of people. Now we are in the personal computing era, person and machine staring uneasily at each other across the desktop. Next comes ubiquitous computing, or the age of calm technology, when technology recedes into the background of our lives."

During one of his talks, Weiser also outlined a set of principles describing ubiquitous computing:

  • The purpose of a computer is to help you do something else.
  • The best computer is a quiet, invisible servant.
  • The more you can do by intuition the smarter you are; the computer should extend your unconscious.
  • Technology should create calm.

In Designing Calm Technology, Weiser and John Seely Brown describe calm technology as "that which informs but doesn't demand our focus or attention."
(From Wikipedia, adapted)



top top