Performance Glossary

By Gerd Waloszek, SAP User Experience, SAP AG – Updated: September 30, 2010

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z

The following glossary collects terms from the area of human and computer performance as well as responsiveness.

 

A

AJAX

Ajax (asynchronous JavaScript and XML), or AJAX, is a group of interrelated web development techniques used for creating interactive Web applications or rich Internet applications. With Ajax, Web applications can retrieve data from the server asynchronously in the background without interfering with the display and behavior of the existing page. Data is retrieved using the XMLHttpRequest object or through the use of remote scripting in browsers that do not support it. Despite the name, the use of JavaScript, XML, or its asynchronous use is not required.

Source: http://en.wikipedia.org/wiki/AJAX (Wikipedia)

(Wait) Animation

See Wait Animation

B

Busy Feedback

Busy feedback tells the user that the system cannot keep up, that it has become unresponsive, and that the user will have to wait for the computer's response. It may also assure the user that the system is still working and is not hanging. In its most advanced form, it can also tell the user the reason for the delay and its expected duration. We distinguish between the following types:

  • Busy indicator feedback is the simplest form of busy feedback: Some sort of indicator tells the user that the system is busy and will not accept user input, but neither why it is busy, nor for how long.
  • Working indicator feedback is a variant of this feedback (McInerney & Li, 2002). It tells the user that the system is still busy and is not hanging, typically by providing some sort of animation. In practice, both types of feedback are often provided by the same indicator, so that I will refer only to busy indicators in the following.
  • System messages tell the user the reason for a delay and might therefore be regarded as "informed feedback." The message may change according to the current process. Again, no information about the duration of the delay is provided.
  • Progress indicators tell the user (ideally) how long a process will last and what is currently going on. They, too, offer "informed feedback" – the best feedback one can give, provided that the progress indicator does not try to fool the user.

Busy Indicator, Busy Indication

System feedback using a busy indicator (the feedback can be called busy feedback or busy indication). Busy indicator feedback is the simplest form of busy feedback: Some sort of indicator, the "busy indicator," tells the user that the system is busy and will not accept user input, but neither why it is busy, nor for how long.

Hourglass cursor

Figure: An hourglass cursor can be used for busy indication

C

Computer/System Idle Time

Time that the computer or device has been powered but has not been used. If a computer or computer device is idle for a set period of time it may go into a standby mode or power off. (ComputerHope.com)

Note: This time can be used as a resource for planning and calculating ahead and thereby reducing latencies (most of the time, modern computers are idle and wait for user input).

(Wait) Cursor

See Wait Cursor

D

Delay

Short for a delayed system response to a user action. From a user's perspective also called waiting time. Depending on the duration of the delay, the user's expectations, and the type of process, the delay causes different psychological reactions.

See also Waiting Time, Response Time, Time Ranges, and Tolerance for Waiting Times

E

"End-of-Process" Feedback

"End-of-process" feedback is given after a lengthy process has finished. Tognazzini suggests on his Website: Make the client system beep and give a large visual indication upon return from lengthy (>10 seconds) processes, so that users know when to return to using the system. Thus, this feedback is useful because users may have turned to other work after about 10 seconds. The feedback should not only be given visually but also acoustically because the user may have hidden the window that belongs to the process or even have left the computer.

Efficiency

Efficiency is regarded as a component of usability and addresses the question:

  • Once users have learned the design, how quickly can they perform tasks?

According to Deborah Mayhew (2008), efficiency is:

  • The speed with which a user can accomplish a task, given:
    • The strengths and weaknesses of the human information processing system
    • The user interface design of the tool(s) being used

Designing for efficiency means (Mayhew, 2008):

  • Optimizing the average task time made possible by a user interface design when all other factors are also optimized

Efficiency is the primary usability goal when (Mayhew, 2008):

  • Users are highly trained
  • Users perform tasks repetitively with a very high frequency
  • Tasks are fairly structured and linear without a great deal of variability in how they are performed
  • User time is at a premium

Source: Mayhew, Deborah (2008). User Efficiency: Evaluation and Design.

F

Feedback, Busy

See Busy Feedback

Feedback, System Feedback

In the context of responsiveness, Jeff Johnson (2007) talks about "timely feedback" and discusses several types of it, such as immediate feedback, busy, working, and progress indicators.

Different time ranges require different types of feedback:

  • At the perceptual level, immediate feedback is mandatory to maintain the relationship of cause and action: To assure users that a command has been acknowledged, a mouse move or drag goes into the right direction, a key press has been registered, and so on.
  • At the dialog level, user and system take turns, and users are deeply engaged in the ongoing man-machine dialog. Any delays irritate users and therefore have to be indicated to them. Here, the mere indication of a delay is sufficient (busy feedback, working feedback).
  • At the cognitive level, users have to wait for a certain amount of time before they get a response. So they start to think about why they have to wait, evaluate the system's performance, and guess what it may have to do. As a result, the impression of slowness comes up, the users' attention starts to wander, and their focus on the task wanes. Users may even switch to other tasks if the delay is too long, or they get annoyed. This "cognitive awareness" on the users' side requires a more qualified feedback, ideally one that informs them about the cause and duration of the delay and also asks them for their understanding for the delay. It may be also useful to provide an "end-of-process" indication because the users may have turned to other work after about 10 seconds.

The table below shows the connection between human time ranges and different types of feedback:

Time Range 0.1 (0.05-0.2) sec * 1 (0.2-2.0) sec 3 (2-5) sec 10 (5-15) sec >15 sec
Human Level Perceptual level: Perception, motor actions

Dialog level: Dialog, operation, action
Cognitive level: Thinking, attention, motivation
Hand-eye coordination, cause-and-effect relationships, smooth animation Focus on task Focus of attention starts to wander (3 sec) Focus of attention lost (10 sec) Motivation lost
User inputs commands and data and observes system User is engaged in dialog System is perceived as slow (3 sec) User probably moves over to other tasks (if possible) User gets annoyed
UI Level Direct manipulation: Mouse movement, drag, click, key press, smooth animations, highlighting, open menus, ...

Dialog

Present result of simple task**

 Present result of common task** Present result of complex task** or finish compound task (unit task) Present result of very complex task
Type of Feedback

Immediate feedback (acknowledgement): Highlighting, menu open/close, ...

Immediate feedback (options for action): hover effects, ...

 "Busy" indicator: Wait cursor "Working" indicator: Loading animation (no clue indicator)
Progress indicator (duration, type of operation)
Activity message(s) (type of operation, step number, ...)
--- "End-of-process" indicator (signals that process has ended)

Combining time ranges and their psychological interpretations with interaction contexts leads to the following matrix of recommendations for providing system feedback:

Duration
(Variation)		 Human
Level
Interaction Context
Loading Tools Work Actions Monitor. Processes User Input
0.1
(0.05-0.2)		 Perception, motor actions --- --- Informed feedback: progress indicator ** Immediate feedback
1
(0.2-2)		 Dialog, operations Busy feedback: static/animated cursor, animation Working feedback: static/animated cursor, animation Informed feedback: progress indicator ---

> 3
(2->15)

 Cognition*:
Thinking, attention, motivation

Busy feedback: static/animated cursor, animation

Informed feedback: progress indicator, system message ***

Informed feedback: progress indicator, system message

"End-of-process" feedback ****

Informed feedback: progress indicator

"End-of-process" feedback ****

 ---

Table: Recommendations for the use and the timing of different kinds of feedback in the event of a delay

Feedback, Informed

See Informed Feedback

H

Human Performance

Human performance is characterized by the amount of useful work accomplished by a human compared to the time and resources used. (Preliminary definition)

Human Time Ranges

See Time Ranges

I, J

Immediate Feedback

Immediate feedback assures users that a command has been acknowledged, a mouse-move or drag is going into the right direction, a key press has been registered, and so on. It can be provided in many ways, such as by changing the color, the shading (see figure below), or the look of the screen element that the user interacts with.

Changing the shading of a button in order to show that a button has been clicked

Figure: Changing the shading of a button in order to show that a button has been clicked

In the case of menus, an animation effect – such as the opening or closing of the menu – shows that the direct-manipulation action (not the menu command!) has been acknowledged. Similarly, for mouse drags, either the screen element itself or a simplified representation of it – such as a rectangular frame – is moved. The latter effect should be used if the object itself cannot be moved in a timely manner. "Hover" effects are a special case. Rather than indicating that an action has been acknowledged, they indicate an available option. For example, a hover effect over a button or link indicates that this screen element can be clicked (see figure below, center).

Three variants of a button in a Web application - not pressed, hover, pressed

Figure: Three variants of a button in a Web application – not pressed, hover, pressed

Informed Feedback

Feedback that informs users about the cause and duration of the delay and also asks for some understanding for it – such as by explaining what is going on in the computer (in user-oriented language, of course). There are two types of informed feedback: progress indicators and system messages.

Windows progress indicator

Figure: Progress indicator for file copy in Windows XP

As soon as users have to wait for a few seconds before they get a system response, they switch to the cognitive level: That is, they have time to think and speculate about the reasons for the wait, and they make remarks about the system's performance. A mere three seconds suffice to make the system appear slow to users; some authors even say that one or two seconds are enough. The users' attention starts to wander; their focus on the task at hand wanes and is eventually lost (at about 10 seconds). Users may even start other tasks, such as checking their e-mail, if the delay lasts too long, and eventually they get annoyed (at about 15 seconds). Thus, the cognitive level includes aspects of thinking, attention, and motivation.

Cognitive activity on the user side requires more qualified feedback from the system; feedback that informs users about the cause and duration of the delay and also asks for some understanding for it – such as by explaining what is going on in the computer (in user-oriented language, of course). There are two types of informed feedback that I would like to discuss in the following: progress indicators and system messages.

L

Loading Animation

See Wait animation

P

Perceived Performance

From Wikipedia

  • Perceived performance, in computer engineering, refers to how quickly a software feature appears to perform its task. The concept applies mainly to user acceptance aspects.
    The amount of time an application takes to start up, or a file to download, is not changed by showing a startup screen (see Splash screen) or a file progress dialog box. However, it satisfies some human needs: it appears faster to the user as well as providing a visual cue to let them know the system is handling their request.

Source: http://en.wikipedia.org/wiki/Perceived_performance (wikipedia)

From Perceived Performance - Tuning a System for What Really Matters (White Paper by Tim Mangan, June 2004)

  • Perceived performance is our term for describing a methodology where one analyzes the system with a goal of improving user productivity. It focuses on issues that affect the performance of the system as perceived by the people using it.

Source: http://www.tricerat.com/support/docs/files/whitepapers/WhitepaperSimplifyResources.pdf; http://www.tmurgent.com/WhitePapers%5CPerceivedPerformance.pdf (September 2003)

Perceived Speed

More or less the same as perceived performance but reduced to the "speed" aspect.

Performance

From Wikipedia

Computer performance is characterized by the amount of useful work accomplished by a computer system compared to the time and resources used.

Depending on the context, good computer performance may involve one or more of the following:

  • Short response time for a given piece of work
  • High throughput (rate of processing work)
  • Low utilization of computing resource(s)
  • High availability of the computing system or application

Source: http://en.wikipedia.org/wiki/Computer_performance (wikipedia)

Note: This definition includes response times, which may not necessarily mean the same as responsiveness.

From Jeff Johnson GUI Bloopers

Performance, (...), has to do with how quickly the software computes and displays the desired results.

Note: Johnson distinguishes between performance and responsiveness and criticized that both are often equated.

Source: Jeff Johnson (2000). GUI Bloopers: Don'ts and Do's for Software Developers and Web Designers. Morgan Kaufmann Publishers (Chapter 1: First Principles; Principle 7: Design for Responsiveness, p. 368).

Planning Time

See User Planning Time

Progress Indicator, Progress Indication

Progress indicators tell the user (ideally) how long a process will last and what is currently going on. They provide the best busy feedback because they inform users about the remaining waiting time and typically also the ongoing process. Progress indicators can be graphical, textual, or a combination of both. According to Jeff Johnson (2007), they greatly increase the perceived responsiveness of an application or a system, even though they do not shorten the actual time required to complete an operation. This is probably due to the fact that users can adapt their behavior according to the expected waiting time. They are also given reasons why they have to wait rather than being left in the dark.

Windows progress indicator

Figure: Progress indicator for file copy in Windows XP

Detailed guidelines for designing progress indicators can be found in McInerney & Li (2002).

Figure: Example of a double progress indicator

Phony Progress Indicators

Regrettably, not every progress indicator can be trusted: There are a lot of so-called "phony" progress indicators out there. According to Jeff Johnson, in a "phony" progress bar the fullness of the bar or the displayed value does not show the operation's actual progress. Phony progress indicators are just fancy busy indicators – even worse, they try to fool users. There are several variants of them:

  • Some indicators show the progress for each item; they fill up once for each item, starting over for the next one; users do not have any orientation with respect to the whole process.
  • Others simply fill up cyclically in correspondence to nothing particular.
  • Some progress indicators fill up to 100%, state that the remaining time would be short, and remain unchanged for minutes.
  • Animations of the progress bar can indicate progress by inducing an illusory movement. This is particularly annoying for protracted processes where the progress bar creeps forward very slowly (or not at all).

R

Response Time

From Designing the User Interface

Response time is defined as the number of seconds it takes from the moment users initiate an action (usually by pressing an ENTER key or mouse button) until the computer begins to present results on the display, printer, loudspeaker, or mobile device. (S&P, 455)

When the response is complete, users begin formulating the next action.

The user think time is the number of seconds during which users think before initiating the next action. (S&P, 455-456)

Simple stages of action model:

User initiates activity Computer responds  
  <------ Response time ------>   <------User think time ------>  

Figure: Simple stages of action model of system response time and user think time

More realistic model:

User starts     
typing       		User initiates
activity		 Computer
begins response     		Computer
completes response 		 
            
<------ Response time ------>
------>
<----- User planning time ------>
------>
<------
<------User think time ------>

Figure: Simple stages of action model of system response time and user think time

Source: Ben Shneiderman & Cathérine Plaisant (2004). Designing the User Interface (4th Edition). Pearson Addison-Wesley, pp. 454ff.

From Wikipedia

In data processing, the response time perceived by the end user is the interval between

(a) the instant at which an operator at a terminal enters a request for a response from a computer and
(b) the instant at which the first character of the response is received at a terminal.

In a data system, the system response time is the interval between the receipt of the end of transmission of an inquiry message and the beginning of the transmission of a response message to the station originating the inquiry.

Sources: Federal Standard 1037C and MIL-STD-188; http://en.wikipedia.org/wiki/Response_time_%28technology%29 (wikipedia)

From TechTarget

According to the IBM Dictionary of Computing (which cites International Organization for Standardization Information Technology Vocabulary as the source), response time is:

The elapsed time between the end of an inquiry or demand on a computer system and the beginning of a response; for example, the length of the time between an indication of the end of an inquiry and the display of the first character of the response at a user terminal.

There is also the concept of perceived response time, which is the time a user senses as the beginning of input and the end of the response. It is actually possible (though not usual) for perceived response time to be too fast (it can be mildly disconcerting if a system responds almost instantly). However, this is not the usual complaint.

Source: http://searchnetworking.techtarget.com/sDefinition/0,,sid7_gci212896,00.html

Responsiveness

From Jeff Johnson GUI Bloopers

Responsiveness is the software's ability to keep up with users and not make them wait.

>> Research also suggests that improving the responsiveness of software, in addition to increasing user satisfaction, actually improves the productivity of its users (Brady, 1986)

Source: Jeff Johnson (2000). GUI Bloopers: Don'ts and Do's for Software Developers and Web Designers. Morgan Kaufmann Publishers (Chapter 1: First Principles; Principle 7: Design for Responsiveness, p. 49).

From Wikipedia

The responsiveness of an interactive system describes how quickly it responds to user input.

Long delays can be a major cause of user frustration, or let the user believe the system is broken, or that a command or input gesture has been ignored. Responsiveness is therefore considered an important issue for human-computer-interaction (HCI).

Responsiveness is not the same as performance. A system with a good performance can have a poor responsiveness, and vice versa.

Source: http://en.wikipedia.org/wiki/Responsiveness (wikipedia)

Round Trip, Server Round Trip

In a Web as well as a server-based application, there are client-side and server-side tasks running. For example, when a page is refreshed some process will be performed between the client and the server going forth and back. This mechanism is known as server round trip, that is, a server roundtrip takes place whenever you send data to the server and it responds. (Preliminary definition)

S

Speed

More or less the same as performance but reduced to the "speed" aspect.

System Feedback

See Feedback

System Messages

Often, the duration of a delay cannot be predicted. In such a case, the developer can at least send a system message that indicates the currently active operation – preferably in simple user terms, not in technical ones. Such messages show that the developer cares about the user and wants to ask for their understanding.

System Performance

See Performance

System Response

The action that a system performs in response to a user's input (action). For users, the time that they have to wait for the system's response is relevant.

See also Response Time, Waiting Time, Time Ranges, Tolerance for Waiting Times

System Responsiveness

See Responsiveness

T

Think Time

See User Think Time

Time Ranges

Alan Newell (1994) defined a logarithmic time scale of human action, similar to size scales for the universe. His student, Stuart Card, together with coworkers, took a section from that scale and placed it into the context of human-computer interaction. Recently, Alan Cooper et al. (2007) and, most prominently, Jeff Johnson (2000, 2007) republished and further refined the scale from Robertson, Card, and Mackinlay (1989, 1993). Further relevant timing data can be found in Shneiderman & Plaisant (2004). The tables below summarizes the results of our literature search. It defines user-relevant time ranges* and their meaning from different perspectives.

Time
Range		 User Interface:
Acceptable Response Times		 User:
Response/Feedback Does Not Appear Timely		 Comments on Untimely Response
0.1 sec (0.0-0.2) Acknowledge user input
(Direct manipulation: key press, mouse click, open/close menu, highlighting, …)		 Perception of smooth animations and cause-and-effect relationship breaks down Must not happen!
1.0 sec (0.2-2.0) Present result of simple tasks
(navigation, open new window), finish unrequested actions (auto-save)		 Engaged user-system dialog breaks down Should not happen, provide feedback, allow user to abort the process
3 sec (2.0-5.0) Present result of common tasks (simple search, calculation) User has time to think: the system is perceived as slow, the user’s focus starts to wander, and the user may turn to other tasks
10 sec (5.0-15) Present result of complex tasks (complex search, calculation) User looses focus on task and may turn to other tasks
>15 sec Present result of very complex tasks (extensive search, calculation) User gets annoyed; detrimental to productivity and motivation

A variant of the table puts the focus on the human aspects:

Time Range 0.1 (0.05-0.2) sec * 1 (0.2-2.0) sec 3 (2-5) sec 10 (5-15) sec >15 sec
Human Level Perceptual level: Perception, motor actions

Dialog level: Dialog, operation, action
Cognitive level: Thinking, attention, motivation
Hand-eye coordination, cause-and-effect relationships, smooth animation Focus on task Focus of attention starts to wander (3 sec) Focus of attention lost (10 sec) Motivation lost
User inputs commands and data and observes system User is engaged in dialog System is perceived as slow (3 sec) User probably moves over to other tasks (if possible) User gets annoyed
UI Level Direct manipulation: Mouse movement, drag, click, key press, smooth animations, highlighting, open menus, ...

Dialog

Present result of simple task**

 Present result of common task** Present result of complex task** or finish compound task (unit task) Present result of very complex task

*) Because the times are only rough guides, most authors simply list the powers of 10, such as 0.1, 1, and 10 sec; I have extended the numbers into consecutive ranges but please do not take the transition points too seriously. The limit of 2 seconds for the dialog level may already be too high.
**) The terms "simple," "common," and "complex" tasks haven been taken from the literature. In my opinion, they are only of limited use because they do not clearly describe the task types at the respective levels.

Basically, we have three levels or time ranges:

  • The perceptual level at about a tenth of a second
  • The dialog level at about 1 second
  • Beyond that the cognitive level, starting at about 2-3 seconds

The cognitive level can further be subdivided with respect to the effects on the users' attention and motivation.

See also Tolerance for Waiting Times

Tolerance for Waiting Times

The users' tolerance for longer delays/waiting times depends on:

  • Their prior experience (-> expectations)
  • Whether they are given proper feedback about a delay
  • The position of an action within an action chain: At the start, in the middle, at the end (a concluding action can take longer)

Otherwise the 1 second range has to be attained (= no knowledge/context, thus no expectations)

See also Time Ranges

U

User Planning Time

This term is used by Shneiderman & Plaisant (2004), but no definition is given (apart from a diagram). According to the diagram, user planning time consists of a user activity, for example, typing, and the computer response time.

For further details see Response Time.

User Think Time

The user think time is the number of seconds during which users think before initiating the next action. (S&P, 455-456)

Simple stages of action model:

User initiates activity Computer responds  
  <------ Response time ------>   <------User think time ------>  

Figure: Simple stages of action model of system response time and user think time

More realistic model:

User starts     
typing       		User initiates
activity		 Computer
begins response     		Computer
completes response 		 
            
<------ Response time ------>
------>
<----- User planning time ------>
------>
<------
<------User think time ------>

Figure: Simple stages of action model of system response time and user think time

Source: Ben Shneiderman & Cathérine Plaisant (2004). Designing the User Interface (4th Edition). Pearson Addison-Wesley, pp. 454ff.

Further details see Response Time.

Note: Contrary to S&P's diagram, this time can be also used as a resource for planning and calculating ahead and thereby reducing latencies because at least part of it is computer idle time. That is, user think time may not be finished as the computer finishes its response.

W

Wait Animation

Wait animations are animated graphics that appear prominently on the screen and may be much larger and thus more conspicuous than a cursor. Examples of these are the "downloading data" animations in Web browsers and the startup animation of the Apple Macintosh. They also indicate that the system is still working (but see below).

Example of a loading animationExample of a loading animation

Figure: Examples of loading animations

Wait animations are more user-friendly than static "wait" cursors or images, because they clearly indicate that the system is still working and has not crashed or hung (which is why McInerney and Li (2002) call them "working indicators"). However, if an animated GIF is used for a wait animation, the animation may continue even though the application in the background has hung.

Regrettably, we could not find any clear recommendations about when to use wait cursors (static, animated) and when to use wait animations. Some systems use one only type, others both. In the latter case, a wait animation should follow a watch cursor in order to indicate a longer delay. Some systems display both types of animation at the same time, which can be confusing for users. In certain cases – such as during system startup – there may be no cursor available, and an animated graphic is the obvious choice (see right example in the figure above).

Wait Cursor

A mouse cursor that indicates through its shape and/or animation that the user has to wait. It can act as a busy indicator or as a working indicator, or both. There are:

  • Static wait cursors, such as watch cursors and hourglass cursors, are the simplest form of busy indicators.
  • Animated cursors, such as a rotating beach ball or hourglass cursor, are more sophisticated than static cursors and indicate that the system is still working.
Hourglass cursor   Apple wait cursor
Figure: Typical hourglass cursor of earlier GUIs; static or animated   Figure: The Apple Mac OS X spinning ball cursor is an example of an animated wait cursor

Waiting Time

The time that the user has to wait at the computer between a user action, such as a button press or click on a link, and a system response or the ability to interact with the system again. Depending on the duration of the waiting time, the user's expectations, and the type of process, the time causes different psychological reactions.

See also Response Time, Time Ranges, and Tolerance for Waiting Times

Working Feedback

Working feedback (McInerney & Li, 2002) tells the user that the system is still busy and is not hanging, typically by providing some sort of animation.

See also Working Indicator, Feedback

Working Indicator, Working Indication

System feedback using a working indicator is a variant of busy feedback (McInerney & Li, 2002). It tells the user that the system is still busy and is not hanging, typically by providing some sort of animation. In practice, busy and working feedback are often provided by the same indicator.

Working indication can be given through wait cursors and wait animations.

 

 

References

 

top top