Group: testing and evaluating user interfaces

topics > computer science > Group: user interface

collecting statistics on a UserInterface
ease of use
ease of learning
examples of usability studies
keystroke-level model for user interface evaluation
interactive response time
problems with usability testing
testing user interfaces by transcripts
usability errors
information retrieval

children vs. adults
comparing paper to electronic access to information
experimental results on programming
experimental results on structured programming
learning a programming language
man-machine symbiosis
minimal manuals and guided exploration
novice users and the UserInterface
programmer productivity
selecting with a mouse
user-centered design
user interface design

Subtopic: man-machine symbiosis up

Quote: evaluate an interactive computer system in terms of the total performance of the combined man-machine system [»cardSK7_1980]

Subtopic: human factors up

Quote: the central, quantitative human factors are learning time, performance speed, error rate, retention, and satisfaction
Quote: project managers, designers, and marketing must clearly state the primary human-factor goals for a project

Subtopic: user-interface specialist up

Quote: user-interface specialists found twice as many problems as programmers with guidelines and 12x more problems than user-interface testers [»tognB_1992]

Subtopic: ethnography, video up

Quote: ethnography is the study of users in their own work environment; video helps capture tacit skills, tricks of the trade, and ways of communicating [»jiroM5_2006]
Quote: British airspace controllers use flight progress strips arranged in racks; records all instructions to the pilot [»hughJA11_1992]

Subtopic: quantitative vs. qualitative up

Quote: small differences in a user interface can cause large differences in usability; e.g., command names. A quantitative approach is inappropriate [»carrJM10_1986]

Subtopic: ideal vs. actual up

Quote: predicting behavior tokens in ideally skilled performance ignores errors, boundary conditions, and why people act [»carrJM10_1986]
Quote: real interface design requires real users using real applications to achieve real goals; laboratory studies of simple and repetitive tasks are often irrelevant [»carrJM10_1986]

Subtopic: user testing techniques up

Quote: discount usability engineering--when two or users have a problem that can be explained, then best to fix the problem at once [»nielJ3_1990]
Quote: analyze failures by users when interacting with a system [»furnGW11_1987]
Quote: study user interfaces by an invisible operator that corrects user mistakes [»goodMD10_1984]
Quote: testing blind users with screenreaders; listened incredibly fast; scan with their ears [»theoMF11_2003]

Subtopic: user-centered design up

Topic: user-centered design
Quote: need a process to attain ease of use; e.g., establishing testable behavior specifications and testing them [»goulJD3_1985]
Quote: need to understand users through direct contact, interviews, observation, training exercises [»goulJD3_1985]
Quote: the primary test of a user interface is its success with users; ask if they understand and if they can perform tasks easily and efficiently [»apple_1987]
Quote: critical parts of a system should be tested on users [»smitDC4_1982]
Quote: for interactive software, should pay attention to the needs and preferences of users; simple changes can improve the product [»ledgH10_1980]
Quote: DEC uses contextual or action research--action under uncertainty is the norm, can't postpone judgments, real situations [»whitJ5_1989]

Subtopic: personal diferences up

Quote: user interface race revealed dramatic differences in tools and skills, invoked a lively discussion [»shneB_1990]

Subtopic: model-based design up

Quote: Card, Moran, and Newell summarized existing psychological knowledge through a simple model of the human processor. They use the model to numerically analyzing human-computer interaction [»cardSK_1983]
Quote: use an approximate system level description of the mind to help predicate behavior [»cardSK_1983]
Quote: the Model Human Processor consists of a perceptual system, cognitive system, and motor system; characterize each component by processor cycle time, memory capacity, decay rate, and code type [»cardSK_1983]
Quote: the cycle time for each processor in the Model Human Processor is on the order of a tenth of a second
Quote: analyze complex behavior with GOMS: goals to be achieved, operators to do things, methods as procedures, and selection rules for choosing among competing methods [»cardSK_1983]
Quote: a GOMS method is a learned procedure, i.e., a conditional sequence of goals and operators

Subtopic: low-level usability up

Quote: derive Fitts' Law by assuming that the relative accuracy of movement is constant; the measured time for acquiring a target is within the computed range [»cardSK_1983]
Quote: use the steering law for trajectory-based tasks; like Fitts' law [»accoJ5_1999]
Quote: for steering tasks, tablet and mouse are faster than trackpoint which is faster than trackball or touchpad [»accoJ5_1999]
Quote: use point-completion deadlines to separate pointing time from search time in usability studies [»hornAJ9_2001]
Quote: search for upper-left targets faster than lower-right targets; 2x to 4x; roughly linear for three column layout; lower middle targets slower [»hornAJ9_2001]
Quote: 1/5 second search time for target-only layouts; independent of position
Quote: comparison of constrained pointing tasks and goal-crossing tasks for speed and accuracy [»accoJ4_2002]

Subtopic: paper prototype up

Quote: design and test user interfaces with simple paper prototypes that are moved in response to user's actions; allows early, repeated testing of ideas [»rettM4_1994]
Quote: paper-based prototypes were just as effective in uncovering usability problems as high-fidelity prototypes or the product itself [»virzRA4_1996]

Subtopic: automated usability testing up

Quote: automated usability testing by uploading statistics, suggestion boxes, and dialogue boxes that test user comprehension at error messages [»duttG9_1993]
Quote: automated usability testing is less expensive, more consistent, greater coverage [»ivorMY12_2001]

Subtopic: video up

Quote: dealers can make deals every five seconds over the phone; propose prices and quantities using context, abbreviations, and jargon; through video analysis [»jiroM5_2006]

Subtopic: log analysis up

Quote: use logs and analysis tools to study the use of web applications [»manbU8_2000]
Quote: usability information is often easy to record but hard to interpret (keystrokes), or useful but hard to automatically label (task completion); makes automation difficult [»ivorMY12_2001]

Subtopic: query-based testing up

Quote: open-ended testing of Hypertext; measured correct responses to queries [»whalT5_1989]

Subtopic: usability walkthrough up

Quote: usability walkthroughs with users, developers, and human factors experts; everyone writes down reactions then users speak [»biasR9_1991]
Quote: with usability walkthroughs get actual actions by users, problems uncovered by users, and opinions by usability experts; synergy [»biasR9_1991]
Quote: industrial data processing professionals could identify 44% of the serious faults in a human interface [»moliR3_1990]
Quote: use multiple designers to evaluate a user interface; a single designer will only detect a small portion of the usability problems [»nielJ5_1990]

Subtopic: layout testing up

Quote: small differences in layout appropriateness do not affect performance
Quote: using subtitles to group related items clearly improves search time without changing error rates [»hornAJ9_2001]
Quote: a well-organized screen layout allows many items without hurting search time [»hornAJ9_2001]

Subtopic: presentation vs. domain testing up

Quote: while presentation code is difficult to test, can test the domain code through direct calls [»fowlM3_2001]

Subtopic: test by disabling features up

Quote: test interface by disallowing certain options, e.g., link options in NetBook [»graySH2_1989]
Quote: use sequential disclosure of a function's components when testing an interface; identifies the important feature [»knoxST5_1989]

Subtopic: evaluate by time up

Quote: user interface race ;e.g., type in quote, place in alphabetic order with repetitions on a line, repeat
Quote: using pop-up menus instead of dialogue-boxes saved 11 seconds for certain telephone queries; estimated savings of $1.6 million in two years [»nielJ11_1993]

Subtopic: evaluate by task up

Quote: establish human-factors goals in terms of a benchmark set of tasks for each kind of user [»shneB_1998]

Subtopic: simulated system up

Quote: SmartHelp simulated intelligent help with a person, but none of the participants voiced suspicions [»carrJM9_1988]

Subtopic: pretesting up

Quote: pretesting is a powerful aid in designing an experiment that uses subjects [»ledgH_1981]
Quote: in user testing, the experimenter should teach subjects about the editor; otherwise takes too long to learn correctly [»ledgH_1981]

Subtopic: recruiting subjects up

Quote: for user testing, found that on-the-spot recruiting was better than schedules; otherwise many subjects never report [»ledgH_1981]

Subtopic: problems with usability testing up

Quote: less than 1% overlap in findings between independent usability teams; wide variation even within same evaluation technique [»ivorMY12_2001]
Quote: 75% of HTML guidelines appear in only one style guide; low validity

Group: testing and evaluating user interfaces up

Topic: collecting statistics on a UserInterface (12 items)
Topic: ease of use (47 items)
Topic: ease of learning (38 items)
Topic: examples of usability studies (31 items)
Topic: keystroke-level model for user interface evaluation (6 items)
Topic: interactive response time (32 items)
Topic: problems with usability testing (16 items)
Topic: testing user interfaces by transcripts (13 items)
Topic: usability errors
(6 items)

Related Topics up

Group: information retrieval   (25 topics, 674 quotes)
Group: psychology   (9 topics, 307 quotes)
Group: testing   (18 topics, 557 quotes)

Topic: children vs. adults (33 items)
Topic: comparing paper to electronic access to information (35 items)
Topic: experimental results on programming (75 items)
Topic: experimental results on structured programming (11 items)
Topic: learning a programming language (15 items)
Topic: man-machine symbiosis (46 items)
Topic: minimal manuals and guided exploration (44 items)
Topic: novice users and the UserInterface (25 items)
Topic: programmer productivity (57 items)
Topic: prototyping (18 items)
Topic: selecting with a mouse (44 items)
Topic: statistics (12 items)
Topic: user-centered design (65 items)
Topic: user interface design
(36 items)

Updated barberCB 3/06
Copyright © 2002-2008 by C. Bradford Barber. All rights reserved.
Thesa is a trademark of C. Bradford Barber.