Multimodal Warehouse Application
By Samir Raiyani, SAP Research, Palo Alto, and Janaki Mythily Kumar, User Experience, SAP AG – 08/28/2006 • The definitive version was published in INTERACTIONS, see the copyright note below
This is a case study of a multimodal device that was introduced into a real-world
warehouse setting and observations made on its usage by warehouse workers.
Does it fit into their business process? Does it enhance their efficiency?
Do workers enjoy using it? These were the questions that our team, consisting
of researchers, warehouse experts and user experience professionals at SAP
Labs, set out to answer. The goal of our project was to enhance an existing
warehouse management software module with multimodal user inputs and system
outputs, and to study its effects on worker productivity.
Business Case
Driven by the increase in offshoring of manufacturing and the demand for consumer goods, warehouses have grown in size and complexity. At the same time, the volume of orders and the diversity of goods stored in each warehouse have also increased dramatically. These factors have increased the work load of the warehouse worker, especially the picker who is responsible for locating the various line items in each order, and getting them ready for shipping.
Here is an overview of the typical business processes in a warehouse: Goods arrive at a warehouse by truck. They are received at a loading dock, then moved away by forklift operators to the various shelves, storage containers or picking areas inside the warehouse. When a customer order is received, warehouse workers pick the line items of the order and place them into totes. These totes are taken to a staging area via a conveyer belt, then packed and shipped to their final destination. An overview of this process is shown in figure 1.
Figure 1: Overview of Warehouse processes (click image for larger version)
In this project, we focused primarily on the picking process. The traditional picking process starts when the picker gets the order printed on a sheet of paper. The warehouse storage location (Aisle/Bin number) and the quantity ordered are noted next to each line item. Figure 2 shows a typical order sheet:
| Order #25434 | |||
| Aisle | Bin | Item | Quantity |
| 23 | 01-02 | Harry Potter and the Half-Blood Prince (Book 6) | 3 |
| 30 | 03-03 | 20 Questions Handheld Game | 5 |
Figure 2: Typical order sheet handed to a warehouse picker
The picker walks to the bin, picks up the items and places them into a tote. She repeats this for all line items. Once she is done, she takes the tote to the staging area where it is packed and shipped.
In some warehouses, the picking process is entirely manual. In others, the picker carries a mobile device with a barcode scanner. The order information is displayed on the screen of the mobile device. The picker reads the order information on the screen and scans the barcode on each item as she picks it.
Picking is the most labor-intensive operation in a warehouse and also the most error-prone. The size of modern warehouses and the increased volume and complexity of orders increases this inherent inefficiency. Errors increase the cost of order fulfillment dramatically because of the effort involved in processing returns and shipping a replacement. Companies, especially those with slim operating margins, are interested in investing in automation and looking for ways to increase the efficiency of the picking process.
Design
We visited several warehouses and observed and interviewed multiple users before developing a prototype. We also tested noise levels and measured the time taken for everyday tasks.
We identified the following key design objectives:
- Facilitate hands-free and eyes-free access to the system, since a warehouse picker has to work with their hands and lift heavy objects
- Avoid manual data entry and capture pick updates in real time
- Ensure the system is operable in a noisy setting by offering a choice of modalities to the warehouse worker
We built a prototype with the following modalities:
- Screen display (output)
- Touch screen (input)
- Voice (input and output)
- Bar code scanner (input)
The user interface of our prototype consisted of a handheld device (Intermec 700 PDA running an IBM multimodal web browser) that had a visual display and wireless capability. This device had a built-in barcode scanner and a headset for voice input/output. We provided users with a holster to carry the mobile device to free their hands to perform typical warehouse operations.
The screen design was kept simple so as not overload the warehouse worker. Refer to figure 3 for example screen designs:
Figure 3: Screen designs (click image for larger version)
Since users need their hands free, our assumption was that the user would get their task done without holding the device. It was further assumed that the device would remain in a holster and interaction would occur using voice input and output. The user would only need to hold the device to refer to the display for clarification, or to use the barcode scanner. An external handle was attached to the device to make it easier to use as a scanner. Ideally, we would also have liked to make the device holster retractable, so that it would cling to the user's waist when not in use. However, we were not able to incorporate this into the design during the test period.
To increase speed of operation, the voice dialog had to be extremely short and precise. Also, the words spoken by the user had to be easy to recognize by the voice recognition system. For example, the system recognized "Finished" better than "Done", and the dialog was changed accordingly.
Here is an example voice dialog:
System: Please go to Aisle 30, section 11.
System: Level 04, bin 08.
User: Ready.
System: Pick 10 each.
User: Finished.
…
The voice recognition system was chosen carefully after several trials. The system had to be tolerant to noise from machinery in a warehouse such as conveyer belts.
User Testing
We conducted our tests in three phases at Ditan Inc's video-game distribution center in Hayward, California, with warehouse pickers fulfilling real orders for customers. Prior to the introduction of our system, picking was a purely paper-driven process in this warehouse, with workers picking items after reading instructions on the picking sheet associated with each order.
We monitored the pick time for each user and interviewed them at the end of the picking exercise. Over the course of the pilot, we made improvements to the speed of the application, the ergonomics of the device as well as to the training we provided each of the users. We used a speaker-independent voice recognition system for this pilot – this did not require any customizing of the system for a user's speech patterns. We used a different set of users for each of the three phases.
Phase I : Two relatively experienced warehouse pickers, both male. A 5-minute introduction was provided to the users before the tests.
Phase II : Five relatively inexperienced workers, two of them female and three male. A 5-minute introduction was provided to the users before the tests.
Phase III : Three relatively experienced workers, all male. 10-20 minutes of instructions were provided, which included adjustment of headsets and testing the system with the user, before start of the tests.
Findings
- Instructions: Users benefited with the instructions provided in Phase III, and it helped increase their comfort level and satisfaction with the system. The instructions helped ease the users into the multimodal experience.
- Noise: Warehouse noise did not affect the voice recognition system and was not an issue for the pilot.
- Voice recognition: Voice recognition was very good for Phases I and III, but many problems were observed during Phase II. These problems can be attributed to the following:
- Some users spoke English with an accent which made it difficult for the voice recognition system to recognize, and
- There were insufficient instructions provided during Phase II.
- Speed: Our system was too slow for warehouse workers. During Phase I, each pick took 30-40 seconds. The number came down to an average of 24 seconds by Phase III, but still not close to the 10 seconds average recorded by the paper process. Most of the delays were caused by the multimodal browser and the barcode scanner. These issues have since been resolved by the vendors of the multimodal browser technology.
- Ergonomics: We initially experienced several ergonomic issues, primarily related to the use of the barcode scanner. Once we attached a "trigger" handle to the bottom of the user device, the ergonomic problems were mostly resolved.
- User Satisfaction: The user satisfaction improved considerably during Phase III, where the users were observed to have an enjoyable experience working with the system. Users commented on the "coolness" factor of the multimodal application.
- Error rates: Unfortunately, the sample size during our tests was not large enough for us to draw any conclusions about the reduction in error rates. Anecdotally, we observed frequent errors in the paper process, and none in the multimodal process.
- Modality preferences: Once the warehouse worker established a preference for a modality for a particular task, we observed that they used the same modality throughout the test. This preference was established early in their interaction with the system. For example, if the user was able to use voice successfully for confirming the picks, they relied exclusively on this modality and ignored the screen for the confirmation task. This is consistent with research that shows that once a single modality is proven to be superior and sufficient for a task, users tend to use it exclusively, rather than switch between modalities.
The browser limitations were communicated to the partners who provided the multimodal browser technology. They, in turn, made significant performance improvements that enabled us to achieve the 10 sec target for completion of a picking task in subsequent tests. SAP decided to productize this prototype in 2004.
Today, this multimodal warehouse application is being delivered by the SAP partner Topsystem SystemHaus GmbH and is used by customers in several countries. We are starting to get usage reports from live installations. Here is a quote from the Operations manager of HAZET-WERK, Hermann Zerver GmbH & Co. KG:
"Since the implementation, we have increased our output volume for about 50% without additional staff or longer working hours."
In the future, warehouses will be fully automated and robots will be used to pick orders. Until then, our observations suggest that offering multimodal capability to a warehouse picker not only increases their efficiency, but also makes their experience more enjoyable.
Footnotes
- SAP is a software company specializing in providing enterprise solutions, including a warehouse management system used by manufacturers, retailers and logistics service providers worldwide.
- Dan Gilmore, Supply Chain Digest: "Warehousing – Dead or Alive?" URL: http://www.scdigest.com/assets/News/05-05-12.htm
- Vocollect White Paper: "Voice Technology – Cost Per Error and Return on Investment" URL: http://www.vocollect.com/np/documents/CostPerErrorWhitePaper.pdf
- Sharon Oviatt, Antonella DeAngeli & Karen Kuhn: "Integration and Synchronization of Input Modes during Multimodal Human-Computer Interaction" URL: http://www.cse.ogi.edu/CHCC/Publications/Integration_Synchronization_Input_Modes_oviatt.pdf
- Topsystem SystemHaus GmbH. URL: http://www.topsystem.de/homepage/en/logistics.htm
Reference
© ACM, (2006). This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in INTERACTIONS, {Volume 13, Issue 4 July + August 2006} http://doi.acm.org/10.1145/1142169.1142193
