« Week 4 Progress | Main | Think-aloud books available online »
May 1, 2005
Reconceptualizing think-aloud methodology: Refining the encoding and categorizing techniques via contextualized perspectives (Yang, 2003)
Yang, S. C. (2003). Reconceptualizing think-aloud methodology: Refining the encoding and categorizing techniques via contextualized perspectives. Computers in Human Behavior, 19, 95-115.
This paper summarizes several aspects of Ericsson & Simon's protocol analysis (a procedure to identify the mental processes that take place during problem-solving tasks). In particular, the author describes features of data analysis, and suggests straying from some of Ericsson and Simon's prescriptions.
Points from Ericsson & Simon (1984/1993):
1. There are three levels of verbalization in verbal protocols, and it is important that verbalization be kept at levels 1 and 2. Otherwise, the protocol analysis is subject to error.
2. Data analysis consists of transcribing, segmenting, and encoding verbal protocols.
3. Transcribing may be extended to include other data such as video recordings. Difficulties may arise depending on the clarity and fluency of subjects, technical quality of recordings, and ambient noise/distractions.
4. Once transcribed, data is broken down into discrete segments, which are large enough to contain all the information needed for making an encoding decision.
5. Encoding should be context-free, according to Ericsson & Simon (this is the point the author focuses on). In order to prevent bias, the researcher should select segments of data randomly, and categorize them one at a time, without regard for what came before or after. Thus, the researcher is less likely to hypothesize about what the subject was thinking, and just encodes (categorizes) what the subject actually said.
The author then describes a series of studies that examined learners' cognitive processing while using a Greek culture database. The researcher deliberately did not follow all of Ericsson and Simon's procedures for protocol analysis. In particular, examples are given of mental processes that might not have been revealed had the segments been encoded in random order, without paying attention to context.
The author feels that analysts should engage in "context-appreciative" (vs. "context-free") encoding, especially since it is becoming more and more difficult to adhere to Ericsson and Simon's prescriptions for developing discrete and mutually exclusive categories (the cognitive processes used by learners may be interrelated and complex). However, there is still the possibility of bias and problems of interpretation, and researchers should provide full accounts of their methods for collecting and encoding data, as well as their theoretical framework.
In evaluating this paper's recommendations, we should consider that the tasks used in the studies presented are complex (using a database to complete thematic assignments related to Classical Greek Studies, e.g., attitudes toward competition and its manifestations in Classical Greece), and undoubtedly produced especially rich verbal protocols. If our data will not be as complex, we may not need to worry about deviating from Ericsson and Simon's recommendations; however, the issue of context is well worth considering.
Posted by Erin at May 1, 2005 10:03 PM
Comments
Post a comment
Thanks for signing in, . Now you can comment. (sign out)
(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)