Quality of measurement in instruments measuring styles? A review of five decades of articles and overview literature

Brief abstract Over the course of five decades, many theories/models and instruments of cognitive, learning, teaching, thinking and intellectual style have been developed, and have found widespread use in educational and workplace contexts. Whether the purpose for measuring styles is research, or serves as a basis for group-wise or individual intervention, counselling or assessment, an issue of great concern is that of the psychometric quality of the instruments used. The aim of the present study was to review the ways by which the quality of these measurements is investigated within the field of learning styles. A systematic search for literature from the last five whole decades on quality of measurement within the field of styles identified ten review texts and 101 articles. Eighty-eight of the articles investigated issued related to reliability and/or validity, while the remaining texts were reviews, discussion articles, or validation studies of something other than measurement of learning styles. The single most investigated theoretical construct was Kolb´s theory of experiential learning, and the most thoroughly studied instrument was the Psychological Inventory of Criminal Thinking Styles. Keywords: Quality of measurement, measurement of styles, review Extended abstract Over the course of five decades (1960s to 2000s) many theories/models and instruments of cognitive, learning, teaching, thinking and intellectual styles have been developed. How many is not entirely clear; Coffield et al (2004a, 2004b) proposed theories/models, and in connection with Nielsens (2012) review the count of instruments reached 208. Styles are usually measured for one of two purposes: 1) as a basis for group-wise or individual intervention, counselling or assessment. 2) In connection with research into the styles of different groups of persons, how styles are preceded by, associated with or can predict other psychological, behavioural and sociological phenomenon. Whether the purpose for measuring styles is research or intervention, an issue of great concern is that of the quality of the measurements used. Classical test theory has long, and is still, the dominant school of thought within the field of psychology, while the field of education to some degree has embraced modern test theory. This difference is interesting, as the styles field is for most purposes situated in the cross-discipline of educational psychology with its focus on preferences for learning, teaching, thinking, etc. The review is aimed at examining how quality of measurement is investigated in the styles field. The review examines a pool of ten review/overview texts and 101 articles resulting from a systematic search for literature on the issue of quality of measurement within the field of styles during the last five whole decades; the 1960s to the 2000s. The search criteria included the five major style terms (cognitive, thinking, teaching, learning and intellectual style) in combination with each of ten search terms frequently and interchangeably used by authors of articles on the quality of measurement within a range of disciplines. The discovered texts make up only 19% of the overview texts and 3% of the articles identified in the broader search by Nielsen (2012). In the analysis, it was decided a-priori to pay particular attention to: types of validity and reliability, issues of bias and fairness, comparability of language or culturally defined test-versions of instruments, and overall adherence to classical or modern test theoretical conceptions of psychometric quality. Also, it was decided a-priori, to address the aforementioned issues in general, in a decade-wise time perspective, within the five major style concepts, and where possible within single theories. Using an iterative coding strategy, each article was coded according to general criteria (publication decade, major style concept, specific theory/theories, and instrument(s) employed), and a number of specific criteria (different types of reliability and validity, ect,) evolving for each article. Some results were: of the 101 articles, 88 were empirical articles investigating reliability and/ or validity in different ways. The remaining articles were reviews, discussion articles, or validation studies of something other than style measures. Sixty-three articles included one to five different types of reliability estimates. The most frequent (53) reliability estimate was internal consistency, and this had the highest prevalence in articles on either learning or thinking styles (approx. 2/3). The second most frequent (28) was test-retest reliability with an average length between points of measurement of 2.6 months, and this had the highest prevalence in ResearchGate Logo Discover the world's research 17+ million members 135+ million publications 700k+ research projects Join for free Public Full-text 1 Content uploaded by Tine Nielsen Author content Content may be subject to copyright. Nielsen,T.(2014).Qualityofmeasurementininstrumentsmeasuringstyles?Areviewoffivedecadesofarticlesandoverviewliterature.AbstractfromEarliSIG4&SIG17Conference2014,Leuven,Belgium. Quality of measurement in instruments measuring styles? A review of five decades of articles and overview literature Brief abstract Over the course of five decades, many theories/models and instruments of cognitive, learning, teaching, thinking and intellectual style have been developed, and have found widespread use in educational and workplace contexts. Whether the purpose for measuring styles is research, or serves as a basis for group-wise or individual intervention, counselling or assessment, an issue of great concern is that of the psychometric quality of the instruments used. The aim of the present study was to review the ways by which the quality of these measurements is investigated within the field of learning styles. A systematic search for literature from the last five whole decades on quality of measurement within the field of styles identified ten review texts and 101 articles. Eighty-eight of the articles investigated issued related to reliability and/or validity, while the remaining texts were reviews, discussion articles, or validation studies of something other than measurement of learning styles. The single most investigated theoretical construct was Kolb´s theory of experiential learning, and the most thoroughly studied instrument was the Psychological Inventory of Criminal Thinking Styles. Keywords: Quality of measurement, measurement of styles, review ExtendedabstractOver the course of five decades (1960s to 2000s) many theories/models and instruments of cognitive, learning, teaching, thinking and intellectual styles have been developed. How many is not entirely clear; Coffield et al (2004a, 2004b) proposed theories/models, and in connection with Nielsens (2012) review the count of instruments reached 208. Styles are usually measured for one of two purposes: 1) as a basis for group-wise or individual intervention, counselling or assessment. 2) In connection with research into the styles of different groups of persons, how styles are preceded by, associated with or can predict other psychological, behavioural and sociological phenomenon. Whether the purpose for measuring styles is research or intervention, an issue of great concern is that of the quality of the measurements used. Classical test theory has long, and is still, the dominant school of thought within the field of psychology, while the field of education to some degree has embraced modern test theory. This difference is interesting, as the styles field is for most purposes situated in the cross-discipline of educational psychology with its focus on preferences for learning, teaching, thinking, etc. The review is aimed at examining how quality of measurement is investigated in the styles field. The review examines a pool of ten review/overview texts and 101 articles resulting from a systematic search for literature on the issue of quality of measurement within the field of styles during the last five whole decades; the 1960s to the 2000s. The search criteria included the five major style terms (cognitive, thinking, teaching, learning and intellectual style) in combination with each of ten search terms frequently and interchangeably used by authors of articles on the quality of measurement within a range of disciplines. The discovered texts make up only 19% of the overview texts and 3% of the articles identified in the broader search by Nielsen (2012). In the analysis, it was decided a-priori to pay particular attention to: types of validity and reliability, issues of bias and fairness, comparability of language or culturally defined test-versions of instruments, and overall adherence to classical or modern test theoretical conceptions of psychometric quality. Also, it was decided a-priori, to address the aforementioned issues in general, in a decade-wise time perspective, within the five major style concepts, and where possible within single theories. Using an iterative coding strategy, each article was coded according to general criteria (publication decade, major style concept, specific theory/theories, and instrument(s) employed), and a number of specific criteria (different types of reliability and validity, ect,) evolving for each article. Some results were: of the 101 articles, 88 were empirical articles investigating reliability and/ or validity in different ways. The remaining articles were reviews, discussion articles, or validation studies of something other than style measures. Sixty-three articles included one to five different types of reliability estimates. The most frequent (53) reliability estimate was internal consistency, and this had the highest prevalence in articles on either learning or thinking styles (approx. 2/3). The second most frequent (28) was test-retest reliability with an average length between points of measurement of 2.6 months, and this had the highest prevalence in Citations (0) References (6) Handbook of Intellectual Styles: preferences in Cognition, Learning and Thinking Jan 201221-46 T Nielsen Nielsen, T. (2012). A Historical review of the styles literature. In Robert Sternberg, Li-fang Zhang & Stephen Rayner (Eds). Handbook of Intellectual Styles: preferences in Cognition, Learning and Thinking. Springer, pp 21-46. The criminal lifestyle. Patterns of serious criminal conduct Jan 1990 G D Walters Walters, G.D. (1990). The criminal lifestyle. Patterns of serious criminal conduct. newbury Park, CA, Sage. Experiential Learning: Experience As The Source Of Learning And Development Book Full-text available Jan 1984 David A. Kolb View The Psychological Inventory of Criminal Thinking StylesPart I: Reliability and Preliminary Validity Article Sep 1995CRIM JUSTICE BEHAV GLENN D. WALTERS View Show abstract Learning Styles And Pedagogy In Post-16 Learning: A Systematic And Critical Review Article Jan 2004 F. J. CoffieldD. V. Moseley Elaine HallK. Ecclestone View Should We Be Using Learning Styles? What Research Has to Say to Practice Article Jan 2004 F. J. CoffieldD. V. Moseley Elaine HallK. Ecclestone View Recommendations Discover more Project PUF: Psychosocial Development in the first year of life Bjørn E Holstein Svend Kreiner Tine Nielsen[...] Janni Ammitzbøll View project Project Learning styles of students and teachers Tine NielsenA.E. HvasA. Kjaergaard View project Project Measurement of Self-efficacy and Motivation in higher Education Tine Nielsen Inmaculada Martinez-Garcia Enrique Alastor Construct and other validity of instruments for the measurement of motivation and motivation-related constructs in higher education View project Project Student well-being and the lack thereof Tine Nielsen Svend Kreiner Jesper Dammeyer[...] Inmaculada Martinez-Garcia Increasing focus on student stress, loneliness and other mental health issues might suggest that such student well-being is declining and student mental health issues are on the rise. The project w ... [more] View project Article Experiential Learning Theory as a Guide for Effective Teaching September 1987 · Counselor Education and Supervision Patricia H. MurrellCharles S. Claxton Discusses D. Kolb's (1984) experiential learning theory, which involves a framework that can be helpful in designing courses that meet the needs of diverse learners. Kolb describes learning as made up of 2 dimensions: prehending or grasping information, and transforming or processing that information. Kolb suggests that learning occurs as the individual moves through a cycle of concrete ... [Show full abstract]Read more Article Diversity of didactic approaches in teaching Geography Lea Nemec This paper discusses different didactic approaches in teaching geography as viewed through the prism of Kolb's experiential learning theory and Kolb's learning styles inventory. The survey, based on Kolb's model of a four-stage cycle (concrete, experience, reflective observation, abstract conceptualization and active experimentation) and a four-type definition of learning styles (diverging, ... [Show full abstract]Read more Article Flipped Classroom Teaching and Learning Pedagogy in the Program Planning, Implementation, and Evalua... April 2019 · Pedagogy in Health Promotion Bojana Berić-Stojšić Naiya PatelJanice BlakeDaryl Johnson This article provides support to flipped classroom pedagogy in the preparation of graduate-level public health practitioners. We describe the participatory, interactive, collaborative, and liberating process of teaching and learning in the Program Planning, Implementation, and Evaluation course, and we provide the results of a cross-sectional study into students’ perceptions of this process. Our ... [Show full abstract]Read more Article Full-text available The learning style preferences of hospitality and tourism students: Observations from an internation... December 2006 · International Journal of Hospitality Management Conrad Lashley Paul Barron This paper reports on a study of the learning style preferences of new entrants onto hospitality and tourism programmes in Australia and the United Kingdom. It suggests that a majority of students on these programmes in both countries have strong learning style preference that present some challenges to educators and the planning of learning experiences in higher education.