Development and Testing of an Instrument for Summative Assessment of Practical Skill Performance: A Generalizability Theory Approach

Britta Vesterager Stenholt; Irene Sommer; Larsen Karin; Ida Torunn; Inger Åse Reierson; Monika Ravik; Knut Hagvet

doi:10.1891/JNM-D-20-00003

Development and Testing of an Instrument for Summative Assessment of Practical Skill Performance: A Generalizability Theory Approach

Britta Vesterager Stenholt
, Irene Sommer
, Larsen Karin
, Ida Torunn
, Inger Åse Reierson
, Monika Ravik
, Knut Hagvet

Aarhus Univeristy Hospital
University of Oslo
Høgskolen i Telemark
University College of Southeast Norway
University of Southeastern Norway

Research output: Contribution to journal › Journal article › Research › peer-review

Abstract

Background and Purpose: Many newly graduated nurses lack proficiency in practical skill performance. Presently, nursing students’ practical skill is assessed by summative instruments with overarching items. The purpose of this study was to develop a more detailed instrument to use in summative assessment of nursing students’ practical skill performance and to assess its psychometric properties. Methods: A 50-item instrument was developed. Video-recorded performances were rated by experienced clinical supervisors. A multifacet measurement design was developed. Relevant parameters were estimated by generalizability analysis. Results: Findings indicated that error of measurement were mainly caused by raters far more than by items. Conclusions: The present study suggested that summative assessment in realistic settings may not apply one rater only. Two to three/four raters appear necessary to dependably measure most skills

Original language	Danish
Journal	Journal of Nursing Measurement
Volume	29
Issue number	3
Pages (from-to)	E162-E191
Number of pages	29
ISSN	1061-3749
DOIs	https://doi.org/10.1891/JNM-D-20-00003
Publication status	Published - 2021

Access to Document

10.1891/JNM-D-20-00003

Cite this

@article{586ab334d4ca48a0943e9eb8af321043,

title = "Development and Testing of an Instrument for Summative Assessment of Practical Skill Performance: A Generalizability Theory Approach",

abstract = "Background and Purpose: Many newly graduated nurses lack proficiency in practical skill performance. Presently, nursing students{\textquoteright} practical skill is assessed by summative instruments with overarching items. The purpose of this study was to develop a more detailed instrument to use in summative assessment of nursing students{\textquoteright} practical skill performance and to assess its psychometric properties. Methods: A 50-item instrument was developed. Video-recorded performances were rated by experienced clinical supervisors. A multifacet measurement design was developed. Relevant parameters were estimated by generalizability analysis. Results: Findings indicated that error of measurement were mainly caused by raters far more than by items. Conclusions: The present study suggested that summative assessment in realistic settings may not apply one rater only. Two to three/four raters appear necessary to dependably measure most skills",

author = "Stenholt, \{Britta Vesterager\} and Irene Sommer and Larsen Karin and Ida Torunn and Reierson, \{Inger {\AA}se\} and Monika Ravik and Knut Hagvet",

year = "2021",

doi = "10.1891/JNM-D-20-00003",

language = "Dansk",

volume = "29",

pages = "E162--E191",

journal = "Journal of Nursing Measurement",

issn = "1061-3749",

publisher = "Springer Publishing Company",

number = "3",

}

TY - JOUR

T1 - Development and Testing of an Instrument for Summative Assessment of Practical Skill Performance: A Generalizability Theory Approach

AU - Stenholt, Britta Vesterager

AU - Sommer, Irene

AU - Karin, Larsen

AU - Torunn, Ida

AU - Reierson, Inger Åse

AU - Ravik, Monika

AU - Hagvet, Knut

PY - 2021

Y1 - 2021

N2 - Background and Purpose: Many newly graduated nurses lack proficiency in practical skill performance. Presently, nursing students’ practical skill is assessed by summative instruments with overarching items. The purpose of this study was to develop a more detailed instrument to use in summative assessment of nursing students’ practical skill performance and to assess its psychometric properties. Methods: A 50-item instrument was developed. Video-recorded performances were rated by experienced clinical supervisors. A multifacet measurement design was developed. Relevant parameters were estimated by generalizability analysis. Results: Findings indicated that error of measurement were mainly caused by raters far more than by items. Conclusions: The present study suggested that summative assessment in realistic settings may not apply one rater only. Two to three/four raters appear necessary to dependably measure most skills

AB - Background and Purpose: Many newly graduated nurses lack proficiency in practical skill performance. Presently, nursing students’ practical skill is assessed by summative instruments with overarching items. The purpose of this study was to develop a more detailed instrument to use in summative assessment of nursing students’ practical skill performance and to assess its psychometric properties. Methods: A 50-item instrument was developed. Video-recorded performances were rated by experienced clinical supervisors. A multifacet measurement design was developed. Relevant parameters were estimated by generalizability analysis. Results: Findings indicated that error of measurement were mainly caused by raters far more than by items. Conclusions: The present study suggested that summative assessment in realistic settings may not apply one rater only. Two to three/four raters appear necessary to dependably measure most skills

U2 - 10.1891/JNM-D-20-00003

DO - 10.1891/JNM-D-20-00003

M3 - Tidsskriftsartikel

SN - 1061-3749

VL - 29

SP - E162-E191

JO - Journal of Nursing Measurement

JF - Journal of Nursing Measurement

IS - 3

ER -