To Count or Not to Count, that is the Question? Evaluative practices in Higher Education

Due to the emergence and prevalence of ‘quantification cultures’ and the ensuing ‘audit society’ (Power, 1999), higher education institutions are increasingly faced with demands of performance measurement and rituals of quantification. University ranking is but one example of the ‘control by numbers’ (Lynch, 2015) that has institutionalized the educational landscape and actively informs political institutions such as OECD and EU in educational policies and strategies. Citation indexes and numeric ‘impact factors’ working as proxies for the quality of academic work is yet other practices that is highly political (Lawrence, 2007) and might lead astray (Todd and Ladle, 2008), or blur professional and administrative commitments (Christin, 2018). Likewise, teachers and researchers at universities of applied sciences in Denmark are required by law to account for working hours on daily basis. In this regard, numbers and benchmarks create transparency and a verifiable argument (March 1994; Power 2004), by which managers and organizations are able to compare and evaluate decisions and thereby manage future allocations of resources. By ranking higher education institutions or making academics ‘work’ visible either by counting ‘impact’ or ‘hours’, the institution and its workers become measurable and hence comparable and in the end ‘objects of evaluation’. As research on sociology of valuation points out, dominating calculation regimes prosper under such conditions. Espeland and Stacy (2015) argue for how such regimes categorize and label in order to make visible and thus comparable, by which control of behavior is made possible. Making certain aspects of work visible while other aspects are left invisible is however highly problematic (Kornberger et al. 2015). A too narrow focus on only what we see on behalf of what we don’t see’ (Espeland & Stacy, 2015) may devalue otherwise valuable work taking place ‘in the shadows’ (Star and Strauss, 1999). This aspect of higher education evaluation is often overlooked in the public debate as well as research (Lynch, 2015). In the full paper, we study two illustrative cases of calculative devices used by a Danish higher education institution. Each device enables the existence of evaluative infrastructures (Power, 2015) that guide, instruct and control the behavior of employees. We have selected the two cases as they represent conflicting calculation devices. While BFI is a system for and indicator of research quality, SmartKLIK is an IT system for time recording that accounts for hours spend on work activities and projects, which has to be balanced by the end of each term. The former makes research output visible, while the latter makes the research input visible. However, they are rarely aligned in terms of what ‘counts’ and what ‘not counts’ or what is clear and visible as a number and what is in the ‘shadows’ of the number. In the full paper, we show how numbers have become key in modern rituals of control both facing higher education institutions as well as driving them. We conclude by discussing the future role of evaluative practices in higher education institutions, and why we need to ask the question of whether to count or not to count? References: Christin, A. (2018). Counting clicks: Quantification and variation in web journalism in the United States and France, American Journal of Sociology, 123(5): 1382-1415 Espeland, W. N. & Stacy, E. L. (2015). Noticing Numbers: How quantification changes what we see and what we don’t, In: Kornberger, M., Justesen, L., Madsen, A. & Mouritsen, J. (2015). Making Things Valuable, Oxford Univsersity Press Lynch, K. (2015). Control by Numbers: New managerialism and rankings in Higher Education, Critical Studies in Education, 56(2): 190-207 March, J. (1994). A primer on decision making: how decisions happen, New York: The Free Press Power, M. (1999). The Audit society: Rituals of verification, Oxford: Oxford University Press Power, M. (2004). Counting, control and calculation: Reflections on measuring and management, Human Relations, Vol. 57 (6): 765-783 Power, M. (2015). How accounting begins: Object formation and the accretion of infrastructure. Accounting, Organizations and Society, 47, 43-55. Star, S.L. and A. Strauss (1999), ‘Layers of Silence, Arenas of Voice: The Ecology of Visible and Invisible Work’, Computer Supported Cooperative Work 8, 9–30. Todd, P., & Ladle, R. (2008). Hidden dangers of a citation culture. Ethics in Science and Environmental Politics, 8, 13–16.