The challenges and benefits of analyzing feedback comments in surveys: Lessons from a cross-national online survey of small-scale cannabis growers

It is common practice in survey questionnaires to include a general open and non-directive feedback question at the
end, but the analysis of this type of data is rarely discussed in the methodological literature. While these open-ended
comments can be useful, most researchers fail to report on this issue. The aim of this article is to illustrate and reflect
upon the benefits and challenges of analyzing responses to open-ended feedback questions. The article describes the
experiences of coding and analyzing data generated through a feedback question at the end of an international online
survey with small-scale cannabis cultivators carried out by the Global Cannabis Cultivation Research Consortium.
After describing the design and dataset of the web survey, the analytical approach and coding frame are presented.
The analytical strategies chosen in this study illustrate the diversity and complexity of feedback comments which pose
methodological challenges to researchers wishing to use them for data analyses. In this article, three types of feedback
comments (political/policy comments, general comments of positive and negative appreciation, and methodological
comments) are used to illustrate the difficulties and advantages of analyzing this type of data. The advantages of
analyzing feedback comments are well known, but they seem to be rarely exploited. General feedback questions at
the end of surveys are typically non-directive. If researchers want to use these data for research and analyses, they
need a clear strategy. They ought to give enough thought to why they are including this type of question, and develop
an analytical strategy at the design stage of the study