During my recent tutorial, an important point was raised by my tutor that added another layer of reflection to my project, about the usage of survey as a feedback tool and the potential perception of the questions. She noted that the way these questions are framed could imply an expected positive change, possibly influencing respondents to report a shift even if none occurred.
I realized on the spot that even if unintended, the design of a survey could lead respondents towards a certain type of answer, potentially skewing the data collected. Acknowledging this, I recognized the need for more neutral and standalone questions that measure the effectiveness of an intervention without implying a desired direction of impact.
Questioning the Questions
In my attempt to measure the immediate impact of the resources, I asked, “Before viewing this page, how confident where you in your ability to support a socially anxious gamer?” followed by, “After reading the content, how has your confidence changed?” This design was meant to capture the change in confidence directly influenced by the intervention.
However, it was highlighted how such a design could be perceived as leading to a favorable outcome. I explained that if my intention were to manipulate outcomes favorably, I wouldn’t have provided options for negative feedback, I would frame it as “After reading the content, how has your confidence increased?” Nonetheless, the sequential nature of the questions might indeed make respondents think comparatively to their answers. I acknowledge this, and it made me recognize the need for more neutral and standalone questions, or maybe even if I just randomize the sequence of the questions.
Survey as Feedback Tool vs Intervention
An interesting point was raised during my tutorial about surveys. My tutor mentioned that I could have framed my survey as not just a tool for gathering feedback but potentially as interventions themselves. It is explained to me that the very act of asking certain questions in certain ways could influence respondents’ thoughts and feelings, empowering them, and thereby serving as a form of intervention.
However, this raises another question about the objectivity and reliability of the survey as a research tool. If a survey is used as an intervention, it has compromised its function as a neutral tool for data collection. As I discussed with my tutor, I questioned that if the survey itself is designed in a way to prompt a shift in perception, can it still be considered a reliable source of feedback?
The inherent purpose of my survey was not to serve as an intervention, and was not purposefully designed to skew towards a positive outcome. The survey was used as a tool to gather insights that would inform the development of my project, not to serve as an intervention itself.
Final Thoughts
This tutorial was a reminder of the complex dynamics that can underly a seemingly straightforward research tool such as a survey. Keep in mind: asking the right questions, the right way, in the right sequence, without implying the answer.