During my recent tutorial, an important point was raised by my tutor that added another layer of reflection to my project, about the usage of survey as a feedback tool and the potential perception of the questions. She noted that the way these questions are framed could imply an expected positive change, possibly influencing respondents to report a shift even if none occurred.
I realized on the spot that even if unintended, the design of a survey could lead respondents towards a certain type of answer, potentially skewing the data collected. Acknowledging this, I recognized the need for more neutral and standalone questions that measure the effectiveness of an intervention without implying a desired direction of impact.
Questioning the Questions
In my attempt to measure the immediate impact of the resources, I asked, “Before viewing this page, how confident where you in your ability to support a socially anxious gamer?” followed by, “After reading the content, how has your confidence changed?” This design was meant to capture the change in confidence directly influenced by the intervention.
However, it was highlighted how such a design could be perceived as leading to a favorable outcome. I explained that if my intention were to manipulate outcomes favorably, I wouldn’t have provided options for negative feedback, I would frame it as “After reading the content, how has your confidence increased?” Nonetheless, the sequential nature of the questions might indeed make respondents think comparatively to their answers. I acknowledge this, and it made me recognize the need for more neutral and standalone questions, or maybe even if I just randomize the sequence of the questions.
Survey as Feedback Tool vs Intervention
An interesting point was raised during my tutorial about surveys. My tutor mentioned that I could have framed my survey as not just a tool for gathering feedback but potentially as interventions themselves. It is explained to me that the very act of asking certain questions in certain ways could influence respondents’ thoughts and feelings, empowering them, and thereby serving as a form of intervention.
However, this raises another question about the objectivity and reliability of the survey as a research tool. If a survey is used as an intervention, it has compromised its function as a neutral tool for data collection. As I discussed with my tutor, I questioned that if the survey itself is designed in a way to prompt a shift in perception, can it still be considered a reliable source of feedback?
The inherent purpose of my survey was not to serve as an intervention, and was not purposefully designed to skew towards a positive outcome. The survey was used as a tool to gather insights that would inform the development of my project, not to serve as an intervention itself.
Final Thoughts
This tutorial was a reminder of the complex dynamics that can underly a seemingly straightforward research tool such as a survey. Keep in mind: asking the right questions, the right way, in the right sequence, without implying the answer.
To confirm whether I’m reaching the intended audience, I initiated the survey by asking if they knew someone who displayed confidence in online settings but appeared shy or reserved offline. This aimed to confirm the prevalence of the issue among my target audience (gaming circles) and to gather firsthand descriptions of these contrasts in social confidence.
Contrasts in Online and Offline Social Behaviors
Most of the respondents (28, 71.8%) knows someone who is confident in online interactions but shy or reserved in offline settings. The open-ended responses provided some insights into these cases.
-Behavioral Disparity: Several responses highlight a significant disparity between how individuals engage in online versus offline environments. Descriptions like “very talkative and social online, but very quiet in person” and “can chat for hours on discord or while playing games but super quiet if we meet up in person” shows that the online platforms provides a more comfortable experiences that is not there in face-to-face interactions. -Role and Leadership Transition: Some respondents noted that individuals who take active leadership roles in online settings (e.g., leading a group) do no exhibit the same confidence in social situations offline, such as workplace settings. -Family Dynamics: It’s noted that the disparity also extends into family interactions, there are individuals who are more withdrawn at home compared to how they interact with people in gaming environments.
Pre-content Confidence
Before viewing the content, the majority of the respondents (29, 74.4%) reported low confidence in their ability to support a socially anxious gamer improve their offline social interactions.
This distribution verifies a need for support and guidance, suggesting that the audience I was able to get in touch with may benefit greatly from educational content designed to improve their confidence in supporting socially anxious gamers.
Post-content Confidence
After viewing the content, respondents showed a diverse range of confidence changes. A notable number of respondents reported no change in confidence (15, 38.5%), with an equal distribution of respondents who felt more confident (13, 33.3%) and less confident (11, 28.2%) after engaging with the content.
This suggests that there’s some varied impact of the content on the respondents.
-Increased Confidence: 13 respondents felt more confident after viewing the content, indicating that for some, the information provided was effective and empowering. -Decreased Confidence: 11 respondents felt less confident, which may suggest that the content could have raised awareness of the complexity of the situation, but did not provide clear solutions to the problems raised -Unchanged Confidence: 15 respondents reported no change, this points to the possibility that the content only reinforced their existing knowledge without providing any new insights, or the content failed to engage them sufficiently to empower them.
Perceived Effectiveness
The majority found the information provided on the page moderately useful (17, 43.6% rated it a 3), with others rating it as quite useful (4 and 5 ratings combine for a total of 11, 28.2%.
While the majority found the content at least moderately useful, this spread highlights room for enhancement to increase perceived utility, perhaps if I could include more in-depth information or actionable steps.
Perceived Utility of Suggestions
Intent to use the suggestions provided shows a positive trend, with 4 and 5 responses accounting for close of half of the responses, indicating a likelihood of utilizing the suggested steps.
This suggests that the suggested actions are considered actionable and relevant by many of the respondents. There still is a significant portion who are unsure about it, and some who don’t feel like these suggestions are valid, which I will touch on later.
Open-Ended Feedback Section
To gain deeper insights and understand any potential reservations or concerns that participants might have, I concluded the survey with an open-ended question. Participants were asked if they had any concerns or comments about the information shared and the proposed use of tabletop games to help socially anxious gamers improve their offline social skills.
As seen above, the responses varied widely. -Engagement Challenges: A common concern was the difficulty in inviting a socially shy gamer to participate in tabletop game activities. This reflects a broader issue of getting them to engage in face-to-face settings. -Doubts: Some respondents were skeptical about the actual impact of tabletop games on real-world social skills. For instance, one asked, “How much can a game really do?”, I think this indicates a need for clearer evidence and rationale behind the use of gaming to help. -Anecdotes: There were mentions of personal experiences where gaming sessions seemed to facilitate enjoyment and interaction. But there was also another note stating that the tabletop games did not really help outside of tabletop gaming. It made clear that while tabletop gaming can be a positive social outlet, it may not uniformly translate to improved social confidence outside of gaming contexts, some other help might be needed. -Perceived Credibility: The remark that the it “looks like a scam” points to a critical need for enhancing the credibility and presentation of the intervention to build trust. Also the feedback “not a lot of info on the page but I get the idea” indicates a need for more detailed and clearly communicated information.
Implications for Future
As I reflect on the feedback regarding skepticism and the perception of this being a potential “scam page,” it’s clear that providing clear evidence and rationale is important. However, I’m conscious of the balance needed to avoid overwhelming with dense informational content, which could cause people to lose focus. To tackle this, on the top of my head I’m considering moving towards social media platforms, which would allow for the dissemination of information in short, digestible bursts.
After doing this second intervention fully online, I am thinking about moving back to physical in person interventions where I can get more qualitative feedback. I recognize the value of direct human interaction in building trust and communicating complex ideas. How can I introduce physical elements, I need to dwell on this a little bit longer and I need to think about how to reach my target audience, which was my main concern over moving this fully digital and online. I could link the in person activations to online social media, creating a cohesive experience that combines both online and offline interactions.
I’ve made a rough digital page filled with relevant information aimed to inform and empower friends and family of the shy gamers in a way that equips them to be able to help nudge these gamers onto a path of being more comfortable in face-to-face social settings. Initially, I considered distributing a PDF but then opted to have a webpage link instead to ensure that the info remains accessible and won’t be easily lost.
I kept the page concise, my goal is the provide just enough insightful content without overwhelming visitors, which could lead to potential early bounces from the site. Keeping it brief also makes it easier to consume the material, complete the page and provide feedback through the follow up survey.
The page was distributed on August 2nd through Discord, a popular platform among gamers. Specifically, I shared it within the CSGO Asia server, which has around 5,539 members, mostly English speakers from the Asia Pacific region. Additionally, I spread the word through personal gaming friend groups on Discord I’ve joined over the past decade, reaching the US and EU as well. Looking forward to see the results.
