Shared Confidence and Uncertainty Using Google

Shared Confidence and Uncertainty Using Google: The Effects of Mental Models on Older and Younger Users’ Search Experiences

Abstract

Prior research found disparities in information-searching processes between older and younger adults. This study shows that generational differences between older and younger users’ mental models and experiences of online search engines are lessening over time as users gain experience and Google’s pervasiveness in daily life expands. 17 younger adults and 18 older adults, all experienced search engine users, participated in a series of methods that facilitated comparative analysis, including a questionnaire, realistic information search session using Google, and interview. The results indicated that previous experience strongly influenced mental models, instilled self-efficacy, and heightened expectations of Google for both groups. In the face of uncertainty, however, both older and younger users abandoned their confidence in Google and employed manual strategies to overcome Google’s limitations. While mental models developed over time generated satisfactory results for both older and younger users, identifying where their confidence faltered revealed opportunities for improvements to search engine design. This study concludes with recommendations that address areas where users’ mental models mismatched the system and promote a positive user experience for all generations. 

---

Research Questions and Objectives

• Research Question: Do similarities and differences in how older and younger users use search engines in their everyday lives help to explain users’ mental models of search engines? 
  • A: How do older and younger users’ previous experiences using search engines inform users’ search experiences? 
  • B: How do older and younger users’ perceptions of search engines inform users’ search experiences? 

To achieve the research aims, this study carried out the following objectives: 

1. Investigate and compare how the role of information search in older and younger users’ everyday lives informs users’ mental models. 
2. Conduct interactive information search sessions to identify patterns within and compare between older and younger users’ mental models, search strategies, and resulting search experiences. 
3. Measure and compare older and younger users’ number of search modifications made, perception of task difficulty, level of satisfaction with results, and level of satisfaction with own performance to quantify users’ search experiences in relation to mental models. 

Methods

This study was conducted between 1 July 2021 and 1 October 2021. A naturalistic user-centered evaluation of Google was conducted to understand how well the user, retrieval mechanism, and the database interact to retrieve information under real-life conditions. The study was conducted with 35 participants, where the younger group was made up of 17 participants ages 21 to 36 years and the older group was made up of 18 participants ages 60 to 75 years. Participants completed an online questionnaire to record demographic data, interactive information search session with think-alouds, post-task metrics to collect complementary quantitative data, and a structured post-search interview. Realism of search tasks was central to the observation of genuine search behavior and assessments of user experience. Using multiple methods with a focus on qualitative data enabled the collection of varied data through which to build a deeper, richer understanding of participants’ mental models and search experiences. A pilot study was also conducted in three rounds. All data was collected remotely. 

Qualitative data was analyzed using the thematic analysis framework presented by Braun and Clarke (2012), which facilitated identification of important themes and comparison between older and younger users’ search experiences. The distributions of the quantitative data collected were analyzed to pinpoint patterns or disparities in user experience, satisfaction, or search tactics between age groups. Differences were tested for statistical significance and analyzed for implications in light of the corresponding qualitative data. 

Summary of methods

Pre-Search Questionnaire

The questionnaire asked participants about demographic information, the role of information search in their everyday lives, and their levels of experience and confidence using online search engines.

Both younger and older participants reported using search engines for many years; on average, 21 years for the older group and 16 years for the younger group. Both groups also reported high levels of confidence using their preferred search engine, with an average of 8 out of 10 for the older group and 9 out of 10 for the younger group. Both groups have had time to build mental models, strategies, confidence, and perceptions of search engines.

The younger group reported spending significantly more time on the internet per day (p value=0.002) and using their preferred search engine significantly more frequently (p value=0.005) than the older group. While these differences suggest that the younger group would be more confident searching, they did not impact the main outcomes of the study because the older group had been using search engines for longer and reported similar levels of confidence as the younger group.

Interactive Search Session

Participants were given 4 problem-solving tasks to complete using Google. Task design and facilitation followed the user-centered information retrieval evaluation technique detailed by Borlund (2009) offering both realism and control. The level of difficulty of the tasks was determined based on previous research, balancing value to the research questions with ethical implications of task complexity.

TROUBLESHOOTING TASKS: 

Imagine you have a MacBook Air. When you press the “8” key on your keyboard, nothing appears on the screen. The key does not feel stuck and you haven’t spilled anything. Use Google to find a few possible solutions. [If only one solution is found, ask participants to show what they would do next if the solution did not fix the problem.] 

Your shower is making a high-pitched noise when you turn it on. You bought a new shower head 6 months ago and never had an issue with your old shower head. Use Google to find a few possible solutions. [If only one solution is found, ask participants to show what they would do next if the solution did not fix the problem.] 

COMPARATIVE SHOPPING TASKS: 

Use Google to help decide which mobile phone to purchase next for yourself or a family member. 

Imagine you’re buying a gift for a friend or family member who likes music, traveling, and eating out. Use Google to help. 

All 35 participants were familiar with the majority of or all tasks having done the same or similar tasks before. Familiarity with tasks was influential as participants often repeated strategies used previously; “I knew what words to put in to find what I wanted because I’ve searched for comparing products to each other before” (PY4), or anticipated the type of results they would get; “Because I actually did this search once, I know how it works” (PO1). Participants frequently drew on their familiarity and previous experience, key factors in the development of mental models (Norman 1983) to formulate search queries, filter results, and make decisions about which results explore further. 

Participants were also familiar with Google, using Google to find information instinctively for daily questions and problems. Participants spoke about a sense of spontaneity, where Google was used to search for “whatever’s going on in your life at a given time” (PY5). Both groups also addressed how their use of Google has changed as their life has changed, for example, “I’m starting to look at stuff that has to do with being a senior or being an older person” (PO17), further evidencing a high level of familiarity with Google formed over time. 

Post-Task Questions

After each task, participants were asked 3 questions to record their perceived level of difficulty of the search task, satisfaction with the results, and satisfaction with their performance, measuring participants’ immediate reactions to their search experience. The quantitative data collected supplemented observational and interview data, allowing for comprehensive comparison between age groups.  

1. On a scale of 1-5, how easy or difficult did you find this search task?
   1. Very difficult 
   2. Somewhat difficult 
   3. Neither difficult or easy 
   4. Somewhat easy 
   5. Very easy 
2. On a scale of 1-5, how satisfied are you with the results? 
   1. Highly dissatisfied
   2. Dissatisfied
   3. Neutral
   4. Satisfied
   5. Highly satisfied
3. On a scale of 1-5, how satisfied are you with your performance?
   1. Highly dissatisfied
   2. Dissatisfied
   3. Neutral
   4. Satisfied
   5. Highly satisfied

Summary of Findings

Participants were highly confident finding information using Google based on previous experience, and their familiarity and fluency with the tasks given in the interactive search sessions as well as the format and content of Google results created a natural environment for the observation of participants’ genuine search strategies. 

Both the younger and older groups showed confidence in Google’s capacity to understand their query and return suitable results, as well as in their own abilities to search effectively, manipulate results, and find helpful information using Google. While these two realms of confidence may seem incongruous, they together made tasks feel easier, amplified participants’ satisfaction with the search results and their performance, and improved the search experience. Self-assurance and confidence in Google were built on an understanding, or mental model, that led to success often enough to have built confidence, regardless of the mental model’s technical accuracy. Participants developed search strategies over time using Google and discovered what worked well for them and what made sense with their mental models, for example ways of filtering results or influencing Google’s top results. These mental models informed search strategies, which, although differing, worked for participants as shown by their answers to post-task questions. Participants continued to build these mental models and modify their search strategies as they identified frustrating or undesirable parts of their experience in their daily lives based on Google’s shortcomings. 

The major exception for both the older and younger groups to the confidences shown was moments of uncertainty when people found Google’s limitations. Contrary to expectations, identification of these limitations was not largely skewed towards one group or the other. The majority of participants in both the older and younger groups faced uncertainty, recognized Google’s limitations, and used strategies they had formed to overcome those limitations. The distribution of older and younger participants across those strategies varied, however. Younger users were more likely to save options for later reference and verify results or options across different sites. Older users were more likely to go directly to a known site. Both groups compared options by using tabs or comparative language in search queries. The limitations participants faced and strategies they used to overcome these limitations are the result of mental models formed over years of experience.

Key Similarities

• Built mental models, search strategies, and confidence over many years of using search engines, demonstrated by satisfaction with results of tasks and performance during tasks
• Showed high levels of both self-confidence finding information using Google and confidence in Google’s ability to return relevant results
• Used Google to find information spontaneously and instinctively for daily questions and problems
• Developed strategies for formulating effective search queries and choosing known results based on previous experience
• Expected efficiency, as evident by engagement with top results and featured snippet
• Conducted search queries to return results to be used as inspiration or a starting point
• Recognized Google’s limitations in comparison, decision making, and specific services
• Felt that despite their attempts, superior results may exist
• Used strategies to compare between results, solutions, or options and overcome Google’s limitations

Key Differences

Older group	Younger group
More often mentioned benefits of a large number or results or options	Used more strategies to target the top results: “top”/”best” in query, lifting keywords from results, including a known website in query, and personalizing tasks
More often carefully considered search strategy or query given anticipated outcome of a search based on previous experience	More often filtered results in their mind, as opposed to adding constraints to their search query
More often commented on appreciation, admiration, and enjoyment using Google	More often discussed using Google to access “other people’s” experiences
Interacted more often with ‘People also ask’	Interacted more often with YouTube videos
Demonstrated patience and perseverance more often	Demonstrated impatience and haste more often
More often overwhelmed by large numbers of options	More often compared options to verify validity or quality
More often went directly to a known site rather than using Google	More often used strategies to save options for later reference or comparison
Found troubleshooting tasks easier	Found comparative shopping tasks easier
More satisfied with their performance than with the results of the search within a given task	Less satisfied with performance when tasks were perceived as easier, implying high expectations of themselves

Recommendations for Improving Search Engine Design

While participants generally had highly satisfactory experiences, there are opportunities for improvements to search engine design that promote a positive user experience for users of all generations. Along with improving the experience for today’s groups of older and younger users, the recommendations made in this section will remain relevant even as more tech-savvy generations retire; “We predict that today’s younger adults will experience loss of confidence in technological ability as they age; that they will favour familiar ways of accomplishing their goals, which are intertwined with the technology available at the time they established these practices” (Knowles and Hanson 2018). Search engine design and functionality will no doubt continue to advance, and it is crucial that designers are attentive to the following factors when evaluating and promoting the user’s experience. 

ENCOURAGE EXPERIENCED USERS TO SHARE THEIR PERCEPTIONS TO IMPROVE USER EXPERIENCE. 

Be clear how the design of online search engines corresponds to the role of search in users’ everyday lives. 

Seek both positive and negative perceptions from experienced users when evaluating the user experience of online search engines. 

Recognize and include a broad range of generations in definition of ‘experienced user’. 

LEARN FROM EXPERIENCED USERS’ SEARCH STRATEGIES TO IMPROVE SYSTEM DESIGN. 

Embrace common search strategies experienced users implement to overcome system limitations. 

Help users identify similarities and differences between the content of top results. 

Recognize search queries that seek comparison and generate a succinct comparison table. 

Allow users to indicate where they have had success and include that website in the top results of related future searches. 

Conclusions

Far more similarities than differences were found between older and younger users’ search strategies in everyday use of Google. Both groups showed self-efficacy and confidence in Google based on an ability to predict and an understanding, or mental model, built out of previous experience using Google in their daily lives and familiarity with search tasks. The exception to this confidence for both groups was where users displayed uncertainty and identified Google’s limitations. The similarities in search strategies formed to overcome limitations identified in this study suggested that older and younger users formed mental models based on experience and faced similar uncertainties and limitations regardless of their generation. 

The data substantiated earlier research, providing evidence that previous experience was a very important factor in the development of mental models of search engines. Previous successes, failures, and trial and error influenced users’ search query wording, decisions on which results to interact with, and choices of if or when to modify their search. Positive previous experience built self-efficacy over time. With substantial previous experience, users instinctively repeated strategies that resulted in success in the past and formed strategies over time to overcome Google’s limitations, enriching individual satisfaction with their search experience. 

The results indicated that experienced users’ perceptions of search engines influenced their expectations of ability to access information in an efficient manner, implying a high level of confidence in Google. These expectations were accurate the majority of the time, with the exception being when participants reached Google’s limits. In the face of uncertainty, frustration, or impatience, users abandoned their confidence in Google and turned to their own abilities, employing strategies to overcome Google’s limitations. 

Users of both generations were mindful of Google’s limitations and formed strategies established through previous experience to overcome these limitations, including comparing between options to determine the best choice at hand, corroborate results, or seek out differences. Participants’ natural use of these strategies suggested that their mental models informed approaches that surmounted the negative impact to the user experience from Google’s shortcomings. These conclusions are indicative of improvements that can be made to search engine design to promote a positive user experience for users of all generations.

A full report is available upon request.

CONDUCTED AS PART OF INM363 INDIVIDUAL PROJECT (PRD3 S 2020/21) AS PART OF MSC HCID AT CITY, UNIVERSITY OF LONDON.

More projects:

Portfolio

Accessibility Review

Portfolio

Assisting Family Members with Technology in a Remote World

Portfolio

Usability Evaluation

Portfolio

Artist Next Door

Portfolio

Spread the Light

Advertisement

Spread the Light

Objective

Originally designed during the coronavirus pandemic for Memory Cafe, a specific community of individuals with dementia and their caregivers. I aimed to design an interactive technology to bring caregiver communities together in a highly meaningful way, reminding them that they are not alone in the challenges they face, that is nearly effortless to use and requires minimal time commitment to learn and to use.

User-centered design process

User research was conducted with community members via semi-structured interviews on Zoom. My goal was to understand the caregiver community in terms of who the people are, how and why they connect, what activities they engage in, the problems they experience, and how things have changed during the pandemic. The trends I found in user research informed two personas whose needs, goals, and characteristics represent the larger community of users.

Persona 1: Virginia Wilson

Virginia is the caretaker for her father, who was diagnosed with Alzheimer’s disease three years ago, in her home. Virginia spends the majority of her time with her father, helping him with everyday tasks such as cooking, remembering medications, and cleaning.

She appreciates both giving and receiving support from the caregiver community; “We come together in an atmosphere that’s very accommodating, where we don’t have to explain what the limitations are. I’m able to connect with folks who are sharing a similar journey.”

Virginia is worried watching her father struggle with isolation and the changes brought about by the pandemic. She feels alone in the difficulties she faces as a caregiver, but often doesn’t have time to reach out to friends in her community.

Persona 2: Henry Jones

Henry is a retired operations manager who spends much of his time caring for his wife, who was diagnosed with dementia two years ago. He enjoys seeing his two adult children, hiking, reading, and playing cards with his wife.

Henry enjoys watching his wife engage in the art and music activities the community facilitates, and he has found the caregiver community to be a valuable support system: “We’d chat about how we were doing ourselves, and how our loved ones were doing. The care isn’t just there for my wife, but also for us as the caregivers.”

Henry’s daughter used to come help out, but the pandemic has made that much harder. He feels overwhelmed watching his wife’s symptoms worsen with increased isolation during the pandemic and feels he has lost touch with his friends from the caregiver community.

Prototyping Spread the Light

I brainstormed a range of different forms, technologies, and means of interaction that might meet my design goals. It was important that the design not add another item to the users’ list of things they must accomplish; instead, connecting with their community should be a break from everyday stress. The more promising designs balanced the goals of encouragement and support with technology and functionality that comes naturally and easily.

Spread the Light is a tabletop spherical light on a base with an interactive touch screen. Users send light messages to and receive light messages from their community. The 8 light messages are based on sentiments that came up in user research: love, friendship, smile, positivity, thumbs up, hug, missing you, and happiness. Each light message is associated to a different color, where when sent, everyone in the community’s devices light up that color for 10 minutes; for example, if someone sends “Smile”, everyone’s lights turn yellow. Happiness is rainbow, and the light cycles through all seven colors.

Small screens, big accessibility

The visual design considers the layout, labelling, and structure, of which the main goal is simplicity. The visual design is minimalistic, supporting the primary goals. There are not a lot of options, and the available options are clear in their functionality. This is based on user research, as community members do not have excess time to spend figuring out new technology and are more comfortable when technology is easy to use.

The Home screen displays eight colored icons – love, friendship, smile, positivity, thumbs up, hug, missing you, and happiness – that the user can tap to send a light message to their selected community.

Because of the small screen size, it is important to consider accessibility in the screen designs. The text is large and readable. There is high contrast between the foreground and background colors (5.61:1 between the white text color and gray background color), ensuring users can read the text on the screen. Actionable items are indicated, either by visual button design or by textual instructions on the screen. In development, it would be important to ensure reasonable target size and spacing for each of the interactive elements, such as the icons on the Home screen. Lastly, the amount of text entry is reduced as much as possible.

• 

• 

• 

• 

• 

• 

User Feedback

“The idea of just being in your home and then that light going on, that someone is thinking of you, is just a delightful idea.”

“It’s very bright and uplifting. It brought a smile to my face. It would be like a shower of love.”

“It feels very engaging. Simple, but colorful. And it feels like it’s not complex to make happen.”

Created for INM452 Interaction Design (PRD1 A 2020/21) as part of MSc HCID at City, University of London and awarded a distinction.

More projects:

Portfolio

Shared Confidence and Uncertainty Using Google

Accessibility Review

This expert accessibility review was conduced on behalf of Library of Things and examined a key customer journey: landing on the home page, adding a product to the cart, and checking out. The accessibility review considered a range of users with different needs, including sensory, physical, and cognitive impairments. The intent of the review was to provide guidance to Library of Things concerning the most important accessibility issues customers might face and suggest fixes to those issues. Ensuring equal access to the Library of Things website has the potential to expand Library of Things’ customer base, strengthen the organization’s mission, and remove barriers to participation.

Method

Dates of Review	October-December 2020
Website Name	Library of Things
Pages Reviewed	Home, Browse the Things, Cordless Jigsaw, Cart, Checkout, Choose Dates modal, See Availability modal
WCAG Version	WCAG 2.1
Conformance Target	Level AA

The web pages were reviewed on a Mac using Safari and Chrome. Standard tools aided in the accessibility review process. WAVE Evaluation Tool, a Chrome extension developed by WebAIM that aligns with WCAG 2.1 guidelines, was used to conduct an automated check on each page. This was used as a starting point for testing several criteria including structure, contrast, language attributes, labelling, and alternative text. VoiceOver on Safari was used to assess the pages’ functionality with a screen reader from the perspective of users with visual impairments. The Elements, Styles, and Accessibility panels in Chrome development tools were used to check specific factors in the HTML and CSS such as labelling, structure, and HEX codes used across pages. Checks were also performed using page zoom and font size in Chrome appearance settings and Easy Window Resize extension for Chrome.Because WCAG 2.1 only covers content, such as images, text, structure, and presentation, WAI-ARIA 1.1 (Accessible Rich Internet Applications) was used to review accessibility and suggest fixes related to dynamic content and advanced user interface controls.

The accessibility review was conducted methodically across the given scope. Each interface element was manually inspected to determine if a WCAG 2.1 guideline applies and if the element passes or fails. WebAIM’s WCAG 2 Checklist was referenced to aid in the comprehensiveness of the review and recommendations provided. Next, pages were cross checked for consistency and predictability, and to establish how many pages in the key customer journey are affected by a given issue. This enabled a consolidation of issues, rating of the severity of each issue, and prioritization of the issues included in this review.

Accessibility Severity Scale

The severity scale used in this review was adapted from Nielson Norman Group’s Severity Ratings for Usability Problems and Userfocus’ Severity Level Decision Tree, both of which measure usability problems. These scales were adapted to assess the severity of accessibility issues based on whether the problem occurs during a frequent or critical task, the impact of the problem in terms of number of potential users impacted and how difficult the problem is to overcome for those specific users, and the persistence of the problem.

Critical	Urgent to fix: this accessibility problem will make some users unable to complete a common task.
Serious	Important to fix: this accessibility problem will significantly slow down some users when completing a common task.
Minor	Low priority: this accessibility problem will frustrate some users but does not affect task completion.
Cosmetic	Bonus fix: this accessibility problem refers to quality or cosmetics and should only be fixed if extra time is available.

Findings

Fourteen accessibility issues were included in the findings presented to Libary of Things, arranged into four categories – Interaction, Presentation, Navigation, and Structure – then sorted by severity. Four issues were rated “Severe”, meaning they prevent groups of customers from completing important tasks on the site and are of the highest priority to address: appropriately labelling the interaction required to select reservation dates, alerting assistive technologies to error messages, raising contrast ratios, and improving readability when pages are highly magnified. Less urgent recurring issues, including heading structure, mark-up of user interface controls, and keyboard focus, were also described in detail, along with examples and suggested fixes.

A full report of findings is available to view upon request.

Created for INM313 Inclusive Design (PRD1 A 2020/21) as part of MSc HCID at City, University of London and awarded a distinction.

More projects:

Assisting Family Members with Technology in a Remote World

Published as a poster in American Psychological Association’s Technology, Mind, and Society Conference 2021

apa-tms-posterDownload

Epstein, B., & Marengo, L. (2021). Assisting Family Members with Technology in a Remote World. In TMS Proceedings 2021. Retrieved from https://tmb.apaopen.org/pub/iximiuvh

---

Abstract

The coronavirus pandemic imposed a dependence on technology unfamiliar to many, amplifying the need to learn new systems and troubleshoot problems. This study aimed to understand user behavior in the context of adult children assisting their parents with unfamiliar digital technologies in order to recommend improvements to digital technologies’ capacity to support less confident users. Naturalistic observations were conducted over Zoom of 3 pairs. Participants chose a desktop-based task the parent needed assistance with from their child. Natural communication between participants throughout their interaction with their chosen systems enabled robust data collection and unobtrusive observation. Data was analyzed through two theoretical Human-Computer Interaction frameworks: Resilience Strategy, proactive contributions users make to avoid mistakes and maintain performance, and Distributed Cognition of Teams (DiCoT), which examines how users with a common purpose interact with technology. The results show that children more readily deployed resilience strategies, transferring their comfort using technology to unfamiliar systems and communicating with their parents to avoid errors. Digital technologies can adopt this role by anticipating common errors and making the resilient action evident, supporting less confident users and reducing anxiety around making mistakes. Analysis through DiCoT revealed how children filter interface feedback into task-centered language, helping their parents understand the result of an action, reassuring them, and prompting them to continue. Hints, descriptions, or tooltips placed where users need confirmation and written from the user’s perspective can explain the purpose of an interface element, extract important information, and empower less confident users to make sense of unfamiliar systems. 

---

Theoretical Approaches

Resilience Strategy

Resilience strategies are proactive contributions users themselves make in interacting with technology that help to avoid mistakes and maintain performance. In this study, analyzing how younger users’ resilience strategies compensate for older users’ susceptibility to errors promotes understanding of older users’ needs and informs principles for supporting less confident users by nurturing these strategies.

Distributed Cognition of Teams

Distributed cognition of teams (DiCoT) examines the context in which a pair of users with a common purpose interacts with a system and how contextual factors, coordination of knowledge, physical layout of the environment, and flow of information impact performance and ability of users to collaboratively accomplish their goals.

Observation and Data Collection

Participants were asked to choose a task and system the parent currently needed assistance with from their child. While participants knew the purpose of the study, they were unaware of any specific behaviors being analyzed so as not to influence their actions. Participants were asked to behave as naturally as possible, as if they weren’t being observed.

Participant-chosen tasks	Participant-chosen system
Upload documents	Bank website
Create an account and rent a pre-chosen audio book	Audio book website and app
Edit an existing digital design	Online graphic design platform

All participants carried out their chosen task in their homes using their own devices. Observation was conducted remotely over Zoom, enabling video, audio, and screen recording of qualitative data. While observation in the home has previously been considered a rather controlled environment, restrictions due to the coronavirus pandemic made this context natural for participants.

Inherently in the task of helping another person, participants communicated out loud regarding their thoughts, actions, and understanding of the system at hand. This achieved similar goals to a think-aloud study by furthering insight into participants’ reasoning while avoiding data bias from probes, allowing the observer to remain as unobtrusive as possible and collecting more candid data.

Findings

Resilience Strategy

Examining how these younger users’ resilience strategies compensate for older users’ susceptibility to errors gains important insight into how technology might better support less confident users.

• Perception: Parents, in turning to their adult children to assist them, perceived the children as more skilled with technology despite the fact that these were not tasks the children carried out on a regular basis. Equally, the children assumed they would be successful in helping their parent with the technology at hand despite its unfamiliarity to them, a confidence in part based in their adoption of resilience strategies.
• Compensation: Instead of the parent being aware of their own limitations concerning technology, their child, having seen them make mistakes previously, utilized resilience strategies to avoid potential mistakes and make the task go more smoothly.
• Communication: When using a resilience strategy, all three children communicated to their parents the action they were taking and why, and while the parents were not explicitly asking for their children to look out for potential errors, this communication seemed to reduce parents’ anxieties at having to recover from mistakes in the process.

Distributed Cognition of Teams

How participants collaborated to achieve the aim of the task itself, as well as aid the parents’ understanding, learning, and comfort provide insight into supporting the act of assisting less confident users in completing tasks when more confident users are not available to help.

• Physical arrangement of the space: In all three observations, parent and child sat directly next to each other facing the computer, so both people could see what was happening on the screen, access the mouse and keyboard controls, and hear each other well. This common arrangement determined how information moved through the environment. In all three teams, both parents and children referenced what was happening on the screen out loud and pointed to elements on the screen with either mouse or finger, facilitating more effective communication and a shared understanding between parent and child.
• Filtering information: In all three observations, communication from the child to their parent focused, distilled, and translated information and feedback from the interface to directly connect it to the task at hand and make it easy for the parent to understand. This act of filtering information on the children’s parts helped the parent to see feedback on the screen and successfully complete the task at hand, informing recommendations for future technologies to take on the role of filtering information to make its meaning more easily understandable for all users.

Design Principles

Incorporate assistance for less confident users into digital technologies. Younger, more confident users more readily deploy resilience strategies, even when using unfamiliar digital technologies, as their comfort recognizing conventional elements extends their confidence. Children identified shortcomings in their parents’ ability to use a given system more aptly than the parents themselves, and used this awareness to prevent potential mistakes before they were made. Digital technologies can take on this role by anticipating common errors, making the resilient action evident, and explaining the reasoning to users. Integrating the resilience strategies children utilized in aiding their parents into digital technologies can better support less confident users and reduce anxiety around making mistakes, encouraging longer-term confidence using technology.

Translate feedback into task-centered language. A close analysis of how children filtered information from the interface exposed a common theme of translating feedback into wording directly connected to the task at hand. Doing so made it easy for the parent to understand the result of an action, reassuring them that they were on the right track and prompting them to continue. Hints, descriptions, or tooltips placed strategically where users need confirmation can be made to function in a similar way by explaining the purpose of an interface element. Explanations should be written from the user’s perspective in the context of common tasks and free of system-specific jargon. Empowering less confident users to explore and make sense of systems themselves will help to build confidence when children are not there to fill that role.

Design interface elements to support the act of helping others with technology. Less confident users rely increasingly on others to help them with unfamiliar technologies. Understand that someone may be helping the user, or multiple users may be collaborating to complete a task that is typically thought to be more individual. Interface elements can be designed to aid communication by using easily identifiable colors, labels, shapes, or icons that make them easy to describe. Doing so allows users to effortlessly reference elements on the screen in a way that is easy for the other user to recognize, creating a shared understanding between users, supporting effective communication, and facilitating the act of helping others with technology.

Make it easy to help others with technology from a remote location. Not all users have access to in-person support. Screen mirroring technologies that facilitate this same effective, supportive helping when users and their helper cannot physically be in the same place. Users’ understanding and communication benefitted from their ability to view the same screen, point to and reference elements of the same interface, hear each other well, and share access to mouse and keyboard controls. Current or future screen mirroring technologies should consider these factors in their designs. For example, if the less confident user is to share their screen, that action must be as easy to find and initiate as sitting down in front of the same device. The user doing the assisting should also be able to access mouse and keyboard controls from their own device, with permission, to take on the role of showing how an action is carried out. Perhaps, both users’ mouse pointers are visible to facilitate referencing elements of the interface. Keeping the important elements of in-person helping in mind in designing screen mirroring technologies that enable remote helping will aid communication, promote a shared understanding, and support feelings of comfort and reassurance for less confident users.

A full report is available upon request.

CONDUCTED AS PART OF INM314 Understanding User Interactions (PRD1 A 2020/21) as part of MSc HCID at City, University of London AND AWARDED A DISTINCTION.

More projects:

Usability Evaluation

Usability Evaluation of Alzheimer’s Society Website

Summary

Conducted on behalf of the Alzheimer’s Society, this evaluation assesses the usability and user experience of the Alzheimer’s Society website. With millions of users per year, the effectiveness and ease of use of the Alzheimer’s Society website is vital to providing resources to people affected by dementia and supporting fundraising efforts. The purpose of this evaluation is to identify usability problems, defined as any aspect of the user interface that adversely impacts a user’s performance, in order to improve the website.

Remote moderated usability testing was conducted with eight participants, as it allowed for a focus on users’ behaviors and insight into users’ attitudes . Usability problems were documented according to their severity and informed four key recommendations for improving usability and user experience for the diverse community of users of the Alzheimer’s Society website.

Goals & Tasks

The following goals were informed by concerns stated in the brief. Tasks were carefully written to address the goals by collecting data on areas of interest of the site and supporting exploration to understand overall user experience. Success was defined for each task before beginning usability testing sessions.

	Goal	Task	Task is complete when…	Task completion rate
1	Explore how the home page informs users’ understanding of the Alzheimer’s Society and expectations of the site.	Explore the home page but don’t click on anything just yet. Describe your impressions. In your own words, who is the Alzheimer’s Society? What do you expect to be able to find and do on this website?	Participant has described their impressions and answered the questions in the task.	100% (8/8)
2	Determine how easily users can make donations to the Alzheimer’s Society.	You’re interested in supporting the work the Alzheimer’s Society is doing. Give £100 to fund their cause.	Participant has successfully gone through the donation form but before clicking “Complete £100 donation”.	100% (8/8)
3	Determine how easily users can find and sign up for a fundraising opportunity.	You want to raise money for the Alzheimer’s Society another way. Find and register for an activity that you can partake in from home.	Participant has successfully found an event and filled out the registration form (when applicable) but before clicking “Continue”.	75% (6/8)
4	Determine how easily carers can find information and resources related to increasing the independence of someone living with dementia.	Find a few ideas for how you might help someone who has recently been diagnosed with dementia and is living alone to maintain their current lifestyle.	Participant finds 2-3 different resources related to independence and feels satisfied. If a user finds less than 3 resources, prompt once to see if they can find more.	100% (8/8)
5	Determine how easily users can find and engage with the Alzheimer’s Society forums.	Imagine you’re caring for someone living with dementia. It can be helpful to connect with other carers with similar experiences. Find out about other people’s experiences caring for someone living with dementia and how you would share your own online.	Participant lands on a Dementia Talking Point forum page they deem relevant.	75% (6/8)
6	Understand and measure the overall user experience of the Alzheimer’s Society website.	Conduct SUPR-Q questionnaire to measure overall user experience broken out into four parts: usability, credibility, loyalty, and appearance.	Participant completes SUPR-Q questionnaire adapted from Sauro (2015).

Verb-based tasks were utilized to invite users to complete a specific action, strengthening evaluation of the website’s functionality across users. Tasks were worded to be appropriate for participants and, in order to gather unbiased data, avoided using words from the user interface or telling users where to go. Task scenarios were simple, generic, and relatable, enabling users to find information important to them, role playing where necessary. As dementia topics can be upsetting, tasks aimed to avoid emotional reactions, keeping the emphasis on website usability.

Participant Recruitment

As millions of people access the website each year, recruitment of participants focused primarily on experiences that might cause people to behave or feel differently when interacting with the website. Participants were asked about their level of experience making donations online, fundraising online, and with dementia topics, measured using a Likert familiarity scale. Participants were also asked how many hours per day on average they spend on the internet as a means to gauge their level of comfort navigating websites. In order to achieve the desired diversity in all segments of experience, internet usage, and demographics, eight participants were recruited for this study.

Recruiting a mix of demographics was also considered, and the participant pool consisted of four women, four men, four people between the ages of 20-40, and four people between the ages of 40-60.

Moderated Usability Testing Sessions

Because the aim of this evaluation is to improve the website’s usability and user experience, qualitative data from usability testing was emphasized. All usability tests with participants were conducted remotely over Zoom, which allowed for video, audio, and screen recording of participants’ interactions with the website. These methods of collecting raw data made in-depth analysis possible through which to address the goals stated above.

Participants were asked to think aloud as they carried out the tasks, a robust method for collecting qualitative data on why users take each action and what they are thinking and feeling about the website. As prompts that encourage users to think aloud can bias user behavior, precautions were taken to impartially incite users to share their thoughts when it was unclear what they were thinking, looking at, or doing. Follow-up questions were asked about any issues or noticeable impressions to ensure a complete understanding of usability problems and the user’s experience. 

Reporting Results

Qualitative data from usability testing was coded for usability problems, positive observations, and other feedback. Next, the positive observations and usability problems were summarized in a rainbow spreadsheet that for each, provided a description, context, severity rating, re-design recommendation if appropriate, which participants experienced the issue, and a telling quote from the usability testing sessions if applicable. Careful attention was paid to determining where the same usability problems appeared in different situations. Summary statistics were also calculated for each task and SUPR-Q results were analyzed in Excel. A record was kept of the pages visited by each participant in each task including page titles, URL’s, participants, and task number. The page titles match those in the transcripts and rainbow spreadsheet, thus making it easier for the Alzheimer’s Society to understand the range of pages looked at, see overlap with other evaluations, and address the usability problems. 

To respect confidentiality, specific results are not presented here. A more detailed report is available upon request.

CREATED FOR INM315 Evaluating Interactive systems (PRD2 A 2020/21) AS PART OF MSC HCID AT CITY, UNIVERSITY OF LONDON AND AWARDED A DISTINCTION.

More projects:

Artist Next Door

Objective

I love buying gifts for friends and family from local artists – their art is unique and it’s good to support creative people in my community. When the coronavirus pandemic hit, art fairs and craft markets were harder to come by. Etsy has lots of art by small artists, but I felt the loss of connection with local artists and missed hearing the story behind the art. I aimed to design the information architecture of a website that enables people to discover, connect with, and buy art from small artists in their communities. I consider web accessibility guidelines (WCAG 2.1) throughout the process to ensure Artist Next Door will enable everyone to support local artists.

A. Domain Model

I interviewed two domain experts – a retired art teacher who creates her own art, mainly collages and prints, and a young amateur artist who paints as a hobby – with the objective of understanding the scope of the visual art domain. I approached the interviews with curiosity on how and why these artists do what they do and how art fits into their lives both as artists and as consumers of art.

B. Sitemap

In an open card sort, most users separated content related to artwork from content related to artists, resulting in two major navigation paths: Discover Artists (1.0) and Shop Art (2.0). Users can access the content pages in more than one way, as shown by the second-level pages in the database structure in the sitemap (1.1-1.4; 2.1-2.3), fed by databases of artists and artworks respectively. Artist and artwork content pages have a consistent structure. One artist page links to many artwork pages and vice versa, as shown by the key cross link.

Navigation and page labels are intended to meet users’ mental models. I conducted a tree test to evaluate the organization and labelling of the navigation. Users easily found an artist’s page but took multiple routes in shopping by specific art forms. While artists used the term “art forms” in the expert interviews, users were more familiar with labels such as “categories”. I chose to use By Category (2.3) to aid in clarity and familiarity for users.

C. User Journey

The user’s goal is to find an artist local to where they live who creates sculptures made out of recycled materials.

The path highlighted in yellow shows the path most often taken by users when given this task in the tree test. Common feedback from the tree test was that users wanted search and filter functionality to complete this task, as opposed to being limited to using only the global navigation options. This feedback helped to build out the alternate routes and iterative filtering shown in the user journey.

D. Wireframes

I created wireframes for four key pages from the user journey to show what navigation, content, and functionality are available on the site and how they will be organized. I conducted usability testing to determine how these users might navigate the site, setting tasks focused on finding important content or functionality. While wireframes typically do not show styles, I chose to add minimal color, typographical styling and differentiation between artist profile pictures and images of artwork to help my participants visualize what the site might look like. I observed which elements were easily understood and which caused difficulties for users, making changes to labelling, faceted navigation, content prioritization, and other elements accordingly.

• 

  D1: Home

• 

  D2: Search Results

• 

  D3: Artists Near Me

• 

  D4: Artist Page

Annotating for Accessibility

The following annotations communicate accessibility requirements – regions, focus order, and ARIA roles, states, and properties – across teams. The Home page is used as an example.

Important Regions

Focus Order

Roles, States, and Properties

CREATED FOR INM401 Information Architecture (PRD1 A 2020/21) AS PART OF MSC HCID AT CITY, UNIVERSITY OF LONDON AND AWARDED A DISTINCTION.

More projects: