Web Focus Groups: Know Your Users, Know Your Content

For most organizations, the go-to method of content analysis is web analytics. People love numbers: 2,000 inquiries, 500 sign-ups — oh, baby! Unfortunately, these numbers don’t tell the whole story, and if we base our content strategy on quantitative data alone, we’re missing a big piece of the content measurement puzzle.

Web analytics is great at answering "What?" What pages do people view on our website? What actions do they take? But it doesn’t adequately answer "Why?" Why do people view a webpage? Why do they take action — or not take action? And these are questions we need answers for if we’re going to make smart decisions about our content.

To understand what content our users care about and how our website can meet their needs, we have to ask them. All the web analytics insights in the world can’t replace a single user telling you why they came to your website and why they left. Focus groups help us find these answers.

Let’s talk about planning and conducting web focus groups for user research and how to make sense of your findings.

1. Define the Purpose

Just as with all forms of analysis, in order to find answers, you need to first define the questions. What is the purpose of your web focus group? Are you preparing for a full website redesign? Are you looking to update and improve your career services website? Are you trying to figure out how to best use social media to support your website goals? Before moving forward, clearly define your purpose.

2. Decide Who to Invite

Choose web focus group participants based on their direct knowledge of the content being assessed. For example, if you are assessing admissions content, try to find newly admitted students who recently experienced your website as part of the admissions process.

The more removed participants are from the subject matter, the less relevant their responses will be. In the previous example, you can use current students instead of newly admitted students (which is a common and practical alternative), but their opinion of your website will change significantly the moment they arrive on campus. It’s worth the effort to find fresh opinions, if possible.

To get started, create a list of web user attributes that will help you select appropriate participants. For a student focus group, you might consider age, gender, geographic location, area of study, on/off-campus housing, graduation date and so on. It’s nearly impossible to have the perfect mix of attributes in your focus group, so it’s important to prioritize them to make sure you get the best mix possible.

3. Choose a Size That Fits

I find eight to ten participants to be a good focus group size in most cases. You want the group to be small enough to encourage open discussion but big enough to provide adequately diverse opinions. It’s common for a participant to change his or her answer after hearing others talk — and that’s a good thing. Open dialogue is important to uncover insights that both you and focus group participants didn’t know to look for. (More on this later.)

4. Get People to Help

Food and gift cards! It’s hard to turn down a free pizza and an Amazon gift certificate.

However, what you really want are dedicated, interested participants who are motivated to help — not just any hungry student. Talk to knowledgeable staff or faculty for recommendations. Or, if you’re really stuck, you can always ask student staff to help while on the clock. But, yeah, food and gift cards will help seal the deal.

5. Decide What to Ask

When defining your questions, focus on the purpose of your web focus group and what you want to learn. The questions you ask will shape the discussion and affect the outcome, so take care in crafting them. Consider a funnel approach: Start with broad questions that will foster engagement, then offer more specific questions to guide participant responses.

Focus group questions are most successful when they are open and neutral.

Sample questions:

• What is your overall impression of the XYZ website?
• How often do you use the XYZ website?
• What is the primary reason you visit the XYZ website?
• What other reasons do you have for using the XYZ website?
• What types of content do you expect to find when accessing the XYZ website?
• Are you able to find the information you need on the XYZ website? If not, what is missing or difficult to find?
• What suggestions do you have for improving the XYZ website, including design, information and functionality?
• Describe a positive experience you’ve had with the XYZ website. What made it a positive experience?
• Describe a negative experience you’ve had with the XYZ website. What made it a negative experience?
• How do you search for content on the XYZ website? Do you use the navigation or the search box, or rely on email or other external links?
• What devices do you use for accessing the XYZ website? Desktop computer, tablet, smartphone?

Your list of questions should be much shorter. You will likely have time for only five or six questions in a one-hour meeting. Plus, you want to have the flexibility to be able to ask unplanned follow-up questions. Although you’re guiding the discussion, you don’t know where you’ll find all your answers — allow the conversion to move off course when appropriate.

Professor Glenn Blank of Lehigh University offers good advice on conducting a focus group, including tips for leading the discussion.

6. Evaluate Your Findings

One of the challenging aspects of focus groups as a form of content analysis is that participants’ answers leave a lot of room for interpretation. It can be hard to translate diverse opinions and discussion into actionable insights. This is why special care should be taken to document and articulate participants’ feedback.

In addition to evaluating what participants say, also consider how they say it. Are they enthusiastic? Are they frustrated? Are they indifferent? To capture these subtleties, use a voice recorder in addition to written notes to document the discussion.

Focus your findings

To help make sense of your findings, evaluate participant feedback within the context of your content goals and the purpose of your focus group. For example, if you’re evaluating your admissions email communications, what do participant responses tell you about the effectiveness of those communications? The questions you ask will uncover these insights from different angles. It’s your job to piece these findings together to make them meaningful and purposeful.

Identify themes

One of the tremendous benefits of user research is not just addressing known content problems but discovering unknown content problems and new opportunities. To uncover such insights, take note of themes that arise from the focus group. Were there comments that were unexpected or surprising? Were opinions contradictory to your assumptions? Zeroing in on these themes will help you paint an accurate picture of your content and improve questions for future focus groups.

Measure trends

In previous posts, I’ve written about how as your institutional goals and users’ needs change, so does your content strategy need to change. This begs the question, How do you know when your users’ needs change? With qualitative user research, we can answer this question just as we would with quantitative analysis: measure trends.

Conducting a focus group can’t be a one-time event. It has to be part of your content measurement plan. How do people respond differently to your questions over time? How do their content needs change over time?

For example, maybe students are looking for more than FAQs from your mobile website and are looking to conduct real business — like checking their student loan status or registering for a class. Maybe alumni are looking for more than a list of upcoming events and want to find and connect with fellow alumni online. You won’t always like the answers you find, but effective content requires adapting to changing user needs. We have to stay on top of these changes in order to maintain successful content.

What About Surveys for User Research?

Absolutely! User surveys are a valuable measurement tool in your content strategy toolbox. In particular, they are incredibly helpful at quantifying user research, making it easy to evaluate results. The downside is that you lose out on the valuable dialogue from focus groups that can uncover previously unidentified content problems and opportunities. However, depending on your user research goals, a web user survey might be the right choice.

Your Content Measurement Plan

Web user focus groups are only one piece of the content measurement pie. To gain an accurate understanding of your users’ content needs, you need a balance of qualitative and quantitative user research and content analysis methods. Along with focus groups, also consider usability testing, web analytics and web user surveys (as just discussed) to help you glean a more accurate picture.

When developing your content measurement plan, it’s important to consider how these research methods work together. Michael Powers, director of web services at Indiana University of Pennsylvania, offers some good advice on creating a comprehensive (and affordable) user research program.

How do you tackle web user research and content measurement at your institution? What methods have you found to be especially effective or problematic?