Qualitative Data in Nonprofit Program Evaluation: FAQ Guide
If your nonprofit has been running its community programs for a while, you’ve likely figured out how to measure their success. You might collect data on participation or have participants rate their experience on a scale of one to five. Or, you may use mission-specific metrics where applicable—for example, a nonprofit focused on children’s literacy could track students’ reading and writing test scores over time to see if their programs have made a difference.
However, some social impact outcomes aren’t effectively represented by hard numbers, or the numerical information you collect on your programs would benefit from additional context and depth. This is where qualitative data comes in!
In this guide, we’ll answer some frequently asked questions about qualitative data and its role in nonprofit program evaluation. Let’s get started by defining this key term.
What is qualitative data?
According to UpMetrics’ impact measurement guide, qualitative data “captures qualities, characteristics, experiences, and behaviors in a…descriptive or narrative form.” It can take a variety of forms, including:
Testimonials or interviews
Open-ended survey responses
Focus group discussions
Photos and videos
The opposite of qualitative data is quantitative data, which is data that can be expressed with a number or percentage.
How is qualitative data useful for program evaluation?
Although your nonprofit’s initial program evaluation strategies might focus on quantitative data, qualitative data complements the numbers to provide a more comprehensive view of your impact. Specifically, it helps you dive deeper into key dimensions of impact that inform your overall evaluation framework, including:
Who your programs serve
What services you deliver to program participants
The quality of your service delivery
How your programs have improved participants’ lives
The effects of community contributions on your programs
Returning to the children’s literacy nonprofit example, tracking program participation and test scores will give their team a general idea of who they serve and how they improve children’s academic performance. But if they also interviewed participants and their families, they might learn that their programs have sparked several children’s love of reading and helped them succeed in and out of school—in more ways than numbers alone can express.
What are some best practices for managing qualitative program data?
Some aspects of managing qualitative data are similar to how you would collect, store, and analyze quantitative data. However, thematically-oriented information requires a few additional considerations in these areas.
Here are some ways to ensure your qualitative data serves its purpose of enriching your understanding of program performance:
Communicate directly with stakeholders. Program participants are the most obvious stakeholders to collect qualitative data from, whether in writing or through conversations. Staff members and volunteers can also provide valuable insights from their interactions with participants during program activities, and participants’ friends and family may offer fresh perspectives on how your program has impacted their loved ones’ daily lives.
Review third-party data. Participants may not provide every piece of potentially useful qualitative data directly to your nonprofit. Social media posts, news stories, and online reviews on Google or Yelp are all publicly available sources of non-numerical information about your work. Regularly search your organization’s name and relevant hashtags to see what external sources say about your programs.
Practice good nonprofit data hygiene. Even data that doesn’t involve numbers needs to be organized, clear, and accurate for you to glean insights from it. Store all of your qualitative program data in one place and regularly audit it to identify and resolve issues with ambiguous, duplicate, inconsistent, misplaced, or missing information.
Look for patterns and trends. While each individual piece of qualitative data offers unique perspectives, there are still general conclusions you can draw from analyzing datasets. Once you’ve collected a lot of related data, perform thematic analysis, which involves codifying qualitative data to find commonalities that lead to actionable insights.
Take active steps to prevent bias. When your data is based on subjective qualities rather than objective facts, it’s easier to let bias accidentally slip in during collection and/or analysis. Ensure your qualitative data represents the diversity in your nonprofit’s community, and take a step back during evaluation to assess your own biases and determine if they’ve affected your views of the data.
If you need help implementing these best practices as you measure program impact, consider investing in an impact measurement platform. These tools allow you to store and analyze various types of qualitative and quantitative data, ensuring all information contributes to your overarching impact tracking strategy for your programs.
How should my nonprofit apply qualitative program data?
After you’ve analyzed your qualitative program data, it’s time to share your findings with the world! Depending on what audience you want to reach, some ways to report the information and the conclusions you’ve drawn from it include:
Marketing materials. When you promote your organization’s programs, including testimonials or media featuring current and past participants can encourage new signups as prospective participants learn firsthand about how your nonprofit would serve them. You may also inspire donors by including qualitative data in communications about fundraisers that support certain programs. Just make sure to obtain participants’ consent before including their names or photos alongside their stories!
Impact reports. For potential supporters who want to dive deeper, impact reports that feature both qualitative and quantitative data about your nonprofit’s programs will provide the information they need to feel confident contributing to your work. Consider creating more detailed reports for each individual program as well as mentioning some relevant data from your organization’s larger annual report.
Grant applications. Grantmakers see awarding funding as investing in a cause and a nonprofit they believe in. Including real-life stories and details about your programs in your proposal can make them see how they could make a difference by supporting you. Since grants usually have specific designations and application requirements, make sure to share only the most relevant qualitative data and format it to align closely with the grantmaker’s guidelines.
Strategic plans. Aly Sterling Philanthropy’s nonprofit strategic planning guide recommends interviewing and surveying various stakeholders as your leaders map out your organization’s future. This includes recipients of your services since they’ll likely have ideas about where you should focus your efforts to further your mission most effectively over the next three to 10 years.
Remember that reporting qualitative data isn’t enough—you also need to take action based on your results! Use the insights you’ve gleaned to make a concrete plan for improving your programs in both the short and long term. As you implement this plan, let participants know how their feedback contributed to your efforts so they know their voices have been heard.
Although quantitative data is still critical for determining whether your nonprofit’s programs have met their objectives, qualitative data allows you to fully understand how your work impacts your community. Use the tips above to develop a comprehensive strategy for collecting, analyzing, sharing, and acting on qualitative program data, and make sure you have the right software in your toolkit to simplify impact measurement and management.