In the recent webinar, Goals and Metrics: Gathering Evidence of Success, members of the NSF INCLUDES National Network shared the innovative ways they have engaged project stakeholders to think critically about what is needed to build a data system that can be used to document the process of collaboration and track project outcomes.
This Network Spotlight highlights both of the presenters, Karen Peterman, who was the evaluator of the NSF INCLUDES DDLP, Earth Connections, and now leads the backbone organization for the SEAS Islands Alliance and Sarah Hug, evaluator of Computing Alliance of Hispanic-Serving Institutions or CAHSI.
During the webinar, Sarah and Karen discussed strategies for determining goals and metrics and offered excellent resources and ideas for others undergoing a similar process. Continue reading below to learn more about how they onboard partners and ensure all stakeholders come to common understandings about the work of their projects.
***
Contributed by Karen Peterman, SEAS Islands Alliance
Creating a common language
Setting a common agenda includes a lot of words that can be interpreted by people in different ways, particularly if you have people coming at this work from different disciplines. We used a chalk talk activity (a silent activity used with groups to generate ideas) to unpack what we thought some key terms meant, particularly what we thought these words meant in the context of our new project. What do we mean when we say underrepresented minority students, what do we mean when we say sense of belonging?
The idea is that once we agree on what these words mean, this will become our shared language. We will know that when we say underrepresented minority, this is what we are talking about, or when we say sense of belonging, this is what we are thinking about and what we mean when we use that language. Once we have these, we will be able to think clearly about the goals of our program and how those are carried over into relevant programming and metrics that we can use in relation to measuring those goals.
Data to tell a story
We use Salesforce to track our program data. While Salesforce is a customer relationship management system, they have a nonprofit package and an educational package that allow you to track relationships between people; this is one of our primary interests. Even though this is the bean counting, we have been thinking about this as a storytelling and story gathering kind of exercise for our group. The relational nature of the Salesforce system allows us to connect one story, such as about students and their profile, to profiles of other people in the system that are part of their story; that could include mentors, programs that they have participated in, peers that they have mentored themselves, and also their family members who have participated in Alliance projects.
***
Contributed by Sarah Hug, CAHSI Alliance
Setting the tone for collective impact
Each of the CAHSI regions (West, Southwest, Southeast, and North) held at least one on-boarding meeting in year one of the grant. Members who represent industry, four-year, two-year, nonprofit and graduate education interests in computing attended the meeting. During on-boarding meetings, data exploration is used as a tool to discover areas of growth. Specifically, graduation and enrollment data by demographic data disaggregated by gender and race or ethnicity. This examination of data within the group is meant to help develop trust among the group. By sharing our dirty laundry with one another we can build trust. It sets the tone for collective impact in which data is used to formulate problem statements and figure out how to solve them together. Data helps guide understanding of differences and similarities across context. The members share common struggles and develop strategic action.
Ensuring everyone has the same vision
CAHSI’s vision is that by 2030, Hispanics will represent 20% or more of those who earn credentials in computing, where credentials are defined as degrees and certifications that lead to gainful employment and advancement in the field. It is important to CAHSI that all partners are in support of this vision. CAHSI has made social justice values and beliefs a clear emphasis during on-boarding meetings, within its documentation and even in its web presence. Each on-boarding, there is explicit messaging around social justice, inclusion and the need for inclusive practices to come along with diversity.
Paying attention to how data is used
Our focus in evaluation is to understand how members of the CAHSI network are using data to inform strategic decisions in primary activities. We are interested in asking faculty and partners if and how they are using data when they are thinking about how to make great change across their region or across their institution. We are also looking as we are observing their meeting, how they are thinking about sharing data with each other, and how they are really embedding that data use into their development of activities that are happening on the ground.***
Thank you to Sarah Hug and Karen Peterman for sharing their work. Connect with Sarah and Karen through the member directory so that as the NSF INCLUDES National Network, we can continue to learn and work together.