Category Archives: evaluation

Assessing Results: Are You a Quant or a Qualit?

In this post, Non-profit Consulant Derek Link offers his thoughts on balanced assessment and evaluation:

In the world of social entrepreneurship the use of metrics for assessment of results has sparked an ongoing debate. The lines have been drawn between mathematically inclined folks who like to measure things using quantitative data (called Quants) and those who want to describe the social impact of programs using primarily qualitative data (called Qualits).

I would refer to myself as a hybrid, a Quali-quant. For me, the argument about which type of data is better is meaningless unless the right questions are being asked. Once you know what you want to know; in other words, once you know what will best demonstrate that your mission is accomplished, the kind of data needed to measure that reveals itself.

And the type of data is usually not one to the exclusion of the other. Typically a result is explained best by viewing it through data binoculars, not through a data telescope. I use the example of a child who comes to school on test day. The Quant will want to examine the child’s test score to see whether he has achieved to an expected level, whether he has raised his achievement from previous test administrations, how he compares to his peers, and how his test scores aggregated reflect on the teacher’s ability and the school’s curriculum and instructional program.

The Qualits, on the other hand, will want to modify the interpretation of the test score with qualitative information. Perhaps the child arrives hungry because the family was late getting up and she never had breakfast. Perhaps the child is sick or was up all night due to family violence. These qualitative factors impact the ability of the child to score well but are difficult or impossible to quantify.

In the end, I believe it is a disservice to the process/program/organization to have an imbalanced approach to assessment of results. Start off by asking the right questions.

—————-

For more resources to help you with the evaluation of your programs, read some of the articles on our FREE Evaluation Resources page or view some of our free  recorded webinars on program evaluation. For an even higher level of support, become a member of GrantGoddess.com.

Published by Creative Resources & Research http://grantgoddess.com

The Relationship Between Evaluation and Grant Writing

While it’s true that evaluation and grant writing are completely separate disciplines, there is an important link between them. Yes, report writing and grant writing both involve writing, but the connection is even more important than that and, unfortunately, it is often overlooked.

The connection?  Data.

The data you gather to evaluate your programs is very valuable for demonstrating both your need for additional funding and your capacity to implement programs effectively.

Even if there is not a grant proposal on the horizon for you, you should prioritize your data collection and evaluation activities very highly. Then, when a grant opportunity comes up that is right for you, you’ll be ready.

When I teach grant writing workshops, I ask participants to imagine that they are grant makers.  You have $5.00 in your pocket to give to someone.  There are many people competing for your favor, and you are charged with a very difficult decision – who should get your $5.00?  You want to spend it well so it will really make a difference.  Everyone has a need, but some people have solid evaluation data to demonstrate not oly what they need, but to prove that when they implement a solution, it is successful in meeting the need they targeted. Wouldn’t you want your money to go to those programs that have powerful evidence of positive impact?

When you get the grant, the loop gets even stronger because you can use some of your grant funds to support evaluation activities, which help you build an even stronger case in future fund raising and grant writing.

Published by Creative Resources & Research http://grantgoddess.com

The Relationship Between Evaluation and Grant Writing

While it’s true that evaluation and grant writing are completely separate disciplines, there is an important link between them. Yes, report writing and grant writing both involve writing, but the connection is even more important than that and, unfortunately, it is often overlooked.

The connection?  Data.

The data you gather to evaluate your programs is very valuable for demonstrating both your need for additional funding and your capacity to implement programs effectively.

Even if there is not a grant proposal on the horizon for you, you should prioritize your data collection and evaluation activities very highly. Then, when a grant opportunity comes up that is right for you, you’ll be ready.

When I teach grant writing workshops, I ask participants to imagine that they are grant makers.  You have $5.00 in your pocket to give to someone.  There are many people competing for your favor, and you are charged with a very difficult decision – who should get your $5.00?  You want to spend it well so it will really make a difference.  Everyone has a need, but some people have solid evaluation data to demonstrate not oly what they need, but to prove that when they implement a solution, it is successful in meeting the need they targeted. Wouldn’t you want your money to go to those programs that have powerful evidence of positive impact?

When you get the grant, the loop gets even stronger because you can use some of your grant funds to support evaluation activities, which help you build an even stronger case in future fund raising and grant writing.

Using Social Math to Help Your Data Tell Your Story

I’ll admit it.  I’m the kind of person who gets goosebumps of excitement over a well-formatted data table. Pages of charts and graphs make me feel like something important has been communicated. But most people aren’t like me.  In fact, pages of charts and graphs make eyes start to roll back into most peoples’ heads.

This isn’t just an evaluator’s problem. Whether you are a school administrator or a non-profit executive director, if you have a story to tell, you need to find a compelling way to tell it. And you need to speak in a language that the people you want to hear your story will understand.

Social math can help you do that. Social math is a way of presenting data that connects it to something people can easily understand and links it to a broad social purpose.

I was at an evaluation meeting for the past couple of days, and one of the best sessions gave some great examples of social math that I’d like to share with you (thanks to Adrienne Dealy of the Communication and Social Marketing Center for these examples):

You could say, “Each year, over 91,000 infants under 1 year old are victims of child maltreatment.” That sounds like a big number, but is it?  What does it mean? Using social math approach, you would add, “If their cribs were placed end-to-end, they would stretch for 78 miles.” 

Wow. Now I get it.

Here are some more examples:

“Before passing legislation to regulate gun sales, there were as many gun stores in California as Burger King restaurants.”

“If every person in the U.S. were to change their page margins from the default 1.25″ to .75″, we would save a forest around the size of Rhode Island each year.”

“A gasoline refinery emits 6 tons of pollutants per day – enough to fill 25 balloons of toxic pollution for each child in a mid-size town.”

When you’re sharing your data, whether it is data documenting your need for assistance or data documenting your success, think about how you can make it meaningful for your intended audience. Remember, it’s about them, not you.

Now go change your page margins and save a few trees……

Published by Creative Resources & Research http://grantgoddess.com

Using Social Math to Help Your Data Tell Your Story

I’ll admit it.  I’m the kind of person who gets goosebumps of excitement over a well-formatted data table. Pages of charts and graphs make me feel like something important has been communicated. But most people aren’t like me.  In fact, pages of charts and graphs make eyes start to roll back into most peoples’ heads.

This isn’t just an evaluator’s problem. Whether you are a school administrator or a non-profit executive director, if you have a story to tell, you need to find a compelling way to tell it. And you need to speak in a language that the people you want to hear your story will understand.

Social math can help you do that. Social math is a way of presenting data that connects it to something people can easily understand and links it to a broad social purpose.

I was at an evaluation meeting for the past couple of days, and one of the best sessions gave some great examples of social math that I’d like to share with you (thanks to Adrienne Dealy of the Communication and Social Marketing Center for these examples):

You could say, “Each year, over 91,000 infants under 1 year old are victims of child maltreatment.” That sounds like a big number, but is it?  What does it mean? Using social math approach, you would add, “If their cribs were placed end-to-end, they would stretch for 78 miles.” 

Wow. Now I get it.

Here are some more examples:

“Before passing legislation to regulate gun sales, there were as many gun stores in California as Burger King restaurants.”

“If every person in the U.S. were to change their page margins from the default 1.25″ to .75″, we would save a forest around the size of Rhode Island each year.”

“A gasoline refinery emits 6 tons of pollutants per day – enough to fill 25 balloons of toxic pollution for each child in a mid-size town.”

When you’re sharing your data, whether it is data documenting your need for assistance or data documenting your success, think about how you can make it meaningful for your intended audience. Remember, it’s about them, not you.

Now go change your page margins and save a few trees……

A Few Thoughts About Performance Reports

For those of you new to the grant world, grant recipients are usually required to submit some sort of performance report to the funding source to give information about how the grant is being used and how successful the grantee has been in meeting goals and objectives. These performance reports are due at least annually. Some programs require them twice a year. Others require them quarterly.

Here are some of my issues with performance reports:

1) They are usually due at a strange time of year, before most outcome data are available. I understand that funders want to use the information to make decisions about continuation funding for the coming year, but how useful is that when most of the data are missing because they are not yet available? The effect it has on grantees is very stressful. They worry that they won’t be refunded because they don’t have data, but everyone knew the formal outcome data wouldn’t be available until later. My favorite example of this is standardized test scores for schools. Students take the tests in the spring. Results are available in the summer. Grant performance reports are due in the spring. Grantees are forced to fill out all sorts of forms that mean nothing because all they say (in many different ways, in several boxes) is that the data are not yet available. Then they have to fill them out again and submit them as “updates” in the fall once they have the data. Isn’t there a better way to do this?

2) Everyone acts like the reports are unexpected, when they aren’t. As an evaluator, I have many grantees who don’t want to make time to meet with me to discuss evaluation throughout the year, but they panic and want me to drop everything when they get a notice that their report is due in a few weeks. If you keep up with your evaluation as you go, the report is no big deal. If you wait until the report is due to talk about how you’re doing, there will be panic and stress. There is a better way.

3) Funders keep the report guidelines and due dates a secret. OK, maybe not a total secret, but I find it very annoying that at least a due date can’t be published months in advance. Would it hurt anybody to let people have some planning time? Is there a good reason why the deadline date has to be withheld until 3 weeks before the report is due? If so, please share the reason with all of us. If not, please stop it.

4) Report formats are becoming more and restrictive. It used to be that a grantee could report their outcomes and progress following a basic narrative template. Nowadays, more and more funders are requesting restrictive data sheets and limiting the amount of narrative a grantee can provide. I understand the need to force people to report both quantitative and qualitative data – and sometimes a form is the best way to do that. I understand that the people reviewing the reports want to minimize the amount of time they need to spend reviewing the reports. I also understand that reducing paperwork requirements on grantees is, in general, a very good thing. However, in many cases, the minimized reporting formats actually prevent the grantee from fully explaining what they have done, why, and what the outcomes were. How is that a good thing?

Alright, that’s enough whining from me (for now). Back to my reports……

A Few Thoughts About Performance Reports

For those of you new to the grant world, grant recipients are usually required to submit some sort of performance report to the funding source to give information about how the grant is being used and how successful the grantee has been in meeting goals and objectives. These performance reports are due at least annually. Some programs require them twice a year. Others require them quarterly.

Here are some of my issues with performance reports:

1) They are usually due at a strange time of year, before most outcome data are available. I understand that funders want to use the information to make decisions about continuation funding for the coming year, but how useful is that when most of the data are missing because they are not yet available? The effect it has on grantees is very stressful. They worry that they won’t be refunded because they don’t have data, but everyone knew the formal outcome data wouldn’t be available until later. My favorite example of this is standardized test scores for schools. Students take the tests in the spring. Results are available in the summer. Grant performance reports are due in the spring. Grantees are forced to fill out all sorts of forms that mean nothing because all they say (in many different ways, in several boxes) is that the data are not yet available. Then they have to fill them out again and submit them as “updates” in the fall once they have the data. Isn’t there a better way to do this?

2) Everyone acts like the reports are unexpected, when they aren’t. As an evaluator, I have many grantees who don’t want to make time to meet with me to discuss evaluation throughout the year, but they panic and want me to drop everything when they get a notice that their report is due in a few weeks. If you keep up with your evaluation as you go, the report is no big deal. If you wait until the report is due to talk about how you’re doing, there will be panic and stress. There is a better way.

3) Funders keep the report guidelines and due dates a secret. OK, maybe not a total secret, but I find it very annoying that at least a due date can’t be published months in advance. Would it hurt anybody to let people have some planning time? Is there a good reason why the deadline date has to be withheld until 3 weeks before the report is due? If so, please share the reason with all of us. If not, please stop it.

4) Report formats are becoming more and restrictive. It used to be that a grantee could report their outcomes and progress following a basic narrative template. Nowadays, more and more funders are requesting restrictive data sheets and limiting the amount of narrative a grantee can provide. I understand the need to force people to report both quantitative and qualitative data – and sometimes a form is the best way to do that. I understand that the people reviewing the reports want to minimize the amount of time they need to spend reviewing the reports. I also understand that reducing paperwork requirements on grantees is, in general, a very good thing. However, in many cases, the minimized reporting formats actually prevent the grantee from fully explaining what they have done, why, and what the outcomes were. How is that a good thing?

Alright, that’s enough whining from me (for now). Back to my reports……

Published by Creative Resources & Research http://grantgoddess.com