Category Archives: data collection

A Real Life Parable about Data and Hearing What You Want to Hear Regardless of What Has Been Said

A school district contracted with a research firm to conduct a telephone survey of local residents to determine if there was enough support for a parcel tax measure to move forward with it. The research firm was paid over $18,000 to conduct the telephone survey over a five day period. A total of 400 surveys/interviews were conducted and factored into the results. Fifty-five percent (55%) of those surveyed said they would support the measure, which falls short of the two-thirds required for the measure to pass, so the school board chose to abandon the measure at this time.

The survey also revealed that “only 14 percent of those surveyed think the district is doing a good job of providing high quality education or preparing students for a job,” and 60% of those surveyed believe that overall management of the district is poor.  Sixty-four percent (64%) believe that the district is doing a poor to only fair job of managing public funds.

Ouch.

Now, there are many things about this whole process that I could discuss, from the fact that $18,000 is an exorbitant fee to pay for a telephone survey of 400 residents (yes, many reputable research firms, including my own, would do an excellent job for much less) to the fact that the district had other no-cost and low-cost ways of getting pretty close to the same information, but I’m going to focus on the response to the survey results.

Just about anyone I have discussed this with says something like, “Wow. It’s pretty clear that folks in that town think the school district is doing a lousy job. The public doesn’t trust them with their money.”

Interestingly, though, that’s not what the school superintendent got out of those results. Here’s what the local newspaper had to say about that: “She was interested to learn that, based on the survey, the community most valued tutoring for students, curriculum that uses science and technology, and more opportunities for students to take advanced classes.”  And then the superintendent was quoted, “We need to continue to help kids that need extra help, continue to challenge kids that need more (rigor), and we need to do that with current technology.”

Huh?

While all of that may be true, it seems to me that the real message to get is that the community doesn’t trust the school district and thinks it’s doing a lousy job.  That’s what needs to be addressed.

We could debate the value of spending a lot of money on data gathering efforts.  As an evaluator, I’m a believer in investing in data collection to help you demonstrate the value of your programs and evaluate their effectiveness so you can improve them. The questions that comes up is always, “How much money is too much to spend for evaluation and data collection?”

But even that is not the moral to this story. 

The moral to this story is this:  If you’re going to spend anything on conducting a survey, be willing to really listen and hear what people are saying.  If you’re not going to learn from what has been said, even a dime is too much to pay for the information.

——————————

Click here for a free webinar on Tips for Conducting Focus Group Interviews.

Click here to access two free webinars on the Basics of Program Evaluation.

A Real Life Parable about Data and Hearing What You Want to Hear Regardless of What Has Been Said

A school district contracted with a research firm to conduct a telephone survey of local residents to determine if there was enough support for a parcel tax measure to move forward with it. The research firm was paid over $18,000 to conduct the telephone survey over a five day period. A total of 400 surveys/interviews were conducted and factored into the results. Fifty-five percent (55%) of those surveyed said they would support the measure, which falls short of the two-thirds required for the measure to pass, so the school board chose to abandon the measure at this time.

The survey also revealed that “only 14 percent of those surveyed think the district is doing a good job of providing high quality education or preparing students for a job,” and 60% of those surveyed believe that overall management of the district is poor.  Sixty-four percent (64%) believe that the district is doing a poor to only fair job of managing public funds.

Ouch.

Now, there are many things about this whole process that I could discuss, from the fact that $18,000 is an exorbitant fee to pay for a telephone survey of 400 residents (yes, many reputable research firms, including my own, would do an excellent job for much less) to the fact that the district had other no-cost and low-cost ways of getting pretty close to the same information, but I’m going to focus on the response to the survey results.

Just about anyone I have discussed this with says something like, “Wow. It’s pretty clear that folks in that town think the school district is doing a lousy job. The public doesn’t trust them with their money.”

Interestingly, though, that’s not what the school superintendent got out of those results. Here’s what the local newspaper had to say about that: “She was interested to learn that, based on the survey, the community most valued tutoring for students, curriculum that uses science and technology, and more opportunities for students to take advanced classes.”  And then the superintendent was quoted, “We need to continue to help kids that need extra help, continue to challenge kids that need more (rigor), and we need to do that with current technology.”

Huh?

While all of that may be true, it seems to me that the real message to get is that the community doesn’t trust the school district and thinks it’s doing a lousy job.  That’s what needs to be addressed.

We could debate the value of spending a lot of money on data gathering efforts.  As an evaluator, I’m a believer in investing in data collection to help you demonstrate the value of your programs and evaluate their effectiveness so you can improve them. The questions that comes up is always, “How much money is too much to spend for evaluation and data collection?”

But even that is not the moral to this story. 

The moral to this story is this:  If you’re going to spend anything on conducting a survey, be willing to really listen and hear what people are saying.  If you’re not going to learn from what has been said, even a dime is too much to pay for the information.

——————————

Click here for a free webinar on Tips for Conducting Focus Group Interviews.

Click here to access two free webinars on the Basics of Program Evaluation.

Published by Creative Resources & Research http://grantgoddess.com

The Relationship Between Evaluation and Grant Writing

While it’s true that evaluation and grant writing are completely separate disciplines, there is an important link between them. Yes, report writing and grant writing both involve writing, but the connection is even more important than that and, unfortunately, it is often overlooked.

The connection?  Data.

The data you gather to evaluate your programs is very valuable for demonstrating both your need for additional funding and your capacity to implement programs effectively.

Even if there is not a grant proposal on the horizon for you, you should prioritize your data collection and evaluation activities very highly. Then, when a grant opportunity comes up that is right for you, you’ll be ready.

When I teach grant writing workshops, I ask participants to imagine that they are grant makers.  You have $5.00 in your pocket to give to someone.  There are many people competing for your favor, and you are charged with a very difficult decision – who should get your $5.00?  You want to spend it well so it will really make a difference.  Everyone has a need, but some people have solid evaluation data to demonstrate not oly what they need, but to prove that when they implement a solution, it is successful in meeting the need they targeted. Wouldn’t you want your money to go to those programs that have powerful evidence of positive impact?

When you get the grant, the loop gets even stronger because you can use some of your grant funds to support evaluation activities, which help you build an even stronger case in future fund raising and grant writing.

The Relationship Between Evaluation and Grant Writing

While it’s true that evaluation and grant writing are completely separate disciplines, there is an important link between them. Yes, report writing and grant writing both involve writing, but the connection is even more important than that and, unfortunately, it is often overlooked.

The connection?  Data.

The data you gather to evaluate your programs is very valuable for demonstrating both your need for additional funding and your capacity to implement programs effectively.

Even if there is not a grant proposal on the horizon for you, you should prioritize your data collection and evaluation activities very highly. Then, when a grant opportunity comes up that is right for you, you’ll be ready.

When I teach grant writing workshops, I ask participants to imagine that they are grant makers.  You have $5.00 in your pocket to give to someone.  There are many people competing for your favor, and you are charged with a very difficult decision – who should get your $5.00?  You want to spend it well so it will really make a difference.  Everyone has a need, but some people have solid evaluation data to demonstrate not oly what they need, but to prove that when they implement a solution, it is successful in meeting the need they targeted. Wouldn’t you want your money to go to those programs that have powerful evidence of positive impact?

When you get the grant, the loop gets even stronger because you can use some of your grant funds to support evaluation activities, which help you build an even stronger case in future fund raising and grant writing.

Published by Creative Resources & Research http://grantgoddess.com