Looking through the responses I got, I realized I made two false assumptions that led me to pose this question in the first place: One, that small charities do not have the capacity to evaluate in any way, and two, small charities inherently need more resources than large charities.
Saundra Schimmelpfennig, over at Good Intentions Are Not Enough and Charity Rater, made the point that it would be absolutely ridiculous if a non-profit did not evaluate their programs even on a minor scale (all of the following quotes are from email exchanges):
Even lacking the money to do a large scale assessment of their [the charity's] impact, they should still be regularly assessing their work. This should include regular chances for beneficiaries/recipients to rate the assistance they receive and provide suggestions for improving their work. This could be something as simple as a questionnaire sent to the parents at the end of an after school program or feedback from group discussions.Holden Karnofsky at GiveWell agreed that small-size does not necessarily mean charities aren't able to assess their projects. He pointed our that Village Reach is their top-rated charity but only has a budget of $1 million. He said that "[i]t's rare for any charity to have demonstrated impact, but I'm not sure size is the major factor there." So small and unproven are not synonymous. A large charity can be unproven, just as a small charity can be proven.
Schimmelpfennig also noted the importance of creating a needs assessment before expanding or creating programs to ensure there is no unnecessary duplication of work. This seems essential to any organization regardless of size and should be able to be done relatively low cost. The resulting assessment could serve as some proof that the organization's work is needed.
Karnofsky also called out my assumption that a donation matters more at a small charity. Some small charities are over funded, while some larger, established charities could use more funding for program expansion. He also made the distinction between a donation mattering more "to an organization" versus "to the people you're ultimately trying to help." Stop TB, another one of their top-rated charities, is a large organization, but he said your donation will go a long way in helping the communities the organization serves. This distinction isn't something I had thought of before. I think looking at how far a donation will go in a community is a better starting point rather than how far it will go in supporting an organization. (Although don't get sucked into being concerned about an organization's overhead.)
So, after analyzing these two assumptions, I realized that criteria for evaluating small non-profits is the same as for any non-profit: Look for efficiency, transparency and effectiveness. Organizations like GiveWell, GuideStar or Charity Navigator are great places to start evaluating, but they don't cover everything. To evaluate on your own, use things like the Charity Rater (which I erroneously stated was only for aid organizations in the previous post; it has now been updated and is applicable to all non-profits) or New Philanthropy Capital's "How I'd Analyse a Charity in Two Hours" (pointed out to me by Sean at Tactical Philanthropy). Also GiveWell's Giving 101 is a great guide.
Commentor Mark and Karnofsky at GiveWell both brought up the idea of venture capital and the different stages of growth for non-profits, which I will attempt to tackle in another post.