Site icon Kenneth Danford

How Do We Know If It Worked?

In our Another Way class for adults learning about Self-Directed Education, we are currently involved in a conversation about how we might know that our approaches “work.” Many of you know that I find this conversation frustrating. People want data: test scores, college admissions rates, some tangible evidence of academic learning. While we can produce bits of data along these lines, I find the conversation difficult for a more basic scientific reason: we have no ability to conduct a “controlled” experiment. The young people we work with at North Star and at other programs only have one life: they can’t both join our program and also attend regular school, and we have no formal way of knowing whether to attribute their young-adult outcomes to their learning approach.

This spring 21 teens (out of 61 members) have let us know they will not be returning to North Star in the fall, about a normal 33%-40% moving-on rate compared to other years. Here’s a quick summary:

Aging Out/”Graduating”: 7

Distance Members Resuming Local Activities: 8

Younger Teens Returning to School: 5

Moving Away: 1

For the last three categories, there is little to be analyzed. North Star offered these teens and families an alternative to school for a year (during the pandemic), and all of them feel grateful for the opportunity. They have fairly simple reasons for moving on. We could survey them, but nearly all of them would say that North Star “worked” for them. (12 would be 86%; 13 would be 93%)

Moving Away: the family is moving to England. This child had an older sibling at North Star, and the parents are loyalists to our program. Best wishes to them!

Distance Members: most have said “thank you for your help in this pandemic year.” In a couple of cases, we have connected them with local SDE programs. North Star shares these families’ preference for in-person community when possible.

Younger Teens Returning to School: these teens spent a single middle-school year with North Star and now will attend various private and public schools in the fall. Each of them has said, “If it doesn’t work out, you will hear from me again!” We will root for these young people to thrive in their new settings, and we won’t be shocked to get a phone call or a visit from any of them in the future.

With the “Aging Out/Graduating” category, we can explore a bit more deeply. These teens are ages 17 to 19 years old who have been with North Star several years. They have “grown up before our very eyes.” Five of these seven teens were active members for three to six years at North Star, and one of the others had an important two-year experience with us. Only one was

a pandemic-year-only member. All six of these multi-year members would say, emphatically, that North Star “worked” for them.

Of these seven teens, one is moving on directly to a four-year college. Five or six of the others are heading for more academic work, either at community college or other programs. (Several have already taken college classes while at North Star.) Two or three of these new alumni plan to be primarily working in the fall, and some have already held part-time jobs.

At first glance, having one out of seven teens moving on directly to four-year college may seem low in comparison to well-regarded local high schools. The reality is much more complicated.

Historically, relatively few teens have moved on directly from North Star to a four-year college, as our alumni often engage in community college, training programs, or pursue various interests when they move on. We need a longer window to see how North Star alumni grow through these activities in a “gap-year” way to the conventional school goal of attending four- year college. Also, North Star does not hold admissions to a four-year college as a primary first goal, and we do not push or aim for this outcome as the first step after our program the way that many conventional schools do.

Several teens in this “Aging-Out” group have faced serious challenges including physical health, mental health, and family instability. Their times at North Star have been full of conversations about how to make the best use of each year, and they have felt supported to consider a wide range of opportunities. From the North Star staff perspective, we “know” we have helped these young people grow up well. In fact, we are fairly certain each is better off than if they had stayed in school, but the point of this post is that I find this impossible to prove this conclusion in a scientific way.

From the family perspective, these teens (and their parents) have expressed gratitude that they have had two to six years to grow up outside of the conventional school setting. They will look back on these years as a positive era in their lives. They intend to maintain relationships with North Star staff and friends going forward. They each possess solid self-awareness, and see their short-term futures as quite open and full of options. Our final meetings were deeply moving. In short, these are mature and thoughtful young adults.

But, how do we know we all aren’t just fooling themselves, and that we haven’t been coddling and enabling them during their years at North Star? What would have been the result if these teens had stayed in school? We can’t know for sure. Most likely, they would have been shepherded by their parents and the school staff to graduate on time, though they would have had to overcome significant challenges to do so. For the most part, I believe they would have experienced more stress, and more messages of being “inadequate.” Some may have been directed into “special education”, “therapeutic programs”, or perhaps they would have dropped out and sought a GED and a first job on their own. It’s unlikely they would have developed friendships and felt part of a social community as they have done at North Star.

On the other hand, they would likely have spent more time on conventional academic subjects and skills and would have produced more academic output than they did at North Star. They would likely have a traditional high school diploma (rather than the HiSET) to show for themselves. Perhaps they would have met an inspirational teacher, or found a meaningful club or activity. They might have grown up in the system and matured at much the same pace as they did with North Star.

How can we know that one school approach or one learning approach “worked” for a particular young person? Does it mean personal growth during the high school years, social belonging, and self-awareness? Does it mean academic development and readiness for college or work? How do we know if high school graduates possess these qualities? Even more, how can we know whether to attribute these outcomes to what happened in school or outside of school?

Ah, Science! It is hard to “know” things. At North Star, we tend to focus on the present. How are teens doing with us right now? Are they defining interests, taking risks, following through? Do they feel they are living interesting and meaningful lives right now? Is being part of our community better than the most-likely alternative? If so, we celebrate and keep on going. If not, we celebrate and wish them well. For the most part, we see growth and satisfaction among our long-term members. These qualities tend to defy measurement, and therefore you will continue to mostly hear illustrative stories from me.

We wrestle with these issues every day, and we don’t have any magic way to be sure we are right. Neither do schools. My frustration lies with an impossible burden of proof placed on North Star while not doing likewise with conventional schooling.

I look forward to following the lives of these 21 new North Star alumni. They will continue to be part of our community, and their examples will continue to inspire me.

Featured Photo: The Shindig: North Star’s End-of-the-Year Party!

Exit mobile version