| |
Educators Gather at Stewardship Institute to Share
Ways of Assessing Place-Based Education
"My Neck of the Woods" is an example of the community mapping work occurring in schools working with the Roger Tory Peterson Institute of Jamestown, New York. A student has mapped a local neighborhood. How can one assess what was learned by researching, conceptualizing, and representing the local information embodied here?
|
About 90 rural educators and community representatives convened from April 3-5, 2001 for the Rural School and Community Trust's annual Stewardship Institute at Simpsonwood, a Methodist conference center in Norcross, Georgia. According to steward coordinator and midwest steward Barbara Poore, who organized the program, each Institute "focuses on a topic that has broad local relevance in rural communities yet is national in scope." With this in mind, the stewards decided it was time for the Institute to explore alternative assessment systems for evaluating what children learn when education, often "hands-on" in nature, draws upon local resources and traditions or reflects community interests and needs. State governments are increasingly using standardized tests to measure K-12 student achievement-- a student's graduation or a school's very survival can depend on such tests, often multiple-choice, even to the exclusion of all other factors. By the same token, those who attended the Institute were concerned that the most challenging educational experiences are not easily gauged by standardized tests. They hoped to discover ways of demonstrating academic achievement and the mastery of specific learning standards which would still allow individual schools in individual communities to develop their own projects and approaches to student learning.
The stewards of the Rural School and Community Trust are skilled facilitators who broker resources, network rural schools and teachers with each other, and help develop and sustain place-based learning projects. They planned and hosted the 2001 Stewardship Institute.
|
Before the release of several recent reports tying small schools to academic achievement, the small size of many rural schools was usually regarded as a barrier to good education. Advocates of subject certification for teachers look askance at how rural teachers, when staffing is limited, often teach more than one subject to heterogeneous groups of students with multiple grade and achievement levels. Some of the best rural schools, however, are transforming the former "limitations" of rural education into positive pedagogical assets. They take advantage of flexibility, intimacy, size, and a rural school's time-honored role as the center of community life. Though strapped for classroom resources, schools of this sort turn to what the community has to offer, transforming schoolwork into projects with real-life benefits for themselves and their neighbors. In this sort of "place-based" education, students work alongside local adults with skills to share and stories to tell. They may publish a community newspaper, restore an old theater, prepare a regional guide with GIS technology, wire a town and its businesses to the internet, monitor the local water supply, establish an oral history archive, open a village store, devise and conduct a survey, or collect the traditions of local elders.
Conference participants arrived on Tuesday at noon and set to work immediately sharing problems, work, ideas, and stories in a range of small clusters and large groups.
|
Although particular aspects of such projects frequently correspond to specified state standards of learning and participating students often do well on standardized tests, these achievements are tangential to what makes place-based learning attractive to rural communities and their children. Rather, enthusiasts sense it can empower the next generation of rural Americans to shape and direct their economic, ecological, and social future. Country people do not simply wish their children to be trained for jobs which will force them to move away. They would like to see them learn to seize new economic possibilities, to revitalize the local community, and to enrich the quality of life for all who choose to stay.
Sometimes particpants did a bit of unscheduled "hands-on" learning. Here, one teacher learns sign language from another for the song, "You Are My Sunshine.
|
Proponents are convinced place-based education can help accomplish these objectives, but they need to demonstrate place-based education's achievements on a student-by-student and school-by-school basis. How can they martial the evidence they need to refine their programs and to advocate for such an approach to learning amidst pressures to streamline the curriculum and teach to the test?
Teachers, school administrators, community representatives, designers of curricula, and academics at the Stewardship Institute listened intently to each other, seeking common ground and new ideas.
|
"Our people are very good at assessing the young," said Oscar Kawagley, a Yup'ik scholar who co-directs the Alaska Rural Systemic Initiative, an effort to combine principles and methods of western science with the empirical scientific traditions of native Alaskans. He compared the way Yup'ik elders observed their own young with certain protocols for classroom observation contained in each participant's registration packet. Kawagley set forth some traditional Yup'ik "standards of learning":
We want our children to grow up healthy and know how to survive. We want them to be in balance. When they are, we say they are 'human.' When they are only partly there, we say they are 'falsely human.' If they fall short or have it wrong, we say they are 'not human.' I'd say we have pretty rigorous standards.
Small interactive sessions were interspersed with plenary sessions in which individual experiences and insights were presented in lecture or panel format.
|
Kawagley was one of several plenary speakers who brought fresh perspectives and practical experience to the processes of assessment. Their presentations served as talking points for small working groups. Caucuses discussed specific aspects of the assessment process, referring to the examples of student work some attendees had brought along. In addition, individuals consulted at length about their specific dilemmas with two other educators in "consultancy triads." These highly structured conversations were punctuated by larger gatherings that guided participants through analytical concepts, rubrics and tools. As the conference progressed, participants learned to discriminate more precisely between objectives, skills, evidence, and evaluations. They built a working vocabulary that all could understand.
Empathy, experience, and candid analysis created a supportive atmosphere in which educators shared problems and suggested solutions.
|
If several participants arrived with initial feelings of resentment about mainstream processes of assessment, they soon acknowledged that they themselves assessed their projects and their students on a regular basis. Concern shifted from "whether or not to assess" to questions of "how," "by whom," and "for what purpose?"
"The most important criterion for any testing mechanism or procedure," said Barbara Cervone, Rural School and Community Trust's evaluation consultant, "is that it have validity and reliability." This point was further elaborated upon by Marnie Thompson, a learning research specialist with the Educational Testing Service:
Standardized multiple choice tests are very good at one part of their job: reliability. What that means is that similar students are apt to perform similarly, or that the same student is apt to perform the same way when such a test is taken more than once (provided there's no opportunity to learn between the two administrations of the test). It's like buying shoes of a certain size in a store; they are likely to fit the same sort of feet. It's great that most standardized multiple choice tests are reliable, because reliability is a necessary part of validity.
Doris Terry Williams, Director of Rural School and Community Trust's Capacity Building Program, reminds a small group to keep focused on the goal of "developing credible alternative assessment systems which go beyond the testing of basic skills." Kimball Sekaquaptewa, a research intern at the Santa Fe Indian School's Circles of Wisdom program, looks on.
|
"On the other hand," Thompson pointed out:
standardized multiple choice tests may be of questionable 'validity' in that they do not necessarily measure what the people who use them say they are measuring. Many states and school districts have detailed, lengthy standards for student learning that go way beyond what can be measured in a single multiple choice test. It's a serious validity problem, if the test is being held up as measuring all these things.
Thompson feels that many alternative assessments based on observations, student work, performances, and portfolios have the potential for considerable validity when assessing the sort of student work that is valued by the Rural Trust and that eludes easy quantification. "A student picture shows how a student paints and, in that sense, has validity as an assessment document" she said to illustrate her point. But she added a note of caution:
These sorts of assessments run the risk of being unreliable, and that can ultimately undermine validity and fairness. If parents get upset about the fairness of standardized tests, how would they feel if a state based its assessment on how well their sons and daughters expressed themselves through art! Without a significant amount of training and discussion and calibration, two different teachers or parents looking at the same student painting would easily arrive at two different conclusions about what the student knows and is able to do, and how fair is that?
Kelli Ramsey, who established a pathfinding oral history project at Oregon's Nestucca Valley Middle School, helps a small group categorize different aspects of the assessment process on a chart.
|
Thompson made the case that there is no truly "objective" measure for assessing student learning. Conversely, she pointed out that assessments based on portfolios or performances (often thought of as "qualitative") almost always result in some sort of grade or classification, perhaps in mutliple categories, which in turn makes them quantitative." I'd like us to reclaim every assessment tool we can lay our hands on," she concluded. "and use it to help make our alternative assessments more valid, reliable and fair."
Mel Friedman, an assistant principal and Director of California's South Monterey County Partnership, takes a few minutes to reflect and study on his own.
|
"I don't teach subjects or classes, I teach students," declared Keven Kroehler from the Minnesota New Country School in Henderson. This school, which blends inquiry-based learning, computer technology, family-school interaction, and community involvement, has just been awarded $4.3 million by the Bill and Melinda Gates Foundation to create 15 charter schools along the same lines. Although its 145 students study in one open classroom and work on student-initiated research projects, the school has a detailed and collaborative assessment system which tracks multiple aspects of a pupil's academic, civic, and organizational skills. Institute attendees poured over forms used by the Minnesota New Country School to evaluate skills ranging from "time usage" to "processing information." Even "appreciation for school" is an evaluation criterion, as is each student's ability to "represent the school effectively in the community."
Kroehler said the system is effective at showing students and teachers where a student is succeeding and where he or she needs improvement. Institutions of higher learning are starting to indicate some willingness to accept Minnesota New Country School's assessments -- its students are accepted into a wide range of solid colleges outside the state. It is harder for them to enter Minnesota's public colleges and universities, with their rigid requirements, but the school (which has a growing reputation for academic excellence) is establishing direct contact with in-state admission departments. It hopes to find ways of bending the system a little. In any case, Kroehler demonstrated how a school where "students determine what they learn and how they learn it" can devise useful ways of measuring their efforts.
Jacqueline Johnson, a teacher and assistant trainer for the Algebra Project in Charleston, South Carolina, is enthusiastic about how her own training with the Algebra Project transformed her ability to think conceptually and confidently about math.
|
In small group meetings, attendees explained the various components of place-based projects with which they had been involved. Several of them brought student work to share. Though teachers were usually enthusiastic about what they were doing, they readily sought help for areas in which they would like to improve. Some, for instance, wanted broader community input; others sought to transform their project from an extracurricular activity into something more central to the school's academic work. On the conference's last day, facilitators sketched charts that other group members filled in as a sample assessment of one another's work.
Barbara Poore, Midwest Steward and Steward Coordinator, planned and orchestrated much of the agenda for the Stewardship Institute.
|
While trying to extract meaning from the photos and explanations at hand, a range of questions arose. What, for instance, does a student mural imply about student learning? How is it documented? What does a photograph of the mural convey? How else can teachers and students document the process of creating the mural to reveal more about what is being learned by the students? What is the community's role, and what credence can one give to community testimonials? Is media coverage a form of evidence or an evaluation in its own right? Is the mural project embedded into other academic disciplines -- science, history, geometry, chemistry, computer graphics, civics? And the students, did they organize the opening reception? Did they study other mural traditions? Did they work with community artists?
After hours of sitting and taking notes, Stewardship Institute participants were ready for an evening dance. Here, they try a little line dancing.
|
Some students may do little more than color shapes traced out by a professional muralist -- yet such an undertaking could be a valuable apprenticeship in which skill and craftsmanship are perceived at close range. The mural's professional competence could make it a welcome addition to some public space. Another mural might involve students at every stage of its execution and reflect research into the local ecology. Nevertheless the latter product might be significant only to those within the school facility, adorning a classroom rarely seen by the public at large. How does assessment and evaluation measure the community components of either project? What about a school-based mural that involved the janitors, the parents, and the local senior citizens but has nothing to do with local subject matter, research, or traditions?
Soon people were square dancing to recorded Appalachian fiddle tunes.
|
Participants at the Institute discovered that each project has its own objectives and has to be seen in terms of its particular goals. A school mural reveals only partial evidence of how and why it was created; a portfolio that documents such artifacts rarely provides sufficient evidence without additional, contextual information. In the small groups, participants agreed that they needed to work harder at preserving and documenting several pieces of evidence about student projects, semester after semester, year after year. A few teachers said they kept journals of these projects, and others said they asked their students to do the same.
Mac Legerton (from REAP and the Center for Community Action in Lumberton, North Carolina) was the Stewardship Institute's unofficial evening activities director. He taught dances, called squares, and led a song circle. Here, he can be seen, second from the left, keeping the spirit going in the clear light of day.
|
One recurring puzzle is how to assess student accomplishment when it transgresses certain standards while achieving others. Ruthie Sayles, Professional Development Coordinator of the Algebra Project in Jackson, Mississippi, described a school drama production about gangs and violence in which students and teachers rehearsed and performed together. The teachers gained insights into the dangerous choices their students faced every day; the students started to trust adults they could level with about the violence and trouble in their lives. The play was a rousing success with the audience. More importantly, however, a student whose life was threatened at school actually approached his teachers for help. The adults were unsuccessful in interceding with the young man who was stalking him, but they gave it their best try. They worked so hard at helping the threatened student to stay calm and collected that, when the confrontation came, it was little more than a scuffle between boys. The threatened student had learned to respond with minimal defensive force to frightening, violent situations. The teachers were proud; everyone involved in the theatrical production was proud. The drama had achieved its community-building objectives -- the school just might become a safer place. Nevertheless, the threatened boy, who had used force only when other recourses were closed to him, was suspended because of the school's "zero tolerance" policy against violence. How can this student and his teachers convey what he achieved through this experience? How can others assess it? Surely the mention of a "suspension" on a school transcipt reveals very little of what he actually learned.
Ruthie Sayles, of the Algebra Project, told a stirring story of a school play with a profound impact on school life. She is sitting next to Rural Trust's Southwestern steward, Jose Cochado.
|
When Sayles finished her story, the listening group pointed out that the community (inside and outside the school) provided powerful assessments of its own -- applause, honor, forgiveness. The administration, however, applied a second, more inflexible evaluation in the face of complex circumstances. The student had fought on the school grounds and had broken certain rules.By suspending him, the school conveyed that he had failed to meet its articulated standards. Perhaps Sayles' narrative highlighted what conference particpants fear most about outside assessments: surrendering a parent's or teacher's obligation to temper judgement with mercy and to see each child as a unique individual capable of growth and redemption.
"There are loopholes," said David Ruff, Director of School Reform at the Southern Maine Partnership:
I believe place-based education can take place within the context of standardized testing. I believe place-based education can improve student performance on standardized tests. But if you have an insurmountable problem in your state, look for the loopholes!
Participants form part of a standing circle at the end of the Stewardship Institute. Native-American Steward Elaine Salinas studies her parting resolution on a note as others listen to Oscar Kawagley, second from the right. Kawagley wants to make sure that Yup'ik language immersion programs are not jeopardized by Alaskan tests which are written, for all grade levels, only in English.
|
By the end of the Stewardship Instiute, participants had found few easy answers to the challenges of assessing place-based student work. Nevertheless, they developed a deeper understanding of which assessment strategies might benefit their schools, their students, and their communities.
|
The final plenary session led to murmuring about exceptions to state mandated assessments. People from different geographic areas swapped ways rural schools have sidestepped standardized tests that do not reflect or accommodate their needs. Maine districts that had previously instituted local tests of demonstrated validity and reliability seem to be able to use these instead, according to Ruff. "There is a way for schools to apply for a waiver in Indiana" volunteered Barbara Poore. Pete Ellsworth of Laramie, Wyoming added that several communities rejected state mandates and banded together to develop their own tests. California schools with fewer than one hundred students can opt out of the tests, observed Walt Watson, a school superintendent from Nevada City, California. "We lose money, but it's worth it," he added. For Lydia Chapman, from the Selma-based Coalition of Alabamians Reforming Education (CARE), Alabama's standardized tests amounted to a means of resegregation. There, third-grade achievement tests are often used to separate students into remedial, standard, and advanced groups from which they rarely escape until high school's end. "We have some of the stiffest high stakes tests and some of the teachers least qualified and trained to prepare students for them, " she added. That is why her organization is organizing parents and mounting legal challenges to a form of assessment seen as unjust and operating against the best interests of children.
The youngest participant at the 2001 Stewardship Institute, Everett Joel Martin, seems confident of his future in rural America. He is the son of writer, researcher, and grants coordinator, Linda Martin, who handled the Institute's logistical arrangements from her office in Granby, Colorado.
|
Issues and options were as varied as the states to which the participants were soon to return. As they stood in a final circle, exchanging verbal commitments they had scribbled down on small pieces of paper, many people expressed a renewed sense of purpose. Even though no unified plan of action had been developed or even entertained as a possibility, they felt part of a strong network, diverse but supportive of each student, each school, and each rural community's integrity. "Whatever else you do," urged Donna Leonhard, a curriculum and staff development specialist from Turtle Lake, Wisconsin, "teach from your heart." There wasn't a soul in the room who disagreed.
by Elisabeth Higgins Null
Home | About Rural Trust | Get Involved | Publications | Links
Events | Services | Newsroom | Contact Us | Search
© 2003 The Rural School and Community Trust
| |
