Written by Chelsea Kelly, Milwaukee Art Museum

Over the past four years, I have worked with hundreds of Milwaukee-area teens who love art, and who, over their time in teen programs at the Milwaukee Art Museum, grow to love museums as well.

I have always had a sense that my students grow over their time at the Museum. This year, though, to really study that growth, we designed our longstanding Satellite High School Program as a year-long experience to explore exactly how weekly sessions at an art museum might change the thinking of our teen participants. To that end, our program outcome for students was that they would show an increased ability to reflect upon their own experiences and performance.

Evaluation is a grey area—much like teaching and interpretation—and we as educators need to use multiple methods in order to get a fuller picture of what’s going on with our students.

This means I’ve been thinking a lot about evaluation: How do we show change was made? Years ago, I thought evaluation was more or less a prickly, black-and-white, necessary evil that forced me to use altogether too much math. But over the past two years, I’ve come around to believe evaluation is completely the opposite (though math is still important!). Evaluation is a grey area—much like teaching and interpretation—and we as educators need to use multiple methods in order to get a fuller picture of what’s going on with our students. And further, these methods can be tools to help our teaching, improving programs and our impact on students.

In the end, I found I needed to use reflective practice myself to understand how my students were changing, and to explore and experiment with a number of different methods for articulating their growth. In this post, I’ll share a few of the methods we used in the Satellite High School Program this year to explore how our teen interns changed through reflective practice.

First… What is Satellite?

The Satellite High School Program is a year-long internship for sixteen teens ages 16 to 18 from diverse high schools all over the Milwaukee area. Once a week after school, they come together at the Museum and explore how art can be made relevant to our lives today. They participate in “object studies” (hour-long discussions on a single work of art), behind-the-scenes career talks with staff, and resume-writing workshops, and also mentor elementary school students in tours of the permanent collection.

Teens create a final project that has a real-world impact on the Museum. They choose a work of art in the Museum Collection, research it, and form their own interpretation of the piece. In past years, students have created responses in visual art, writing, or performance. This year, the students used iPads to create videos on their work of art, explaining what the work means to them and how it changed their thinking or art practice. You’ll see a few of those videos later in this post.

Core Evaluation

Let’s start with the core evaluation method we used for the program. We were lucky to work with one of our teen program funders, the Milwaukee Public Schools Partnership for the Arts & Humanities, and the University of Wisconsin-Milwaukee’s Center for Urban Initiatives and Research (CUIR) to develop the outcome above and to establish a tool to measure it.

We settled on one-on-one interviews, doing a “pre” interview on the first days of the program in October and a “post” interview on the final days of the program in May. Each student was privately asked the same set of questions in the pre- and post-interviews, meant to get at their ability to reflect on their experiences in the program. I scored each interview on a rubric that measured level of detail in their responses, and then we compared their pre-program score to their post-program score to see if they had improved.

At the end, every student did improve in their ability to reflect—their answers got significantly more detailed. As someone whose default is to be a more qualitative thinker, it was rewarding to use the rubric to see their interviews as data, in a quantitative, more tangible way.

But as helpful as this was, it’s still just one method of evaluation. Being able to explain in detail is certainly one aspect of successfully being able to reflect. But as I listened to their responses, and thought about what I had seen in the students over the course of the whole year, I realized there is much more to reflecting than just detail. Their responses used stronger vocabulary, they expressed sophisticated ideas, and they asked more and deeper questions. How could I articulate that kind of change?

Unexpected Data

Happily, along the way, we also found that we had collected some unexpected data which helped me more concretely see the change in my students.

Exit Slips

At the end of each session, teens used a web app on their iPads called Infuse Learning to fill out a quick exit slip survey. Exit slips are an easy way to take the pulse of your students at the end of a session. For Satellite, they answered the questions “What is something you learned today?” and “What are you still wondering about?” Though different from our interview questions, these certainly also support reflective practice by thinking back on the day’s session.

As the year went on, I noticed that the teens ‘ responses were growing more sophisticated: they were longer, they used more art vocabulary, and they realized that they might not be able to answer questions definitively, if at all. At the suggestion of Marianna Adams, who specializes in museum research and evaluation, I tried running these responses through two readability tests to see if that would quantify the sophistication of these responses. One test produces the sample’s Fog Scale Level, which measures syllable count and sentence length (a score of 5 being readable, 20 being very difficult). The other was for the Flesch-Kincaid Grade Level, which approximates the average grade level necessary to read and understand the text.

For the first question (“What is something you learned today?”), students’ scores jumped considerably in Fog Scale and Reading Level. Since these tests measure syllable count, sentence length, and grade level, this corroborates with what I found in the core evaluation.

But I was surprised to see that when I tested responses to the second question (“What are you still wondering about?”), students’ scores actually dropped! Yet if you read their responses, there is a drastic change—for the better.

Take Student D’s responses. In his early answer, he asks a relatively basic art historical question about distinguishing one type of art from another. In his later response, he is thinking deeply about the purpose of art and how we even decide what art is. And while Student F uses high-level art history vocabulary in her first response, it’s without context; later on, she’s thinking about how two seemingly opposite concepts may have something in common after all.

The scores of these comments may have decreased, but I’d argue that their reflective quality increased—the teens ask big questions that might not have an answer; they ditch high-level vocabulary to more informally muse on philosophical questions of art, destruction, and race. Running these responses through the tests helped me see, again, that while tools can be helpful, they’re ultimately just one tool—we need more than one to paint a bigger picture.

Videos

To round out that image, I’ll share one final unexpected evaluation tool: the teens’ final project videos as well as a talkback session they conducted at their video premiere.

For their final project, each student chose one work of art in the Museum Collection and looked at it, researched it, and talked about it with others for seven months. (Given that most visitors spend under 10 seconds looking at art in museum galleries, this is a feat in and of itself!) They distilled a school year’s worth of thinking into brief, 2-4 minute videos that answered what the work meant to them, what it had meant to others, and how their own thinking had changed as a result of looking at the piece—all questions with, of course, that familiar reflective bent.

The teens also participated in a talk-back/Q&A at the celebration where we premiered these final projects. Guests—museum staff, teachers, family, and friends—asked the group questions about their experience. If you like, you can watch the teens’ videos, along with the Q&A, in the YouTube playlist below.

Impact — Can Museums Change Teens?

So: does all the above—interviews, exit slips, readability tests, and final projects—add up to a full image of the impact that a year’s worth of reflective practice can have on students?

I’m not sure we can ever paint a full picture of student growth in intensive programs such as this one. I do think combining all of these tools can help, though—especially if the evaluative tools actively support the goal of the program. The interviews, exit slips, and activities were all intentionally structured to be reflective, related to the outcome itself. This relevancy was key, not only in genuinely evaluating the program’s success, but also in supporting the students’ abilities through the methods themselves. It’s also important that we educators make the program goal transparent to the students. The Satellite interns knew from the beginning that they were working on reflective ability—this helped prime them to think reflectively from the get-go.

As far as impact beyond reflective capacity, I also want to share a few quotes from the teens themselves about their time in this program:

“The videos help us think deeper about what we do—so even in school I think deeper about what I’m doing or why this was made or why this happened.”

“I learned that I shouldn’t judge a book by its cover. When I first saw my piece I just thought it was a bunch of different colors and didn’t really think about it actually having a meaning. But now I’ve learned that it actually has a super cool meaning behind [it], and I never would have learned about that meaning if I hadn’t taken the chance to explore. So I shouldn’t be so quick to judge.”

“We had to give tours and I found out that I really like to work with children and art at the same time. I would like to pursue a career in art education for elementary school students.”

“I was able to change and evolve my way of thinking, now being able to look past the obvious… I learned that art holds all the answers to any questions anyone may have, you just have to search for it.”

From the other evaluation tools, we saw that the students developed their ability to reflect on themselves and their own performance. But as seen in the comments above, they were also able to develop skills reflecting on the world beyond them—the world of art history, their future careers, how they interact with other people. All of these are ways of thinking that are valuable for their futures, as they go to college, discover their passions, and pursue meaningful career opportunities.

Can Teens Change Museums?

I’ve shown how this program helped these students grow in many ways. What about the Museum itself? Have these students had an impact on our institutional practice?

Institutions move at a slower pace than most programs, and if change and impact are complex to measure in sixteen individual students, then it’s multiplied tenfold for an organization that serves hundreds of thousands visitors a year. Even so, over the past few years, the work of teens in our programs has slowly but surely worked its way into the daily fabric of the Museum. Teens have interviewed artists on behalf of the institution. They have advised docents on ideas for giving tours to high schoolers. Their video projects will be part of on-site and online Collection Resources at the Museum, as well as our Archives, for all visitors to access while learning about works of art.

Ultimately, evaluation and impact are ongoing, a grey area that has a lot in common with the act of teaching itself. When done well and intentionally, evaluation doesn’t just show if we’ve met a goal. The tools we use to evaluate ideally become part of our teaching practice, because they reinforce the very abilities we are trying to help our students develop.

As for what I’m still wondering about? This year, our evaluation methods for the most part required the teens to have specific existing skills, such as writing for the exit slips or proficiency in using an iPad (though we did have video-making workshops as part of the program). I’m thinking about other ways to holistically gather data. For example, given that much of our evaluation methods emerged from teaching tools, should I document or film our discussions with works of art and find ways to analyze them? I’d love to hear any ideas or tools you’ve used to evaluate your programs, just as I hope this post has inspired you to take a fresh look at your teaching practice and find unexpected ways to see the growth in your participants.

*     *     *     *     *

ABOUT THE AUTHOR

CHELSEA EMELIE KELLY: Manager of Digital Learning at the Milwaukee Art Museum, where she develops educational technology initiatives and oversees and teaches teen programs. She is passionate about using gallery teaching and technology to foster relevancy for art museums in the 21st century. She has previously worked at the Frances Lehman Loeb Art Center, the Metropolitan Museum of Art, the Frick Art & Historical Center, and the Carnegie Museum of Art. Chelsea is a graduate of Vassar College and holds an M.S.Ed. in Leadership in Museum Education from the Bank Street College Graduate School of Education, where she was a Kress Foundation Fellow. She is also the founder and co-editor of The Art History Blog. Chelsea’s postings on this site are her own and don’t necessarily represent the Milwaukee Art Museum’s positions, strategies, or opinions.