Successes and Challenges of a Grant-funded Initiative Supporting Middle School Teachers’ Digital Literacy

This past school year, I acted as grant evaluator on a project to improve teacher quality by supporting seven middle school teachers in gaining digital literacy; the goal of the project was that the teachers be able to implement a digital writing project in their classrooms in the spring semester. Here I will share the methods of evaluating the project’s effectiveness and what we learned from carrying out this project.

I employed methods of formative and summative evaluation. Methods of formative evaluation were employed to ascertain progress toward meeting the project objectives and to identify possible areas of improvement for the workshop. They included four components:

1. Surveying participant-kept blogs that chronicled their process in the digital writing program and as they worked on digital writing projects in their classrooms.
2. Sending periodic emails to participants offering remote support and soliciting questions from participants.
3. Soliciting implementation plans from participants via email.
4. Face-to-face meetings with participants to discuss progress toward their implementation plans and to schedule the summative evaluation visits.

Evaluation 

Below, I discuss each facet of the formative evaluation in relationship to the project goals.

Blogs. The participants created blogs, which we linked using a wiki so that everyone could keep track of each others’ and her own progress. This was a means of creating community and of problem-solving using the collective intelligence of the group. It was also easy for me as the evaluator to keep track of the participants’ thinking and progress and to catch any issues early. In this way, the blogs would facilitate early intervention if things went awry.

The participants updated their blogs during the weeklong workshop, but then did not take them back up later. One participant did decide to start her own blog; she did this, she told me, in order to learn the digital skills she felt that she lacked so that she could “meet the students where they are.” This teacher’s blog chronicles her efforts to integrate technology in the classroom as well as her teaching in general.

The blogs helped to work toward the grant’s goal of developing digital literacy skills in the teachers. Though most of the participants used only linguistic text in their blogs, they did learn how to set up a blog, how to customize the blog, and how to post to the blog, which are all valuable 21st century digital writing skills. In a utilitarian sense, the blog also acted as a digital repository, archiving the learning and the participants’ thinking during the digital writing workshop. The blogs, therefore, provided a means of continuity.

Email and Remote Support. All of the participants were linked into a group using Google+, a social networking platform. All of the participants and facilitators could stay connected in one node on the Internet by this means. Introducing the participants to Google+ was, however, a good way to introduce social networking skills to the participants. However, it was not effective as a means of communication. I sent out a message to participants via Google+, but had only a single response using this means of remote support. The other facilitator’s messages via Google+ were similarly unilateral. Since this was the participants’ first time using Google+, it did not prove effective as a means of communication. As a means of communication, it was not reliable because very few, if any, participants were checking their Google+ accounts. The participants and evaluator and facilitators need to be linked using a communication platform that is familiar.

Email messages with participants were met with more responses than Google+ messages, but even these were spotty. Perhaps because the participants were already compensated for their participation, they did not feel any responsibility to continue the dialogue. In a future iteration of a grant program like this, I would recommend that there is incentive for participants to follow-up built in to the structure of the workshop.

Implementation Plans. We had structured the workshop in such a way that teachers created implementation plans remotely. Instead, I would suggest that the teachers should be required to create at least a rough implementation plan before leaving the workshop. Then, they should be asked to revise and resubmit a new implementation plan by the end of the first month back to school. This will ensure that the evaluator is able to carry out formative evaluations, and will make sure that the material from the summer workshop is carried over into planning for the school year. Any credit received such as graduate credit or Professional Learning Units should be tied to the reception of the finalized implementation plans to ensure that they are received.

Meeting with Participants. We set up meetings with participants at the College of Education’s Writing Project Spring Conference. The participants would be allowed free registration to the conference for their appearance in this meeting. This was a bribe of sorts to ensure follow-up. Six of the participants came to the meeting.

During this meeting, I asked participants to recount their progress and process to that point. Each teacher related a story of how her progress had been impeded by either a hardware or a software issue.  However, these participants compromised and figured out a way to carry out the project despite these setbacks, tweaking her original plan for implementation. For example, one participant could not download the needed software onto the school computers, and so she was using her personal laptop with her students for the project.

Although more slowly than anticipated, all of the participants were able to work out the obstacles they faced and were moving ahead with implementing the digital writing projects. I would recommend that in another iteration of this grant project, participants should be given a matrix listing a variety of digital storytelling tools with their coordinating operating system, price, download link, and affordances. That way, if they had issues with using MovieMaker, the participants have information on workarounds.

Two of the participants were hindered by constraints related to time; they were under pressure to cover material related to state and federal tests and were finding it difficult to schedule the digital storytelling project. One of the participants was having trouble carrying out the project while staying on the “same page” as her fellow grade level teachers. One of our facilitators had the idea to email the teachers the Common Core State Standards that the project would or could meet in case their principals question the use of time (pacing guides, etc). She also suggested that the facilitators should have had the principals sign off. This would have minimized the issues that the participants related during the formative evaluation.

I would recommend another meeting beyond the mid-year meeting that would occur after the participants had a chance to try their hands at implementation. During this meeting, they could talk about obstacles they encountered, how they dealt with glitches, and share ideas as well as student work. A chance to debrief, share their units, talk about the process, and share the final projects would help participants to solidify into a community of teacher-learners.

II. Summative Evaluation

Overview

The means towards a summative evaluation consisted of site visits. The site visits were opportunities for the evaluator to observe the participant(s) as they taught a lesson using the skills and knowledge from the workshop or to view student products.

Evaluation

In what follows, I provide a short description of the nature of the site visit and my summative evaluation of the project in relation to that site visit.

School 1

Three professionals from School 1 worked as a team with a fourth grade class, including a Technology Specialist, a Media Specialist, and a Classroom Teacher. The team completed two digital story projects with students. They first carried out a Saints project with students, and then carried out a unit on personal narratives culminating in digital storytelling.

The classroom teacher began the unit with personal narratives. They learned about the parts of personal narratives, like dialogue and details. They read mentor texts and identified the parts. Next, the students wrote personal narratives on topics of their choosing. These ranged from vacation stories, to outings like football games, to stories about pets, and even stories about extreme weather. The students then began bringing in their pictures and/or drawings to support the multimodal adaptation of the story. Using Movie Maker, the students began recording their stories.

I made my site visit on the day that the students viewed all of the digital stories.  Though they were all personal narratives, they were each very different. Some of the students drew their own pictures, while some brought in or found photos. Many brought in sound effects, such as waves crashing during a surfing story, and applause at the end of a story to signal closing. I was most impressed with the structure of the stories. The participants had obviously worked with the students to give their stories structure and organization not just through the linguistic text of the narrated story, but also through the digital effects such as transitions, repetition of particular images, and distinct openings and closings.

The participants at School 1 reported that they felt that the act of recording their voiceovers into their digital stories was something that benefited students in terms of reading fluency. They benefited from hearing themselves reading on the recordings. Reading and writing, therefore, were recursive processes for the students in the project.

As mentioned previously, the classroom teacher from this site took up blogging on her own outside of this project. In conversation, she told me that this was a direct result of her involvement with the summer digital writing workshop. The workshop was successful in inspiring ongoing inquiry around digital writing skills in at least one of the participants.

School 2

I visited the third grade class at School 2 to view the students’ digital storytelling projects and to talk to the students and teacher about the composing process. The teacher initially had problems early on with securing the necessary technology to carry out the project. In email correspondance she told me:

I have come into a few problems. My school does not have the technology that many schools have so I am still trying to find a microphone so the students can record their voices. I only have one computer in my classroom and have decided that I am going to make one longer movie where they will each get to voice part of the poems they have written. I hope this is acceptable. If they each need to do an individual movie, please let me know. 

I suggested that she purchase an inexpensive microphone at Radio Shack, as they have computer microphones for well under twenty dollars. Ultimately, though, she decided to use iMovie on her personal laptop computer for this project. She had 12 students all using this one laptop for the technology portion of the digital storytelling, but she was happy with the outcome because she was already familiar with iMovie and said it was, to her, “friendlier” and she didn’t want to have to learn a new platform with all of the existing constraints on her time.

Though she had toyed with the idea of creating a class digital story, this teacher had her students complete an individual poetry project for the digital storytelling project. The students wrote a poem as part of a senses lesson based on a color of their choosing. They then created a story map based on that color and the five senses, thinking of a sensory image that tied to each of the five senses related to their chosen color. Next, the students drew pictures of each sensory image and wrote a poem as well. The teacher made her own digital story based on a color poem first, and showed the students. Next, they made a class digital story, and finally, they completed their own. The teacher relied on templates available within iMovie to scaffold the digital storytelling. The students chose a template for their story that came with transitions, title slides, and backgrounds, and then they loaded their media into her computer. Finally, they dragged and dropped the pictures into the project. They carried out a bit of editing within iMovie, including adding titles.

The teacher at School 2 was able to seamlessly integrate the traditional writing and the digital writing facets of this project. The choice of the subject matter for this project (sensory language and color) was clever, as it complimented the digital technology and vice versa. Her students enjoyed the project immensely, and were proud to show their work, which was rich.

School 3

The classes at school three were sixth through eighth grade ESOL classes. The digital storytelling project was a way for the students at School 3, many of whom had only been in the U.S. for a few months, to use resources beyond language to tell their stories. This teacher’s students were not able to use Windows MovieMaker because of software issues with the school, but she used PowerPoint as an alternative. Her students first wrote their personal narratives, and then they went through the writing to find places where they could bring in visuals. Some students then brought in photos and scanned them, while others used the Internet to search for relevant images. They used the voiceover feature in PowerPoint to record and integrate their vocal retelling of the personal narrative. This element gave these ESOL students a chance to practice speaking and fluency skills, while also expressing themselves with means beyond the linguistic text with which they weren’t yet expert. And as the teacher at School 3 told me, the students were automatically learning vocabulary and pronunciation and how to interact in English by doing the project.

I asked the teacher about the possibility for repeated implementation of this project. She responded that “absolutely” she would do it again, if she were allowed to teach ESOL in this way. As funding got cut back, her responsibilities increased and the time to teach has been decreased to make time for an increased load of paperwork. In addition, the ESOL teachers are required to be in lockstep. Therefore, she would have to make sure the other teachers were also doing the project. She was not, however, limited by any state or federal standards, as the ESOL classes are not held to those standards. This gave her the initial freedom with these classes to carry out the project.

This teacher’s confidence and technology skills were evidently increased through her involvement with the workshop and particularly through the process of implementation. She noted the reciprocal teaching that happened when the students showed her and each other how to record voiceovers, insert transitions, scan and import photographs, and other skills. This is important for teachers of digital writing to accept that they are no longer the resident experts and that it is beneficial for the whole class to recruit the strengths of each member of the learning community.

School 4

I visited the teacher at School 4’s 8th grade class as they began the digital writing portion of their more long-term project on poetry writing. At the beginning of the year, the students wrote “Where I’m From” poems, and the teacher had planned to have them remediate their poems, turning them into digital stories, in October. However, this was the year that the school rolled out new CCGPS, a new set of standards, and this impeded her progress toward the digital storytelling facet of the unit.

The teacher began the lesson by showing the students a sample digital story about President Obama’s inauguration in 2008 called “The America We Elected.” The piece was comprised of the author’s original black and white photography with a voiceover describing the energy of that day. A simple and straightforward digital text, it was a solid choice as a model for the students. The teacher asked the students for feedback in a general way, prompting them to tell her what they noticed. The students had specific and thoughtful comments, particularly in terms of the structure of the sample story.

Next, the teacher had her students take out the linguistic texts of their poems, and gave them fifteen minutes to work out the timing and the editing. Students were paired up, with one student from each pair acting as timer. They were timing the vocal performance of their poems, as the goal was to create a one to one-and-a-half minute digital story. Therefore, they may have had to add to their poems, or to tweak their vocal performance. The teacher prompted them to remember a lesson on prosody, and brought in that oft-neglected modality of aurality.

The students worked the majority of the class period to remediate their poems, imagining how they might translate into digital media. As they did so, the teacher circulated the room, aiding the students in revising, adding to, and tweaking their poems. She offered feedback like, “Be more descriptive. Instead of saying, ‘I’m from state government’ say ‘I’m from politics.’”

The next step for the students includes going home and looking through photographs and interviewing parents to jog their memories. Using the photography and parent input as resources affords students an expanded palette from which to develop their poems. The teacher closed the class by having the students think about where the photographs would come in in conjunction with their narrated poem. “Put an x by the part of the stanza where you would want to bring in the photograph,” she told them. She offered the students storyboards, if desired, to map out the photographs that way. I thought this flexible approach toward digital writing process demonstrated her understanding of the recursive nature of the digital writing process; some students will want to meticulously plan using print media, while others will likely do much of the composition within the program itself.

This teacher maintains that she will implement the digital storytelling again, and next time she would start with having students bring in photographs. The photographs would help them to develop content. The photos act as a sort of auxiliary memory, capturing what escapes the students as they reflect on their youth.

The teacher at School 4 admitted that her confidence level with writing with digital media was not high. She told the students that she was “illiterate” and said, “I had a difficult time, so I don’t want you to have a difficult time.” In conversation, she told me that she was working to make sure that her lack of confidence with the technology didn’t hinder her students’ learning; she would often bring in the technology even if she didn’t feel 100% confident, so that they were still afforded the experience. The teacher relied on the students, many of whom had already done some form of digital storytelling (she polled them), creating a real learning community where students were called upon as resident experts. Her confidence level will grow after implementing the project and becoming comfortable with the shift from teacher and class to community of practice. This teacher already demonstrated willingness to relinquish control in this way as she urged students to bring their own devices and try other programs such as iMovie if they wished.

School 5

The teacher at School 5 had fifth grade students who used Windows Movie Maker to create digital stories on animal and plant cells. I observed on the day that the students presented their projects, which they had worked on over five weeks. The students, working with two technology specialists, digitally drew pictures of animal and plant cells, inserted title slides, text labels, transitions, and recorded narrations explaining the parts of the animal and plant cells. The students also included rolling credits.

As we viewed the students’ projects, the teacher asked the students to critically analyze the projects, listing strengths and weaknesses of the projects as a set. While the students were afforded time to edit and revise the stories with the technology specialist, this was an additional opportunity for the students to revise the stories as a group. The students offered comments about the pacing of the movies and the level of the volume in particular.

This project did not use the personal narrative as basis, but instead was based on science content area material. The students were learning to use the technology and were learning science vocabulary at the same time. The teacher explained that the students developed the content for their digital stories in her classroom, and then the students put the stories together with the two technology specialists in the technology specialists’ classroom.

I was particularly impressed with the choice to include the rolling credits. While the students did not rely on any outside source material as they created their projects and went so far as to draw their own illustrations rather than use images from the web, citing sources is an important new media skill. As we move into the Internet age where remix, remediation, and collective intelligence are the lingua franca, knowing how and why to cite sources is an imperative digital writing skill for students.

This teacher asserted that she will do the project again next year, and she seemed to be confident with her ability to carry out the project with the help of the two technology specialists.

Summative Recommendations

Overall, the teachers’ confidence levels demonstrated growth, but were not at the ideal level. One participant recounted that the sound in her digital story made in the summer workshop was off, and she was never taught how to fix it. I recommend that participants leave the workshop with high feelings of self-efficacy before being asked to implement the project in their own classroom. A Likert scale designed to measure affective factors such as confidence might be used to measure these feelings of self-efficacy. Alternatively, exit interviews could also be used to this end.

The participants all related that the accountability built into this project was beneficial; however, the big window of time between implementation and the workshop caused many to lose the skills. I recommend that the implementation be required during the fall semester so that the teachers can immediately apply the skills from the summer workshop.