#QuestionTheWeb Theory of Change

Argumentation as Discourse

#QuestionTheWeb builds our theory of change from an over arching lens of Argumentation as Discourse (Applebee & Langer, Beach & Newell, 2015; 2013, Hillocks, 1986). Teaching students the norms of academic writing remains one of the most consistent challenges for teachers of the English language arts. The earliest comprehensive studies of writing instruction at the high school level suggested that most students were taught highly formulaic structures for producing academic argument (Applebee & Langer, 2013, Hillocks, 1986), and that trend has remained consistent over time (Applebee and Langer, Newell et al., 2015). Teaching and learning academic argument, especially in middle and high school settings, often is reduced to formulaic essay structures. In fact, Newell et al suggests that we need to expand what counts in the teaching of argumentation

To clarify: what counts as argumentative writing, indeed what counts as argumentation more generally, is not a given. It is not something that just exists. It is instead a set of social practices deeply embedded in our everyday lives and the social institutions in which we all participate. It is socially constructed through and exists only through teaching and learning (Newell 1)

It is this set of socially constructed teaching practices related to argument writing that we aim to upend by providing students with a domain of their own to write, give and receive feedback, and critically evaluate outside web sources when creating their own arguments. From a theory of Argmentation as Discourse we will reach our stated outcomes through two phases that draw on empirical research in their design. In the first phase we focus on bridinging the cognitive and social practices of argumentation by focusing on the writer as an agent as We operationalize our theory of change through blogging. Then in the second phase we draw on calls to unite the domains of reading and writing to teach argumentation by having students interact with read alouds, baised avatars, and graphic organizers. Before we can begin to focus on reading and writing skills however we first address the social practices inheherent in argumentation.

Bridging Cognitive Skills and Social Practices of Argumentation

From Argumentation as Discourse we begin by recognizing the need to bridge cognitive and social practices in writing instruction. Reznitskaya & Anderson (2002), for example lay out methodologies and explain the reasoining for cognitive and social integreation in Argument Schema Theory. Furthermore, in a review of research on argumentative writing Newell et al (2015) called for an integration of the skills and social practices. We continue this effort by building our theory of change on self-efficacy, identity, dialogical discourse, and community

Self-Efficacy

We know that self-efficacy has a strong impact on learning (Bandura & Zimmerman, find year). Research has shown that student with low self-efficacy do not engage in metacognitive practices aroud their learning (Walker, 2003). At the same time metacognitive reflection is the corenerstone of much of the instruction and feedback in argumentative writing (Beach review).

If we want students to see writing as part of a universal search for knowledge we must improve self-efficacy skills. Students with low self-efficacy in writing will also focus more on extrinsic motivations such as grades rather than the overarching goal of improving writing (Linnenbrink & Pintrich, 2003).

Students also need to consider their self-efficacy with digital writing tools. We also know that belief in one's skills with digital tools can predict their performance in inquiry task (Leu et al.) We also know that some students approach digital writing with higher self efficacy than traditional paper geners taught in school (Beach)

Agency

A theory of change based on Argumentation as Discourse would recognize the role of the learner and suggest that increasing agency by controlling the topic, audience, and impact of writing will improve learning. Durst (2009) argues we see students express agency by resisiting curriculum and suggests instead we should professionalize and make their goals central. In fact Scardamailia (2000) argues for epistemic agency giving students control over all th range of knowledge building skills. Beyond resisiting the curriculum agency impacts writing ability. Sheehy (2003) found that when students sensed a lack of agency they did not invest effort in writing.

.

Teachers can encourage agency in the classroom. While Sheehy found a negative imapct of lack of agency Though as she worked with students to formalize the genre for specific audiences and this lead to a collaborative sense of agency and the exchange of strategies. Also there may also be benefits to using digital texts to increase agency of writers. O'Brien, Beach, and Scrhaber (2007) found Struggiling reads and writers find agency in digital texts especially is they express resistance to the curriculum. Finally Multiple studies find having students write for authentic purposes for real change improves their writing. In fact giving students critical questions lead to increased frequencies of claims in student writing (Nusbaum and Edwards, 2011). As educators channel learner agency they help to increase student identites as writers.

Identity

Identty and agency are deeply entertwined and once students feel they have ownserhsip over their writing educators can help students build on their identies as a writer. Any theory of change that begins with Argumentation as a Discourse must recognize the role identity plays in writing (Scardamalia, M., & Bereiter, 2006). By focusing on individuals rather than outcomes we hope to break from cycles of prorgammatic writing instruction. In fact Trainor (2009) noted that we must think about our framing when students build identities or they may construct in opposition to the classroom in response to spaces that provide no agency.

We also know that multimodal tools provide opportunites for students to try identity kits (Gee, 2009), or play with different discourses that signify membership.

Johnson (2007) demonstrated how among Hmong students noted they creatively played with dress and identity to navigate social norms. Similary Zoss, Smagorinsky, & O'Donnel & Allen (2008) found multimodal contruction of masks as an avenue for identity construction with adolscent boys. These opportunities to try on different identities are essential as Students learn need to learn to adopt different voices (Elbow, 2000)through classroom and teacher models. (Lee, 2007). As students develop different voices they can begin to engage in dialogical discourse.

Dialogical Discourse

An intervention based on a theory of argumentation as discourse would recognize the role of Bhaktin's notion of doubletalk (Newell et al). Double voicing refers to adopting different worldviews as a tool for building community. In writing research this may mean speaking to a audience, to other students in the class, or even to one's own past through written reflection (Newell et. al).

In research in argumentative writing attending to and adopting different perspectives has shown promise worth exploring in future research. Felton & Herko (2004) found that students speaking to their claims and to the claims of their peers by graphing arguments lead to greater counter claims in their writing. Lunsford (2002) found through double talk that definitions of key terms in graphic organizers or models of arguments, specifically Toulmin, are not settled and evolve with student writing.

We also recognize the dialogical role of double voice through the creation of avatars or temporal identities to better. Nystrand (2002) found students construct personas with strong beliefs when writing. When developing their own belief systems in argumentative writing researchers have found it takes multiple iterations of writing and single persuasive essays do not allow for the internal diaolgue of perspectivesm (van Eemeren & Grootendorst, 2004; Walton, 1998, 2007) as students simply adopt a double talk and write papers to school context rather to a suggested audience (Smidt, 2002).

Studies may have indicated online interactions and role play may increase the efficacy of dialogical disciurse in the classrroom (Newell et al). Beach and & Doerr-Stevens (2011) found that moving from online to real contexts allowed students to use double talk to change school policies. Online interactions have also shown to shift student arguments (Morgan & Beaumont, 2003) and while some studies have shown the face to face arugments to be richer the different conditions did not translate to difference in written essays (Joiner, Jones, & Doherty, 2008).

Others argue that we need reciprocal relationship with the student and teacher (Wallace & Ewald, 2000). Dialogical discourse can be encouraged through the use of feedback and reflective conversations (Gorzelsky, 2009). The use of feedback drives discourse. This in turn encourages peers to engage with each other and thus help to build a community of writers.

Community of Writers

We also know that community is essential to process writing (Applebee & Langer, 2013; Graham, Fitzgerald, Friedrich, Greene, Kim, & Booth Olson, 2016; Graham, & Perin, 2007; National Council of Teachers of English, 2016; Troia & Olinghouse, 2013). Moreover, in writing communities outside of school, community drives all learning (Winn, 2015). This community allowed the web to thrive. In fact Scardamalia & Bereiter, (2006) argue we should look to grow the knowledge of the community rather than the invidiual writer.

Atwell (1998) has long called for teachers to come out from behind their desk and join students as writers. This step starts to form a community that will allow students to see themselves as writers by "changing knowledge, skill, and discourse are part of a developing identity" (Lave & Wenger, 1991, p. 12). If fact numerous studies have shown the power of building a community of writers (Brokerick, 2015; Hall, 2015; Hull, Stornaiuolo, & Sahni, 2010; Wright & Mahiri, 2012) to bridge the cognitive skills with social practices.

Community fo writers has been essential in online spaces (O'Byrne, 2015). This is especially pronounced in English language learners (Yi, 2008) who turn to community of writing for fanfiction (Black, 2005), identity work (Gee, 2015), and connections to home cultures. Online communtiies have also been shown to increase writing by giving students a place to share (Kehus, 2000)

At the same time online spaces can create negative writing behaviors (Chandler-Olcott, 2014) and this requires a holsitic approach to creating community (Hagood, 2011). This requires a recentering of writing communities to consider online spaces (Grisham & Wolsey, 2005). By intentionally designing online spaces for writing communities educators can take advantage of affordances of technology while actively modeling civic engagement in these spaces. This in turn can improve student cogntive skills through the delivery of feedback.

Operationalizing Our Theory of Change in Ohae I

When given the opportunity to develop pieces of writing for their own blog, students will engage in the social practices necessary for argumentative writing. In the first phase, students will explore markers of credibility and consider how an author’s point of view impacts the textual choices if the writer. Then, in the second phase of the intervention, students will encounter biased read alouds and reflective blog posts written by avatar characters who interact with students.

We also believe we can embed and reinforce these social practices through the use interactive reading and writing platform. We cannot improve argumentative writing skills without focusing on sourcing skills (Birkenstein & Graff, 2018; Harris, 2006). The social reader that emerges from this project will allow students annotate sources for claims and evidence, review the claims of others, leave comments for their peers. This will help students see an argument not as a thesis or position but as a socially complex text that unfolds in real-time while being layered in history. Students encounter these texts across different networks with various amplification and authority (McVerry, 2015). By focusing on teaching of first telling the story of oneself we build on our theory of change of argumentation as adiscourse.

In the first phase of our intervention we will operationalize our theory of change by allowing expression of agency, the opportunity to reflect and create identities, relying on dialogical discourse, and building the self-efficacy of participants. This will mainly occur by students developing a personal website and blog. The learning activities included in phase one all draw from previous research drawn from our theory of change.

In the early days of the internet, youth designed their identities in spaces they controlled (boyd, 2014). Now the spaces of reading and writing are algorithmically driven, and the design choices severely limited for the user (Best, Manktelow, & Taylor, 2014). Machines often determine which photographs students see from friends. The news youth read is promoted by bots across social media. As youth engage in these reading and writing spaces, it is imperative that they know to #QuestionTheWeb (Barker, 2009; Leu, 2015). In phase two of the intervention we will build up dialogical discourse amoung students

Domain Of One's Own. In fact we hope to build student agency and efficacy through multiple steps in our intevention by utlizing dialogical discourse about one's own identity. Dialogical discourse is at the heart of our theory of change and we believe providing students with the ability for multimodal identity construction (Zoss et al, 2008). Allowing participants to first question their own voice will will prepare them to speak to claims first by peers and then by others (Felton & Herko, 2004).Given the need for youth to hone skills and dispositions to critically evaluate what they read and write in light of the effectiveness of writing communities and the rise of fake news, the proposed #QuestionTheWeb Instructional model begins by focusing on spaces where student consume information and construct meaning. It is essential for students to “manage and consider the implications of becoming the owner of their own story” (Groom & Lamb, 2014) to understand themselves as reader and as a writer, but also understanding how perspectives shape claims and evidence is a core critical literacy skill. In other words, it is much easier for students to understand how an author uses the affordances of digital tools to shape truths after they have experienced shaping their own.

Power of Student Blogging. We also believe teaching argumentation first through student blogging will create a sense of ownership leadign to greater agency while providing the time and space to build self-efficacy skills with technology. Work done in content areas with Inernet inquiry shows an initial drop in knoweldge acquistion (Leu technical report), and studies of online learnign find it often takes four weeks for students to adjust. By providing time to develop one's identity as a writer we hope to take advantage of growing self-efficacy

Critical Questions. After students build their personal websites they will then choose from a set of critical questions (Nusbaum and Edwards, 2011) around the use of digital technology and social media in their lives. As they delve into this questions students will learn to produce different types of multimedia compositions such as podcast, photos, videos, and infographs. This will allow us to teach critical multimodal composition skills (Newell et al, 2011) while also providing agency to those who might find refuge in digital texts (Beach & Meyers, 2008). As students enage with these questions they will learn to annotate texts for markers of credibility.

Dialogical Reflections.. While students develop efficacy and knowledge wwith new forms of textual communication they will also be publishing reflection posts explaining how they learn. Creating an internal reflective dialogue will encourage the double talk (Bakhtin, 1985) that students need when questioning different genres and perspectives. More importantly we provide students an opportunity to learn how to learn through reflection. This will also professionalize their writing around their goals to improve agency (Durst, 2009).

Choice in Inquiry. Finally in phase one students will have to choose a personal challenge around technology or digital media. This could be a period of not using social media, personal goal to write or shoot photos once a day or a project to examine body image online. By allowing students choice in their expression we hope students learn to see themsleves as a writer with increased self efficacy and agency.

Throughout the first intervention phase we will also strive, and consistently evaluate the space, both the classroom and online interactions, create a community of writers. We will take lessons from out of school literacy spaces (Winn, 2015) and ensure students have a space to explore their own identities. Specifically we will use a code of conduct, modeling, and video guides to create the community. We will take field notes on the participatory practices and seek to increase these goals through constant iteration in our formative design. .

<

Unifying Theories of Reading and Writing

In order to overcome the challenges of teaching argumentation we must not only combine cogntive skills and social practices. We must also address the theory from an interactive theory (Flowers, 1989; Newell et al., 2015) towards reading and writing. To this end the second phase of our intervention seeks  to use dialogical discourse to improve both critical evaluation skills while reading sources and argumentative writing as students undergo activities in the reading of websites and the creation of argumentative texts. Through an interactive approach to reading and writing we hope to see evidence of greater appropriation (Rogoff, 1995) of cogntive skills and social practives.

Reading and Writing Argumentative Texts

Cognitive approaches have identified many skills associated with argumentation yet students still struggle. Researchers often find readers ignore information that does not align to their perspective (Perkins et al., 1991) and skills rarely transfer (Newell et al, 2011). In terms of sourcing schools studentss rarely engage in critiquing author perspective or text (Brante & Strømsø, 2018). As they write students do not make critical judgements (Perie, Grigg, & Donahue, 2005) and lack of analytical Writing (Persky, Daane, and Jin, 2003). Our theory of change suggests Decontextual skill instruction does not effectivley teach argumentation.

Dialogial Interactions

.

Instead research promise has been found in the investigation of interactions through Cognitive Reasoning (Anderson et al, 2001; Jabadallah et al, 2011; Reznitskaya et al., 2007, 2009; Waggoner, Chinn, Yi, & Anderson, 1995) where the intersection of written models and student and teacher talk translate into rhetorical moves for investiagting texts or crafting claims. These textual moves are greatest when the teacher consitently asks for evidence and provides feedback for the use of evidence Jabadallah, 2011).

Therefore any intevention must focus on encouraging interactions that encourage and deliver feedback. This should foucs specifically on the evalutaing and adopting of perspectives. This has to occur in a learning environmentment full of interactions that create dialogues with text (Newell et al, 2015) and their authors.

Text Structure.

Interventions asking readers and writers to evaluate texts structure have a long shown promise. Ambruster, Anderson, and Ortega (1989) explored text frames with expository texts and demonstrated that attuning to structure improves comprehension. In agumentative writing this has usually involved the use of a graphic organizers (Brooks & Jeong, 2006; Easterday, Aleven, & Scheines, 2007; Nussbaun & Kardish, 2005).

Consitently these tools have been helpful in encouraging the use of exploring counter-arguments (Newell et al, 2015) which is an essential step in considering multiple perspectives. Graphic organizers also provide rich data sources for evaluation. Any intervention focused on teaching the reading and writing of argumentation should utilize these tools.

.

Read Alouds

Reading aloud in secondary schools has recently shown promise (Braun, 2010 Fisher & Iver, 2006; Ivey, 2003) includinging argumentative texts (Hillocks, 2010). Ivey (1999) noted taht struggiling readers in middle school espciallay benefit from read alouds and they enjoy performative nature of reading and writing. Thus any intevetion focuse on teaching argumentation with struggling readers should provide scaffolds for reading aloud and should encourage read alouds by the instructor.

Any intervention designed toteach argumnentation must differentiate for reading ability. Including teacher read alouds can help ensure the success of all students. These read alouds can be small group, teacher lead, or even mediated using technology.

Mentor Texts

Providing students with metnor text for students to read and analyze has moderate effects in imporving writing ability as students learn how to well written texts are crafted. (Caswell & Duke, 1998; Duke & Kays, 1998; Maloch, 2008). Knduson (1989) found exposure to models of expository texts to have the strongest effect (.25) of four different interventions. This same trend wiith youn readers is supported with adolescents Graham & Perin, 2007; Hillocks, 1986). Chambliss (1995) noted that teaching argumentation most focus on textual clues. Pytash et al., (2014) through self report intruments found mentor texts imrpoved students understanding of why author's write in economics.

The use of mentor texts in argumentation provides the chance for dialogical interactions and the teacher of reading and writing. Students not only identified elements of the craft but gained understanding of author perspective. Any intervention based on argumentation as discourse should widely deploy the use of mentor texts.

Modeling

Teachers have long used models in teaching of writing. Studies have found struggling readers and writer benefit from teachers modeling (Fisher & Frey, 2012). Schunk (1991) noted that writing models work best when learners perceive they are on their level Graham & Harris (1994) found teacher modeling improved writing. Braaksma et al (2002) confirmed that similarity between the mdel and learner skills lead to the greatest improvements when compared to exper models. Sawyer (1992) found that models work better with skilled students and studies of models and direct instruction with struggiling students find direct instruction will suffice in teaching writing strategies (Schultz, 1997).

Current research reveals modeling of reading argumentative tasks maybe more effective for struggliningreaders but models of of argumentative writing maybe more effective with skilled students. In our theory of chnage with an interaction approach to literacy we believe the modeling of reading and writing of text can not be seprated from each other nor direct instruction. It is also clear from the litrature that degrees of seperation of the quality of a model and a learners pereceived skill. Therefore any intervention should utilize models that are written at the student level while stressing important features such as signal words (Heibert, 2017) through direct instruction.

Online Texts

Much of the research in argumentation has started to study texts in online spaces. Studies examining differences between offline and online argumentation have found that there maybe some differences in face to face discussions but the writing products to not significantly vary in quality Joiner, Jones, & Doherty, 2008). In fact others have found the unique affordnaces of online discussions lead to greater dialogical interactions (Beach & Doerr-Stevens, 2009). Given that the majority of students now consume more online than offline texts (Fox & Grainnie) it is critical we teach argumenttion in online spcaes. Furthermore given the role of misinformtion we have a critical need as a nation to make sure students can evaluate online information.

Researchers have also taken advantage of computer aided tools to scaffold learning (Scheuer, Loll, Pinkwart, & McLaren, 2010). Students have read articles as text maps (Dwyer, Hogan, & Stewart, 2010) and found no difference than reading a full text. Numerous studies look at use of computer generated graphic organizers (Dowell, Tscholl, Gladisch, & Asgari-Targhi, 2009; Salminen, Marttunen, & Laurinen, 2010; Suthers, Vatrapu, Medina, Joseph, & Dwyer, 2008). In these studies computer aided scaffolds lead to increased number of arguments in writing but little is known if this transfers to other topics or situations. Any study utilizing online writing spaces should allow for the use of graphic organizers. Furthermore effort needs to be made to see how granphic organizers can increase dialogical interactions

While the use of models and grpahic organizers maps to similar results with offline text the role of authority shifts online (Newell et al., 2015) through the creation of believable personas or avatars (Bogost, 2007; Jamaludin, Chee, & Ho, 2007;Rourke, Anderson, Garrison, & Archer, 1999; Swan, 2002.) In their study of online role play to teach argumentative writing Beach & Doerr-Stevens (2009) noted that students who created avatars with more credible personas were seen as percieved as experts. Any study examining dialogical interactions in online spaces should attune learners to how markers of authority are amplified.

Operationalizing our Theory of Change in Phase II

Phase two of our intervention, based off of Kuhn and Cohen's (2011) study may lead to improvements in both critical evaluation skills and argumentative writing as we seek to integrate research of cognitive perspectives and social perspectives. (Newell, Beach, Smith, VanDerHeiede, 2011) while addressing the lack of research unifying reading and writing of argumentative texts.

In phase II students will first begin what Kuhn and Cohen labeled the pre-game by completing a series of challenges around the critical evaluation of websites. Researchers will create short ten minute puzzles and class challenges around identifying authors, evaluating authority, and evaluating bias. Students will also complete challenges around identifying claims, sources, and evidence verification.

During the "game component of the lesson (Kuhn & Cohen, 2011) participants will then be put into dyads or triads and enter a simulated web environment. Webpages will be recreated following the methods of Leu et al. (2018) in the development of the IES funded online reading comprehension assessment. There will be pages around consequential topics. Students will choose from the list. Each topic will consit of two perspectives. Each perspective of an issue will have four sources. There will be two websites representing each side of an issue and an informational source.

As they enter a source participants will also be greeted by a video avatar in the lower right hand corner. They will be able to toggle the character on and off. The character will have a background persona and will carry subtle biases. As they read the website the character will annotate the site from their bias. A representative from the Chamber of Commerce might call a "Carbon Tax" too expensive a solution to climate change and call into question the cost listed on a website from an environmental group. Each avatar will read two sources confirming their bias and one source they oppose. As the avatars discuss the text they will pay close attention to textual features such as signal words and rhetorical devices. Particpants will trace the text structure of each source using a graphic organizer. The groups will each publish a blog post explaining why each character is right and another explainign why they are wrong.

In the "end game" (Kuhn & Cohen, 2011) students will then choose a different topic from the list. Using the same interactive graphic organizers students will investigate the issue on the open web using the provided sources but without the avatar read alouds. They will then publish a multimedia essay on the topic.

.

While the use of the topics and sources previously used by their peers may introduce a prior knowledge bias the performance on this essay is not included in our final models. Further by limiting students to our simulated sources we reduce the need to include searching skills as a covariate. Finally this also fits our theory of change given that argumentation around a subject should unfold over time in discussion, reading, and writing.

Measuring our Theory of Change

We belive this theory of change will lead to outcomes of improved argumentative writing skills, critical evaluation skills, and self-efficacy beliefs in oneself as a writer. Argumentative writing is measured using the Analytic Writing Continuum, a validated assessment that is consitently scored reliably by trained raters. Critical Evaluation skills be masured using a previously validated and reliable forced response assessment the Critical Online Information Literacies Quiz. Finally belief in oneself as a writer will be measured using the self-efficacy of writing scales. Each of these measures is discussed in detail in the baseline assessment section.