Big Impact with Big Data: Towards a New Research Design

This is part one of a two part series on my opening keynote of the Big Data Smart Technology Forum held at Tianjin University of Technology on October 13.

I come to you today with a challenge. We have a problem in educational research and I hope we do not recreate inequities of the past in our era of Big Data.

For example, An examination of results from the National Assessment of Educational Progress (NAEP), a common Big Data source, provides a bleak picture of educational progress in the United States.

First in the United States centuries of systemic racism have left massive scars on our great nation. As you can see your chances of born into poverty often have more to do with ethnicity and race than any other factor.

These disparities translate into the classroom. As you can see White and Asian students consistently outscore their peers on the NAEP Assessment. In each f the three years highlighted significant differences remain.

The United States Federal Government throws a lot of money at this problem. Last year the Department of Education spent over 15 billion dollars.

The Institute of Educational Sciences, alone invested over 600 million dollars in research.

Yet if you look at trends in NAEP scores they remain virtually flat. We see some gains in early reading scores but these fizzle in the upper grades and poverty and race still play a major role explaining variance in scores.

I ask you, “Why?” Many of you here today seek degrees in management statistics but you do not need a PhD to understand that 15 billion dollars and no change in progress is a bad Return on Investment.

We need a new paradigm for educational research and Big Data analytics overall. For too long our higher educational systems reinforces inequities and concentrates wealth amongst the elite rather than the people. We chase citations rather than helping communities.

I ask each of you in your respective fields to think back to how many studies over the last forty years have been truly influential. Research that has changed lives? I know for many of you you can probably count these studies on one hand. Again a bad ROI. So where does all this money go?

It doesn’t make it into the communities or our classrooms. Instead we create a false scarcity of intellectual capital. University professors, usually themselves from privileged backgrounds, apply for grants, train new PhD students, do research, and publish in journals that few folks will ever read. Our elitist economy runs on a currency of citation counts. The kicker… the public must pay exorbitant fees to publishers in order to read the research that their tax dollars already paid for. It is a double taxation cloistering money in the hands of the few while the many suffer.

I challenge you today to move our focus out of citations and into the community. In order to have a big impact with Big Data we need a new research paradigm. I turn to the field of community engaged scholarship which grew out of the nursing fields.

This methodological approach suggests we merge our research, teaching, and service into a common direction of helping the communities in which we live. Our research should focus on people not participants. They must be involved in the work and not simply a sample size. Significance should have as much to do with community impact as it does with p-values.


Yet in this era of Big Data I take community engaged scholarship a step further. Today I call us to an emerging field of digitally engaged scholarship. I define this as an interdisciplinary approach of designed based research using distributed talent and networked technologies to open source our knowledge creation for the greater good of both local and global communities.

Interdisciplinary Research

First digitally engaged scholarship must take an interdisciplinary approach. The problems the world faces are too big for one person, or even one nation to solve alone. Pollution, climate change, education. We must all work together.

We also need specialists when it comes to big data. If you try to master all of the fields necessary in Big Data you will be a master of none. In this room we have folks from public health, management, statistics, and public health. Let’s work together.

We will need front end and back end engineers to help scrape, collect, and and analyze data. We need management statistics teams well versed in Python, R, Hadoop and libraries that someone in this room may soon development.

You can’t do it alone.

Formative Design Based Research

We can trace our empirical designs and scientific inquiry back to Kantian humanity. Our commitment to objectivity has guided science for centuries. I say its time to embrace our subjectivity. Have clear goals rather than just questions. Louis Pasteur wasn’t just searching for answers when he developed methods to ensure food safety. He had a problem to fix. Do the same.

Digitally engaged scholarship requires the use of Design Based Research techniques. These methodologies draw on many names but I use the work of Reinking and Bradley in their understanding of Formative Design.

Overall the goal of research should not be fidelity of models but forkability to local contexts. We need interventions centered in the community that utilize inclusive methodologies that allow for iteration.


Distributed Knowledge

I also challenge you to rethink our definition of memory and cognition. In our Western traditions we have placed great emphasis on the self. Yet what if knowledge does not reside inside my brain. What if my memories are situated in the interstices between us, our environment, and our communities?

As the web explodes in size our external knowledge storage tools grow in vast size and complexities. Each of you in your pocket has more computing technology than humankind first took to the moon. As China prepares for the next moon landing I challenge us to rely on the networks that distribute knowledge across the globe.

Networked Technologies

It took the book 800 years to spread across the globe. Moveable print emerged in both China and Europe. Still in almost a millennia literacy reached only a fraction of the world’s population. In contrast the web has spread to a billion people in just under thirty years. A billion people, and in the next decade another billion will come online. No technology for reading and writing as spread with such speed.

We must take advantage of this opportunity while also protecting the way we read, write, and participate from emerging threats. Large multinational corporation suck up our data and sell it to the highest bidder. If we are not careful a new digital colonialism will emerge that will repeat the errors of our past. We must fight for a future in the world of Big Data where we empower people through privacy. You should control your data rather than handing it over to the Google’s and Facebooks of the world.

Improving our Communities

Let’s use Big Data to Build a better tomorrow not just focus on the bottom line. Like community engaged scholarship digitally engaged scholars serve the greater good. Let us use Big Data to not simply understand the past but to light a beacon on where we should head next.

Now let’s look at a few examples of where Big Data can be applied in educational settings and then you folks, as experts in your respective fields can help me help the world. Second half of the talk is here.

Last week I had the honor of accepting the Joan Finn Junior Faculty Research Fellowship Award. The award, is designed to support faculty with nine credits of release time for a research project. I plan on using the time to develop an idea I have dreamed up over the last few years.

My Plan

Over the long term I want to create an online environment to support the teaching of sourcing skills and argumentative writing. My thinking is we decontextualize the inherent bias and perspectives found in the act of reading and writing texts. I want to teach sourcing as a mindset and not a skill set.

Building off of the work of Rick Beach and the lessons I learned studying under Don Leu I want to use role play and bias thinkalouds to contextualize sourcing skills within Internet Inquiry.

Basically students would interact in this online simulation. They would have to visit different buildings in the town. Each building would have its own purpose. Users would encounter an avatar on each side of a contemporary issue. They would also visit a librarian with a more neutral stance. Finally there would be a store where students would have the option to visit. There they could unlock features to customize their avatars by completing learning events centered on sourcing Finally there would be “field work.” Here students would have to conduct online research and collect and analyze data.

Second Life NSF Model

The long term version of my idea is to develop learning activities that can bolster adolescent students’ abilities to use online sources in their argumentative writing. Using the Fellowship I hope to create the biased think aloud videos.

It would be the first step in massive instructional design process. Hopefully I can use the materials I develop and the results I find to successfully seek out external funding.

Why Formative Design

For this work I will draw heavily on Reinking and Bradley’s(2010) work on formative and design experiments. As a Neag Fellow with the New Literacies Research Lab I worked closely with Dr. Reinking on formative design and hope to bring the learning to bear on the project. Reinking and Bradley suggest:

  • Formative and design experiments are grounded in developing understanding by seeking to accomplish practical and useful educational goals.

  • They are focused on less-controlled, authentic environments instead of the tightly controlled laboratory-like settings.

  • They use and develop theory in the context of trying to engineer successful instructional interventions.  Thus, they dwell in the realm of engineering science rather than social science.

  • They entail innovative and speculative experimentation.

  • They are interdisciplinary employing multiple theoretical and methodological perspectives and orientations.

  • They seek understandings that accommodate many complex, interacting variables in diverse contexts.

  • They seek generalizations from multiple exemplars rather than from random samples and controlled experimentation.

Basically formative and design experiments are meant for real classroom research. I cannot develop my entire vision as part of this project I hope to just focus on the biased think aloud. It is an intervention, rooted in theory  that addresses my pedagogical goal ( a more developed post explaining this connection is forthcoming).

My Pedagogical Goal

I will use pre-recorded interactive read alouds that contextualize the bias and perspectives inherent in websites about science topics. In other words students will be given a video of a website that is read and annotated by a narrator with a specific bias. The perspective included in the read alouds will help to contextualize the sourcing skills required for argumentative writing. This lack of contextualization of sourcing skills has long plagued studies designed to improve argumentative writing in science (Guzetti, Snyder, Glass, & Gamas, 1993; Abell, 2007) and the critical evaluation of websites (Goldman et al, 2012).

Distributed Design

One of the greatest take aways I carry with me from my time under Dr. Leu is that issues we face today in educational research are too complex for the broken single research model. If I was to fully envision the role playing I want to create I would need to be part of a team of theorists, programmers, ethnographers, instructional designers, statistician, and multimedia specialists. Ohh and funding. Funding would help.

Until then (and the project will begin full force next spring) I want to invite folks on board. If you are an educational researcher and you are committed to working endlessly for no monetary reward on the hopes of improving connected classrooms I welcome you. The most critical needs of the project would be someone with a background in multimedia, science education, and someone knowledgeable in item response theory. Though enthusiasm for the project and an ability to learn in the open is all this team (currently me) requires.


Many important ideas have bounced around in my head, my writings, and my worlds during the reclaim open initiative.

For those who do not know reclaim open movement (learn more here) seeks to harness the internet so learning occurs as networked events across innovative spaces. When I look across all the conversations happening around reclaim open I coalesce around one common thread: Design Thinking.

Sure open learning involves innovative use of technologies, and  open learning requires a vision  that emphasizes knowledge as  good for the community and not a good to be commidified. Yet after  listening to leaders in the field  I have realized the  transformative power of open learning emerges from  distributed design thinking.

What is Design Thinking?

I am new to this line of inquiry but my nascent reading has pointed me to a history of design thinking that emerged in the fields of architecture and manufacturing. Over time cognitive scientists have (re)desgined many of the ideas. When I translate design thinking into education I borrow heavily from John Chris Jones who put an emphasis on the construct of time:

The main point of difference is that of timing. Both artists and scientists operate on the physical world as it exists in the present (whether it is real or symbolic), while mathematicians operate on abstract relationships that are independent of historical time. Designers, on the other hand, are forever bound to treat as real that which exists only in an imagined future and have to specify ways in which the foreseen thing can be made to exist.

Time. This draws me to the idea of education. In the past we have put so much emphasis on measuring and teaching the past. All this discussion of so-called 21 century skills has made me realize we need to start teaching and assessing for future learning. We need our students discovering solutions to  problems we have yet to realize by using technology that does not exiss. In essence we need to teach a generation of designers.

Design for Teaching

A quick google search as this idea coalesced for me revealed I am not the first one to the design think in education party. In fact the map below from IDEO highlights schools that have made design thinking integral to the curriculum.

Map of design thinking

Teaching as Engineering

My history with examining design in teaching (including stealing the idea for my website tagline) began with my work with David Reinking. Dr. Reinking’s work in literacy research has focused on formative design experiments.  As a fellow with Don Leu’s  New Literacies Research Lab at UCONN we worked closely with David Reinking’s team  to develop learning activities that taught students to read in a learning space that quickly shifted and required each individual reader to construct their own text.

Dr Reinking taught me that engineering, not medical science or agriculture (where many education research paradigms began), serves as the best metaphor for education. We have to start with a pedagogical goal and then identify factors that enhance or inhibit that pedagogical goal.

Teaching as Open

Design thinking not only supports open education but also requires a commitment to open education. I call this the engineering effect. The more we iterate as a community the more we will discover which will lead to even greater iterations and discoveries.

So many scholars  have committed to open learning and this pushes design thinking in education. You have the work of Henry Jenkins, danah boyd, Mimi Ito, Mitch Resnick, John Seely Brown, John Greeno, Howard Rheingold, and Daniel Hickey (to name a few in no particular order) that have helped shaped our belief in open learning.

Teachers have also committed to supporting design thinking by pursuing  open learning. My Twitter feed overflows with ideas from people such as Paul Allison, Kevin Hodgson, Jerry Blumengarten, Starr Sackstein, Ian O’Byrne, and Karen Fasimpaur. In all honestly I could never list all the teachers who regularly think, lead, fail, and learn in the open.

Design For Learning

I debated delineating design for teaching and design for learning. Teaching and learning are inextricably connected. I decided to make the distinction because we learn without teaching and too often teach without learning. I also think we need a consorted effort to design for future learning. This must resonate in our both our  instructional  and curriculum design practice.

Instructional Design

In order to create learning that support design thinking we will have to consider spaces that put an emphasis not so much on targets or skill sets but on the learners. We will need an instructional design that focus on civic problems, engagement, motivation, and of course learning. Creating events that support future learning will be critical. We have come a long way since Gagne.

Backward  Design

Similar efforts to support design thinking have also translated into classroom practice. I see evidence of this phenomenon in the rise of Wiggin’s ideas of backward design where curriculum tries to emphasize understanding over the skills and the content knowledge. There is much debate as to what we should emphasize skills or content knowledge. I believe we ask  the wrong the question. Instead we need to know what pathways get students to deeper learning.

Design for Meaning

Meaning making has always involved the remixing of others ideas. In fact the Greeks originally coined the idea of analysis, literally loosening up, and synthesis, or putting together. In other words “close reading” and “analytical writing” have always involved a bricolage of thinking. The Internet has amplified the speed and reach of these (re)designs of meaning and we see this in the texts we read, the stories we tell, and the languages that emerge.

Design and Text Assembly

As part of my dissertation I spent thousands of hours watching videos of students read online. One of the fundamental characteristics of successful students was engaging in what I called strategic text assembly. These students combined comprehension monitoring strategies and navigational skills to create and read texts that had never existed before. They chose reading pathways that helped them to overcome a lack of background knowledge. They seemed prepared for future learning.

Design and Multimodal Authorship

The signs and symbol systems we use to make meaning have shifted back to the visual, back to the spoken word. The dominance of text, while still the backbone of the Internet, has faded as more traditional non-verbocentric practices have reclaimed a strong position in meaning making. As researchers and educators we need to understand how meaning making when we engage in transmedia and transliteracies practices.

Design and Web Literacies

I am glad all the early talk of Web 2.0, the read/write web, and WYSIWYG editors has faded to the background. I am glad to see such a focus on teaching students computational thinking and coding. We will need cadres of learners who can speak in the language of the web.

Distributed Design Thinking

You do not need to know how to code to be a designer and not all coders,  engage in design thinking. The same reads true for educators. That is my other take away from the reclaim open movement. You can not do design thinking or open education alone.

The issues we face as researchers and educators are too big. The problems too complex. If students are to imagine a future and create things that yet to exist we need to encourage real-time problem solving and emergent literacies in collaborative teams. You can be the idea guy, the coder, the artistic designer. The role really does not matter. As long as students, and us in general, have a shared vision and are willing to reach that goal through multiple pathways of knowledge we will support design thinking in education.

As a teacher I have a fundamental goal of teaching writers that will some day teach writers. I try to do this by modeling what it means to write and making my process as open as possible.  I believed this when I taught in 6th grade and I believe it now at the college level.

That belief is not the only thing that does not change.

My students (I teach all writing intensive classes) still struggle with writing leads (or as I found out on Twitter last night ledes….for the journalism folks). They will also have to teach young writers on how to start a paper.

Library of Mini-Lessons

I decided to continue my mutlimodal writing mini-lesson series. We discussed on #engchat class night how we should work together to create library of mini-lessons that students would use in a blended environment.

Writing Leads 

Here is my next mini-lesson.

I tried to accomplish a few goals.

  • First I explicitly defined what goes into a good lead/lede. I settled on restating the problem and drawing in the audience.
  • I then looked at mentor texts. I selected examples from Medium. When I was taking screenshots I wished I would have written down author information so I could propoerly cite. Discovery on Medium is not great yet.
  • I then created a sock puppet mini-lesson to discuss the lesson (tutorial post coming in next few days).

Here is what is missing.

  • More guided practice. I have not figured out how to include this well in my mini-lesson. I am thinking I would need to to screencast the re-writing of a lede and then have students complete a Google Form or Doc on the re-written lead. They would then need to share and discuss their writing.
  • More indepenent practice. This would hopefully translate into student writing.

Writing Leads for Your Audience

Audience matters. At the secondary level you should know the discourse practices of your field. For example I looked at the last five issues of The Reading Teacher and Journal of Adolescent and Adult Literacy almost all of the articles start with a vignette.

When I compared that result to research journals only about  half the articles started with a vignette. In more technical journals authors often just state the problem. In research journals the first sentence and the first citation seem to carry more weight.


The best writing festers. A small idea takes hold somewhere in my world and begins to gnaw at my time. Then the scheme and design of my  thinking spreads into my  other daily needs. Normal routine gets enveloped by the desire  to write. As if  good ideas replicate like a virus. Once infected I cannot move on until I get these observations and thoughts out.

Getting those thoughts out in a manner others can, and want to, understand gets tricky.

My ideas for writing end up scattered on post it notes, files, docs, napkins etc. I often say I do my best writing when hiking or when I pace back in forth in my office.  Basically for me the writing process gets messy.

Recent postmortum on my writing notes. CC 3.0
Recent postmortum on my writing notes. CC 3.0

I have written in the past that my pre-writing process involves more papers than pixel. I have to sit down with a pen to plan my writing.

How I plan changes based on my medium. CC 3.0
How I plan changes based on my medium. CC 3.0


So I have set a new goal for the next month or so that I hope translates into my normal writing process. I want to  keep ideas in draft form, a writer’s notebook in essence baked into WordPress.

This should add an extra step, or at least allow me to focus on the most important in writing, drafting. I will also hopefully spend more time on the most critical phase of the writing process revising. Finally staying in draft form for longer should allow me to spend some time editing. This we know is often the last, and most important step, before publishing.

In other words if I add this intermittent step I may just use my pre-writing to focus on the single  most important step in writing: drafting, revising, and editing.

List of current posts in drafts. CC 3.0
List of current posts in drafts. CC 3.0

I have made improving my blog a personal and professional goal. I want to document my thinking about education while making the  writing process transparent  in the open.

By drafting before I publish I hope to let ideas fester. Some topics may fizzle others may flourish. Once I have a list of drafts I can work from the writing I already started. Finally I will  publish the pieces that matter…or get finished.

It is usually the same thing.



I have long worked in the field of e-portfolios. I recently found a CD-ROM with the first portolio assessment system I created for a district. It used Hyperstudio and we were creating an assessment system for Vo-AG. I then went on to working with Connecticut Technical High Schools and tried to use Mahara to develop their portfolio system.

My thinking has shifted. I believe in open education, if we want to assess student abilities for future learning, we need to look beyond portfolios. Now we need to think about presence assessment systems rather than portfolio assessment systems. We need to start thinking about teaching students to brand their online identity and measure the signals they share.

I have been thinking about exactly how could we encourage presence in open education. It is not about one spot, or one digital hub, Open learning crosses many networks. You might share a great update on Facebook, write a post on your blog, have an amazing Twitter exchange, or share wonderful photos on Google+ and Flickr.

How could a  student, in a class that values open ed curate this content? Enter Storify.


We have been using Storify to curate the #walkmyworld project and I realized we can utilize the tool to allow students to develop presence assessment systems. Imagine having different time points where students have to tell their story. Imagine having them add artifacts and reflecting on the significance these artifacts played in their own learning.

Then students could share and connect to each other regardless of their preferred network for open learning.

Just an idea but one I want to let percolate.

[relatedkingpro show=”4″ images=true width=”150″ height=”150″ placeholder=false]