A Semester-long EDR Reflection

When our Educational Design Research course began a mere 15 weeks ago, I struggled to grasp both the formatting of EDR articles and the components that comprised the methodology. Though the purpose of EDR as an application of theory to real world settings was clear, the types of cycles, phases, and theoretical underpinnings mystified me. After reading numerous EDR studies and undergoing the difficult challenge of preparing my own EDR proposal, I am left with a deeper understanding of the method. That being said, I still have plenty to learn about the complexities of design in EDR.

An EDR Dissertation?

When this semester began I could not foresee any way of using EDR for my dissertation. At first glance it seemed too lengthy and cumbersome a method to undertake as a culminating doctoral program study. Many EDR studies take place over the course of many years – a time frame no doctoral student hopes for in the dissertation process.

Throughout my tenure in the Ed Tech doctoral program, I have written numerous research proposals meant to take place theoretical settings. Having been an educational consultant, instructional coach, and professor, I have a preference for intervention based, qualitative research. Each time I wrote a proposal, I would find myself worrying about how I would make these studies happen when the time came to conduct research on my own. Who would I contact to participate? If not nearby, how would I get to them?

Though it seems frowned upon, I have always wanted to study my local area – my students, my courses, and my designs. I wanted to be both the researcher and the practitioner. After all, many educators act as defacto researchers in their classrooms, continually adjusting lessons and assessments based on student need. Who wouldn’t want to use that work to develop relevant, publishable research? Before EDR, I didn’t think such a study would be possible.

I am intrigued by the possibility of studying an intervention in a local context through successful completion. I see this as one of the most relevant contributions to research an educational technologist can make to the field. At this point, I’m stuck as to whether I want to pursue EDR for my dissertation. I’m not ruling it out but I do think that I would have to develop a more thorough understanding of study design before proceeding. I worry about my tendency to get lost in ideas – an issue that could prove disastrous when paired with the large scale of many EDR studies.

In order to make such a decision,  I would have to explore EDR further. Even after writing a proposal, I know I have deficiencies in understanding how to craft a theoretical  foundation from the literature (especially when the literature lacks empirical studies) and in the detailed design of the micro cycles. Though I was able to develop these in my proposal, I believe I would need even more development to implement the study successfully.

Thoughts on Peer Review

This semester, each of us conducted a total of four peer reviews of the EDR proposals generated this semester. I felt that learned more from this activity than any other with the exception of drafting mo]y own proposal. Writing an EDR proposal is no easy feat due to the complexities and detail of the study design. At times, I struggled to maintain clarity while conveying detailed information.  Being able to see how others organized and developed their proposals helped me tremendously. Receiving feedback from my peers about areas needed clarity helped to guide my writing and revisions.

Advertisements

EDR Design, Evaluation, & Implementation

In this module, we have spent significant time considering the implementation phases of design based research and how it relates to our literature reviews. Given that this is the third doctoral level methods course I have taken, I expected this portion of the experience to be easy. Of course, as has happened so many times with DBR, I found it more challenging than past study design projects.

I began planning my DBR implementation with the end (evaluation & reflection) and beginning (analysis & exploration) in mind to make the process a bit easier. Since integrating media literacy into my college history courses is something I already wanted to do, I knew which questions I’d have to answer before proceeding. Determining where students are with their digital media skills and faculty beliefs and knowledge of media literacy was an easy starting place. The literature on both faculty and students indicates a lack of knowledge and comfort that would have to be identified and addressed before progressing. The end wasn’t too difficult either – I knew that any study I conducted on media literacy integration would require the development of interventions and a determination of how effective they were to influencing student media literacy skills. Ultimately evaluating those outcomes and making decisions on adjustments when they do not meet expectations is the goal of the entire study.

The middle portion – design and construction – is the hard part of envisioning and planning when conducting a DBR study. Since the design and construction phases are so closely linked to the analysis and exploration phase, even the best made plans can go awry. In all my years of working with teachers – as an educational consultant and instructional coach – I know how quickly plans and reality can diverge. As such, I am always hesitant to draft plans without a deeper understanding of conditions. Since my topic is relatively new and weaves several disciplines together, I cannot gain that deeper understanding from the literature. There are simply too many gaps. It must come from the analysis and exploration.

I was able to come up with a plan for the design and construction of my study. However, I had some concerns about getting a study like this through an institutional review board. What is the best procedure for conveying plans of study when you do not quite yet know exactly how they will occur? If your plans change due to new information discovered in the analysis and exploration phase, do you have to return to the institutional review board with revisions? Will an institutional review board even approve a study of this nature when the steps are not concrete and clear from the beginning?

This is a study that I do intend to pursue – in fact I’ve already submitted a proposal to present it with colleagues at an OER conference this fall. I just worry about our capacity to get approval to conduct such an in-depth, sensitive study – especially when both students and faculty are involved.

References

McKenney, S., & Reeves, T. C. (2012). Conducting educational design research. New York, NY: Routledge.

Literature Review & EDR

Though I have written many literature reviews in the past through both my educational technology and history degrees, the experience of writing a literature review for an EDR study is unique. Though literature reviews for studies of all methods are used to inform the background and purpose for the research questions being asked, a deeper level of ambiguity exists when developing an EDR study due to the issue of context. Researchers conducting a literature review in the analysis portion of EDR must simultaneously consider both the context of the research setting as well as the knowledge existing in the literature.

While writing my literature review for my EDR proposal, I found it difficult to focus. Generally, when I design a proposal on a non-EDR study, I have some sort of hypothesis about how the study will turn out.  However, the problem I have chosen for this EDR proposal is complex and I am not sure how it will result. In essence, I have no answers – just lots of questions. This situation makes it difficult to craft a literature review because I don’t know what situations will inevitably arise later in the study. I have wondered if this is a common experience among researchers conducting an EDR. Is it common to return to the literature review and updated it in the midst of an EDR? I can envision this being a possibility.

Another muddy point I had while conducting my literature review was a distinct lack of research tying the various components of my study together (media literacy education, higher education history, and active learning). Though topics in the articles referenced other topics, there was no single article uniting them. It occurred to me that if my study was conducted, it could be the piece of literature connecting the three areas – a feature which I think is unique to EDR. This lack of information also made it clear why the field-based investigation is a crucial component during the analysis phase of EDR.

Overall, I found myself struggling to craft the literature review and find key studies to help piece together background information necessary. I don’t know how much of this is the lack of pedagogical studies conducted by history faculty (in comparison to other fields) or if it is a typical experience in crafting a literature review for an EDR study. Perhaps it is this lack of knowledge about a problem that makes the EDR study so unique. As we continue through the process of learning about EDR, I cannot help but continue to associate it with the idea European explorers of the 15th century (think Columbus). Those who conduct EDR are truly exploring the major issues of education as they occur at a grand scale. I see lots of potential benefits from this kind of work.

Educational Design Research

In the past month, we have begun exploring Educational Design Research (EDR) methods. Though this research approach informally aligns with the way I have designed past courses and training materials in practice, there are a number of complexities that complicate the methodology. Below is a brief description of each “muddy” point I still have at this point in our studies of EDR.

Connecting EDR Models and Publication

I can’t recall a time where I have struggled to understand a published study quite like I have this semester with the EDR publications. For at least a week, it seemed as though the example studies I was reading and the descriptions of EDR models by McKenney and Reeves (2012) and Bannan (2007) didn’t align. At this point, I am not really sure that they do align fully – especially when taking publication dates into consideration. McKenney and Reeves (2012) description of a general EDR model followed the Hakkarainen (2009) study. I suppose I expected the terminology between models and publications to align clearly. When that didn’t happen, I had to really struggle to identify the model components within the publications.

Identifying EDR Cycles

McKenney and Reeves (2012) describe various types of cycles – micro, meso, and macro – that characterize the iterations of an EDR study. At this point, I am still having trouble applying these cycles to published examples of EDR. I do not know that I fully understand the characteristics that define each cycle which may be the source of my problem.

EDR Concept Map

For our first reflection assignment, we were asked to create a concept map of our current understanding of EDR. The full version of the concept map can be found here.

edr-mind-map

Because I am having such difficulty conceptualizing educational design research as a whole, I dedicated my concept map to organizing the various components presented in the readings as a means of processing. All of the components of this concept map originate in McKenney and Reeves (2012) or Bannan (2007). The concept map begins for me with inspirations. EDR pulls several concepts from both curriculum development and instructional design. Both components must be considered when designing educational design research in terms of the level of reach as well as developmental processes. Next, I move across the concept map to inputs which highlight the various points of reference researchers can draw on to begin conceiving of an educational design research study.

Next, I move to the methods and frameworks which details the components of the study. Though it is not possible to indicate via a Coggle, I can see numerous areas where the Generic Model and the Integrated Learning Design Framework overlap. For example, the informed exploration piece of the Integrated Learning Design Framework is essentially the analysis and exploration phase of the Generic Model. Both phases describe the point of the study where researchers explore the problem of the study in greater depth through literature reviews, needs assessments, initial surveys, etc. Despite the similarities, it does seem that the Integrated Learning Design Framework is geared more towards practical elements of exploration when compared to the Generic Model.

Last are the outputs or outcomes of an educational design research study. Both practical and theoretical contributions should arise following the iterative cycles and results which guide both future research endeavors as well as practical recommendations for implementation.

Though I know this initial concept map is basic, it does identify my relative understanding of the methodology thus far. I kept it simple intentionally, allowing me to adapt and change the model as my conceptual and practical understanding of educational design research grows. Ultimately, time permitting, I would like to return to this concept map and update it at the end of the semester as a means of measuring my depth of understanding through this course.

References

Bannen, B. (2007).  The integrative learning design framework: An illustrated example from the domain of instructional technology. In T. Plomp & N. Nieveen (Eds.). An introduction to educational design research. Retrieved from http://www.slo.nl/downloads/2009/Introduction_20to_20education_20design_20research.pdf

Hakkarainen, P. (2009). Designing and implementing a PBL course educational digital video production: Lessons learned from a design-based research. Education Technology Research and Development, 57(2), 211-228.

McKenney, S., & Reeves, T. C. (2012). Conducting educational design research. New York, NY: Routledge.

Swan, K., Day, S. L., Bogle, L. R., & Matthews, D. B. (2014). A collaborative, design-based approach to improving an online program. Internet and Higher Education, 21, 74-81.