Saturday, March 17, 2018

Rationale? I don't need no stinkin' Rationale!†

 Alfonso Bedoya in The Treasure of the Sierra Madre
Dissertation proposal, draft number...I don't know - lost count - has been submitted.  Things look good, well at least in my humble view, so I hope I am ready to defend this proposal within the coming month and become a "candidate" soon‡. With the draft submitted I can now focus on the other pile of academic work that needs attending to, and is more collaborative than my dissertation.

In any case, a recent incident (incident sound too austere...happening? occurrence?) I was reminded, for the umpteenth time that what we, as educational researchers, are expected to have a purpose in our research. What is the rationale for the study? people ask. Why undertake this study?  Who benefits? What is the problem you are trying to solve?

As you can tell from the title of this blog post I hold the position that I don't need no stinkin' rationale.

I could make something up like "by examining population X, subgroup Y, we can infer that the results might be applicable to generation Z" (or something like that).  If pressed, I could be convincing in writing some rationale like that, but it wouldn't necessarily be the truth, hence, in my view, it should not be included in the published literature just to check off a requirement.  I firmly believe that if a researcher finds some occurrence interesting and wants to investigate this curiosity they should be able to do so (within the limits of ethical practice of course), without being weighed down by needing to produce a rationale for the study when they try to get it published.

This is by no means new to me.  I've been hearing various versions of this from established researchers for the past decade. My first exposure was when I was an MA student and we had a guest speaker in class who boldly proclaimed that if your research doesn't specifically address social injustices and under-represented populations your research is basically garbage♠. I basically wrote her off because she clearly didn't seem interested in a discussion about this topic (very odd for someone in that position of authority if you ask me).

In any case, while my other experiences with academics have definitely been less polemic compared to that initial experience, I am still surprised that academics basically act within a prescribed box.  I am not taking about the necessity for validity, reliability, and/or trustworthiness. Those are important in research. What I am talking about is purpose. At the end of the day, if I am satisfying my own personal curiosity as to why an event is happening, is that any less valuable than doing it for the benefit of others from the start?  It is certainly appears to be more altruistic, I'll grant you that, but does value diminish simply because you didn't work on something that was meaningful to others as well, at least at the time of research?


† In writing this post I learned where "I don't need no stinking..." comes from! (see here). Who would have thought an academic rant would teach me some pop culture?
‡ You know, recently I've felt that doctoral students should have some cool video-game-esque ranks, maybe with some badges to go with them, but I guess that's a thought for another post 😜
♠ Well, OK. No, she didn't say this exactly in those words, but based on what she was touting about her research, and her research agenda, and how it was related to other people's work that was more theoretical in nature, or didn't deal with her populations, it was easy to connect the dots.

Monday, February 5, 2018

What am I training for again?

From PhD Comics

It's been a while since I've had the bandwidth to think about something other than my dissertation proposal.  When I started this process four years ago (starting with matriculation in March 2014) I thought I'd be the first or second person in my cohort to be done (ha!), but like most marathoners I guess I am part of the pack looking at the fast folks ahead of me 😏.  Being part of the pack does have its benefits, such as getting an idea of how long the process takes (having friends in other cohorts also helps with this).  I thought, initially, when someone submitted their draft (be it proposal or final dissertation) that you would get feedback and signs of life from your various committees soonish, but seeing Lisa's journey (currently at 5 weeks and counting) gave me a reality check. Waiting isn't bad per se (we wait for a ton of things in life), but I think it is the expectation of things to come that makes this type waiting much more anxious for us doctoral student. Questions pop in your mind such as: Will they like what we submitted?  How much editing do they need me to do? Will they ask me to go back to the drawing board? How long will that take?  And if I have to defend this thing next week...well, do I have time to prepare? Do I remember everything I read in my review to the literature? eeek!

That said, I think I should rewind a bit.  What have I been up to?  Well, lots, and lots, and lots of reading and then funneling that into some sort of literature review. The past 4 (or 5?) weekends have been about process (and grit?); they have been about sitting down for hours and crafting what I learned into a coherent literature review. They have been about concentration (and probably some weight gain due to all the sitting...maybe some bad posture as well).  And, at last, this past weekend I finished the 139 page monster, put it all into one word file and emailed by advisor (hopefully she won't hate me because of the length 😜 ).  Without counting references, front-matter, and tables of contents, here is what the word count breaks down to:
  • Chapter 1: Introduction  ≅ 3,800 words
  • Chapter 2: Literature Review  ≅ 16,600 words
  • Chapter 3: Methods ≅ 6,700 words
Assuming that the average academic article is around 8,000 words (with references), I've written 4.5 academic articles, and this isn't even the full dissertation!

Now that the draft is submitted I have some free time (maybe 3-4 weeks if other cohort-mates reports are any indication of average length of waiting) to work on a research project that's been on the back burner and that's collaborative. In this project, in order to make it  to the appropriate word length the operative word is cut. This is a little challenging because when it comes to cutting there aren't really that many options.  Do you cut your methods?  Then reviewers will call you out on incompleteness of methods (and you might actually get penalized for it!).  Do you cut your findings?  Well, for a qualitative research paper without some qualitative data (which takes up space) you could be told that there isn't enough data (or they could say that you are making things up).  Do you cut the literature review?  Well, this seems like the most likely place to make cuts, but how is your reading audience assured that you did your due diligence? Hmmmm... dilemma...dilemma...dilemma.

This pondering lead me down another path: a recent (recentish?) tweet by Maha Bali, a critique of doctoral programs.  The gist of it was that PhD programs don't really prepare you for a lot of things that are expected in academia. The traditional pillars of faculty in academia are research publishing (usually of the academic article variety), service, and teaching; however the critique was that doctoral programs don't really prepare you for these things. I think this is is a much larger discussion which first needs analysis of what faculty actually do and what they are asked to do. Maybe this is an opportunity to examine what faculty do and their relation to other roles at the institution, but for now I want to focus on one part of it: the research and publishing.

I consider myself lucky to have had opportunities to research and publish prior to pursuing my EdD, and to do this both alone and in collaboration with others (as an aside, I find collaboration more satisfying as it satisfied both work and social aspects of life). Working on the doctoral degree affords me the opportunity for some directed study to fill in potential areas that I was missing, and to see things from different frames of view; for instance I have a finer understanding of learning in other fields such as military and health-care (just to pick on a couple) because of my cohort-mates.

However the dissertation process, and the reason for this process, seems quite arcane to me. I understand, from a cognitive perspective, that the dissertation is meant to showcase your skills as a researcher; and those with more romantic dispositions among us might also say that it contributes to the overall level of knowledge in our field. But if you are one of those romantics let me ask you this: when was the last time you cited a dissertation in your research?  And, just in case you are a smarty-pants and you have cited one dissertation, how often do you check out dissertation abstracts for your literature reviews? I digress though... Back on point...

It seems to me that as an academic (well, if I chose to go the tenure track once I earn my EdD) I need to contribute to the field by writing research articles, field notes, book chapters, reports, and maybe even a whole book; and I also need to provide peer reviews to fellow authors.  With the exception of book writing (which every academic does not do), the vast majority of writing is between 3,000 and 9,000 words.  A dissertation  is considerably longer. This makes me wonder (again) whether the purpose of the dissertation is one of endurance (i.e. if you can do this, you can do anything!) or of holding us up to romantic, inappropriate, or irrational standards, as in "once you graduate you are expected to write books". As an aside, this may have been the case when there were fewer scholars around, but these days there aren't enough positions open in the traditional tenure-track faculty profession, so the Alt-Ac isn't even addressed or acknowledged...but again, I digress.

The instructional designer in me has pondered the purpose of the dissertation (even before I applied to doctoral programs). If we've already replaced the once prevalent Masters Thesis with other means of assessment (or at least made the MA Thesis as one of a few options), why can't we do the same with the Doctoral Dissertation, which - if we're honest - just another form of assessment.  I should say that my own point of reference here are what are called 'taught' PhDs where there is required coursework before you are allowed to be a doctoral candidate, and not the kind you might find in Europe where you are apprenticed into the discipline by applying as an apprentice (basically) and just work on your dissertation upon completed of a masters program.

So my three questions out there for you:

  1. Do the traditional pillars of academia still hold up or should be re-conceptualized? What might they be? and how do they work collaboratively with other parts of the academy?
  2. Based on these current pillars where does doctoral education fall short (name your field as fields will most likely vary)
  3. Keeping the dissertation in mind: what would you replace it with? What are the underlying assumptions for your model?

Discussion welcomed (if you blog, feel free to post link)

Thursday, January 4, 2018

2017 year in review - school edition

From wikipedia: 1779 illustration of a Catholic
Armenian monk of the
Order of St Gregory the Illuminator,  
Happy New Year! Yeah... it's the fourth of January, but I figure I can get away with it since we're still in the first week of 2018, and this is my first post for the year 😉

Things have been a little quiet here on the blog as of late. Not a lot of MOOCing, not a lot of virtual connecting, not a lot of collaborative or cooperative learning as was the case in previous years.  There has been a lot of reading, mostly in monastic form - you know, lock yourself in a room and read until your inner teenager starts screaming at you "are we done yeeeeeet????" - I guess I am really in the thick of dissertation prep "stuff" (reading and sorting mostly) which I hope I'll get through in 2018 (for the most part anyway)

I thought I would take a break from the monastic lifestyle to put together a few things that really struck me in 2017, at least as far as my own learning, and learning journey, go.

I guess the first thing of interest is that  2017 was my last year of courses. In spring 2017 (winter 2017 in Canadian terms) I completed EDDE 806 which was the last structured and graded seminar for my EdD.  Coming into the seminar I really wanted to get done with classes, so even though I had until Fall 2017 to complete the requirements for the seminar, I made sure I was there each session and doing what was required to be done. 2017 didn't start off energetically though...

I think the major realization I came to in 2017 was that I had over-exerted myself in 2016, and as a consequence I really felt burned out; not starting to feel burned out, but actually burned out. I felt a little guilty because I still wanted to participate in extracurricular academic stuff such as virtually connecting, MOOCs, and working more on research papers with friends and colleagues as these activities encompassed both an academic purpose but also a social purpose. However, in the quest to get moving with the dissertation proposal I needed to sacrifice most of the more fun things in academic life and work on the more utilitarian parts to just get done. The hope is that people will still be there once I am done 😜

Even though I had bits and pieces of my proposal already forming in 2016, I put all of those mostly in the back burner for the spring semester because I wanted to finish off the requirements for the final graded seminar of my studies (EDDE 806). I think what I learned here is that I am mostly a sequential person when it comes to doing stuff. Many people think I can multitask like a madman, but I guess it's all relative from where you are standing.  I think I can do two or three ongoing projects at a time (of varying cognitive intensity), but class + proposal weren't working out for me.

So, once EDDE 806 was done, I took a mental breather, and a month later I begun working on my proposal again.  The introductory chapter (chapter 1) and the methods chapter (chapter 3) seemed to be on a better path, so I focused on that.  I think the core  of what my proposal is was pretty well done in August when I completed writing chapters 1 and 3. The who, what, where, and how were pretty much answered.  The big question still looming was  the "why".  It was hinted at during the introduction, but it needed more exposition, via a literature review chapter. So, September to December I read...I read a lot...and then I read some more (all the while taking notes)...until my inner knowledge glutton decided it was enough. That was after around 500 PDFs which breaks down to about 20 books/conference proceeding compilations and the around 480s articles (overkill?).

This past month I've been going back through the readings and I realized a few things: First is that I had downloaded some articles twice or thrice (since I had been collecting articles on MOOCs since 2014 in anticipation of this moment), but there weren't that many duplicates (less than 10% of the total, definitely).  In reading the same article a few times, over a period of a few months, different things jumped out at time; some things were the same, but my increased knowledge from readings was definitely coloring how I interacted with the texts.

The second part that jumped out at me is that no matter how detailed my lit-review search is, there is still stuff that will be missing from journal database searches. By looking at the references of the articles I read I saw that there were things that could be of interest (maybe), but they were in specific disciplinary journals that didn't deal with education (travel, medicine, geology, etc.), or that were in proceedings of conferences from various professional associations that my library doesn't have the most current access to, or even things that were in a different language.  I could decipher things in French, Spanish mostly without a problem, and some Portuguese, but reading academic discourses in languages that you are not used to reading academic discourses is definitely slower and more taxing. This is not a revelation - I know this from a theoretical perspective as a linguist, but this is one of the few times I felt it personally. Even so, there is probably good stuff out there in languages that are not accessible to me, and things that are not even on my radar, so 100% completion on a specific topic is a fool's errand.

The third thing, that really surprised me, was how much misinformation there was about the history of MOOCs.  In the grand scheme of things it was a small amount of articles, and it was limited to the introductory sections of articles where they were introducing the topic, but having such erroneous info printed in academic journals was a bit jarring. An example: complete lack of discussion around cMOOCs, and xMOOCs being described as evolving from OER. Now, while a nuanced discussion on the topic could reveal that certain xMOOC providers do have dotted lines in their histories to OER, I don't think this is a broad generalization that can be made, and in an introductory section to a journal article it seems quite misleading, especially to someone who might know nothing of MOOCs.  This got me thinking about literature reviews in general, and how little work goes into them sometimes; the write something, cite something, to get it done approach, rather than really thinking about it.  I know that I may have gone overboard here with mine, but I think that the minimalist approach also can be troublesome because it can (and sometimes does) devolve into a find the reference that supports you POV.

This leads me to AK's Theorem of the Funnel of Usage for the literature review (you read it here, so you better cite me! 😝). Basically what my theorem says is that for a literature review you may read hundreds or thousands of pages of stuff, you may comment or find useful things that amount to several hundred pages, bu ultimately only a small amount of what you comments on and find useful will make it your your literature review.  I am looking at a lit-review of 40 pages (double spaced) maximum. Assuming 15 pages per article on average (or around 300 per conference proceeding and book), I'd estimate 13,000 single-spaced pages read.  I have around 200 pages of excerpts and notes, which will go down to 20 pages. All that reading doesn't go to waste, it informs my views and stances (and also impacted some questions I'd like to ask in my interviews, in the methods section), but there is definitely a funneling effect here.

Speaking of overkill, perhaps I didn't have to read as much as I did (we'll never know). I think the fear is that my exam committee will decide to ask me about an article that I actually have NOT read that they think is important and then I'll be standing there like an idiot (and possibly fail 😖).  There is a Greek expression that has stuck with me πιασμένος αδιάβαστος which basically translates to "caught unread" (unprepared). It goes back to the days where children read at home, and they went to school to be quizzed on what they had read. You get caught unprepared, it's a bit of a mark of shame. I sort of felt like this when I was preparing for the comprehensive exams for my MA in applied linguistics and I passed that with flying colors, so I guess I shouldn't be as worried about these things...but I guess students reap what teachers have sowed...even if it was more than 25 years ago...

Another thing that really piqued my interest was thinking about how many web 2.0 technologies have died between now and 2008.  In reading articles about the original CCK, and MOOCs or other collaborations that have happened since I see quite a few technologies that closed down (either recently or some while back).  This makes me ponder a bit about what can be done about digitally preserving some of the artifacts created, or ways in which things worked, or something else.  While describing activities with PageFlakes (for example) is interesting, what happens 20 years down the road when no one knows how PageFlakes worked?

Finally, I realized that this entire process took about five times (or more) the length of time I expected it to take (there are quite a few "LOL :-)" next to self-imposed deadlines I've missed), and in academic articles I've read I noticed that others have also forgotten to include something in the reference list that they referenced in the body of the text, so it makes me not feel as bad when I've done my due diligence but still something goes missing or is erroneously ommited.

I kind of feel like the quote from Francois Truffaut is applicable: "You start a film and you want to make the greatest one ever made. Halfway through, you just want to finish the damned thing." - just substitute film with dissertation :)

Anyway, back to the monastic lifestyle to get some more things done.  How was 2017 for you, academically speaking?

PS Hey! Committee members! I don't know who you are yet, but if you googled me and came across this post, go easy on me 😜

Monday, December 11, 2017

One more thing!

... No seriously! I swear! This will be the last thing I read and then I will start to write my literature review ;-)

I am back up for some air.  When I originally made my plans last May to have the fall semester be the semester that I focused on the literature review part of my dissertation proposal I sort of envisioned a lot of reading.  Reading on the train. Reading on the weekends.  Reading while walking (through text to speech), reading while driving (also through TTS).  My goal was to put pen to paper (figuratively speaking) on November 30th.  Well, that date has come and gone and I still haven't put pen to paper yet.  And, I am still reading. 

A couple of times I've actually come close to being done reading - having my "to read" folder on dropbox empty and all things read, skimmed, or otherwise evaluated for usefulness for my proposal.   When I've come down to 10 items somehow the folder magically populates again.'s not magic - I add things to the folder.  Three weeks ago I remembered that I should have looked at the Horizon Reports to see if there were any prognosticated trends that related to my proposal. A couple of weeks ago I remembered that I didn't look at the Educause Review for related items. And, this past weekend I got notice that the OLJ (OLC's peer reviewed journal) just released a new issue and a couple of articles seemed relevant.  D'oh!

The encouraging news is that I am basically done. There are two or three relevant(ish?) articles on the OLJ, and I have the MOOC Invasion to read (or at least skim).  After that I am truly, 100%, no regrets, calling the literature review reading done and I will start to collect my notes to write the chapter up.

I suspect that I am not the only doctoral student who has suffered from the "one more article! one more book!" syndrome.  When doing research (alone or with colleagues) for an article or chapter-length piece there is a tacit understanding that you just can't fit everything in, and that stuff gets left on the cutting floor.  No one will do an oral exam for an article you submit to a journal for review - so if you haven't read something...well, no one's the wiser.  For a dissertation I feel like what you don't put in (and what you don't read), could come back to bite you in the oral defense, hence better be prepared.  But...can you take preparedness to an necessary extreme? 

Any thoughts from current doctoral students and recent grads?

Friday, December 1, 2017

Letters of recommendation - what's up with that?

It's been a while since I've blogged, or at least it really feels like it.  I've had my nose stuck in (virtual) books trying to get through my literature review - but more on that on some other blog post. I came across an article on InsideHigherEd this past week asking whether or not letters of recommendation are really necessary. My most immediate context is admissions, given that that's part of my work at the university, but the people who gave their two cents also mentioned something I had not considered: academic jobs. I won't rehash the opinions of the people who wrote for the article, but I will add my own two cents, mostly from a graduate admissions perspective. I don't have a fully formed opinion on letters of recommendation for employment purposes, but I'll add my two cents as a prospective hire (in a few years when I might be done with my EdD :p)

For admissions to graduate course of study, be it a masters program, a PhD program, or even a certificate program, I really personally don't see much value in letters of recommendation any longer.   My point of view is framed from the perspective of a student, an instructor, and a program administrator.   When I was applying for my first master's degree I bought into the rationale given to me for letters of recommendation: former professors can provide the admissions committee qualitative information about you as a learner that a transcript cannot provide.  This is all fine and dandy, and for me to worked out: I was working on-campus as an undergraduate student, and I had some professors who I had for more than one course and who were able to write letters of recommendation.  This was a privilege that I had that other students may not have had.  For my second masters I was applying to the same college, and I was applying to a new program of the college, so they looking for student, so getting recommendations wasn't that big of a deal.  Once I finished my second masters, I really didn't want to deal with more solicitations for letters of recommendation - I started to feel odd, and I kept going back to the regular well of people for recommendations.

So, I applied to two programs concurrently so that I could write one statement, and the letters of recommendation could pull double duty.  After I finished my last two masters degrees I took some time off regular, "regimented" school and programs and focused on MOOCs.  Going back to earn an EdD posed some issues as far as recommendations go.  I had previously applied to a PhD program at my university (at the college in which I earned two masters! - never heard a final decision on my application by the way), and by the time I wanted to apply to Athabasca I felt that the well had run dry for recommendations.  Former professors still gave me recommendations, but I kind of feel I was taking advantage of their kindness by asking for a recommendation for yet another degree program I wanted to pursue (don't judge, at least I complete my degree program haha 😜).  Not that I am thinking a ton past my completion of the EdD, but should I want to pursue a regimented course of study in the future (degree or certificate program) the recommendations will be an issue; not because I can't get them, but because I feel bad about asking for them - after all I am asking for someone to volunteer their time to give me a recommendation when my academic record should suffice. This is how I feel about the GRE and other entrance tests, by the way.  If you've completed undergraduate studies then the GRE is pointless - you can do academic work.  If you are unsure of the academic work capabilities of applicants, accept them provisionally.  Just my two cents.

Another lens I view this through the administrative.  Asking for letters of recommendation, and subsequently receiving them (or not) requires time.  It requires time from the student (especially in tracking down referees if they don't submit stuff in time), it requires processing time from admissions departments, and it requires reading time on the part of committees who review applications. When such a system takes that much time and effort into it, you have to ask what the benefit, or net positive gain, is.  Going back to the story I was told - the qualitative component of the transcript, basically - does make sense in theory, but in practice... not so much. 

While I don't make decisions on applications that come to my department for review, I sneak and peek at materials that come in because I need to process them.  What I've noticed is that by and large (1) recommendation are uneven, and (2) they tend to be the same, more or less, just with different names.  The unevenness is partially cultural in nature.  If you get a recommendation from someone employed at a western institution you tend to get (more or less) what you seek.  However, non-western colleagues don't use the recommendation system so for them a recommendation is just an affirmation that the student was indeed in their class, in the specific semester, and from what they remember they performed well.  The "basically the same" aspect of recommendations runs into the same problem as non-western recommendations; that is that recommendations basically boil down to: student was in class, they performed well, so accept them.  It just turns out that western colleagues are more verbose in their recommendations so they happen to add in some anecdotes of your awesomeness as a candidate, but even those anecdotes tend to run along the same wavelength most of the time:  asked interesting questions in class, was the first to post in forums, engaged fellow classmates, submitted assignments early, etc.  From an administrative perspective there is (so far as I know) no background check on these folks providing recommendations so we are taking what they are writing in good faith.

Finally, as an instructor, I am lucky, in a sense, that I haven't had to write a ton of recommendations.  I've done so a couple of times but after a few original recommendations I've basically gone back to the awesome student, accept them, here are a couple of anecdotes formula because that's life, we're not living on Lake Wobegon. I'd gladly give a recommendation to former students who did well in my classes, but it's hard to not feel like I am writing a generic letter sometimes. So why spend time writing something that feels like a template letter if I am not providing much value to the system?

In short, recommendations for admission add no value while taking away time and resources from other areas.

In terms of letters of recommendation for academic employment, on a purely theoretical basis I'd say that they are pointless too.  Both for reasons articulated in the IHE commentary piece, but also for one of the reasons that's similar to graduate program admissions: the genericness aspect.  I think having some references is fine, but I think a quick conversation (or heck, a survey-style questionnaire) would be more preferable to a letter. The reason I think it's not that useful in hiring decisions is the same reason no one gives recommendations anymore (for us regular plebes getting work), and that is that people sue if they get wind that they got a bad recommendation. Generally speaking no one will agree to give you a letter of recommendation (or reference) if they can't give you positive reviews, and HR departments just confirm dates of employment these days.  Nothing more, nothing less; otherwise they risk a lawsuit. So, if you're not getting much information about the candidate, and if the information is skewed toward the positive (because that's how the system works), then is the information you're getting valuable?  I'd say no.

So, what are your thoughts?