Abstract Art: Stories about denial and acceptance

Below you can find some stories about abstracts for conferences and some associated thoughts. It’s pretty much 4 stories and a little bit more than that. I’d welcome other stories and experiences as well as any insights on what I might be missing. It feels like I am missing a lot.

1) I have this friend.  He has some negative feelings about the way conference abstracts are vetted in a certain English teaching focused organization in Northeast Asia. My friend, who chooses to remain nameless and also wishes to not sound too whiny, first applied to present at an international conference around 5 years ago. I saw the abstract and it looked pretty good to me. It was denied. At the time, I told him something like, “You know what man, it is really hard to map out where it might have gone wrong. There are themes and it is a bit of a crapshoot and you never know what they are looking for. You mustn’t beat yourself up.” I probably didn’t actually use the word musn’t, to be honest.  I am not sure I have ever said it, but I digress. I think my friend took my advice for the most part. He was, however,  quite suprised the following year when he saw nearly the same title he’d submitted in a presentation at the same conference. Yeah, surprised and not happy. “Coincidence, themes, different people reading, different ideas” and so on I explained. He seemed to believe me and mostly get over it. Yet he still harbors a deep distrust of the organization to this day. For the next few years my friend was actively involved in that organization and all (or nearly all, I am not quite sure) his abstracts were accepted. He strongly felt that his involvement in the organization helped his case in this regard. We joked that knowing “the secret handshake” was more important than knowing how to write a good abstract. He maintains that his first abstract was just as good, if not better, than his others that were accepted at following conferences.  I still say it is largely a crapshoot and there are a variety of factors in play. What are these other factors,  anyway?

2) In the aftermath of the recent (2013) K0TESOL International Conference there was an interesting exchange on the K0TESOL Facebook Group (which some might know I am not a member of and others might know I can’t resist checking at times) someone  raised the point of how a big name dude in field had both (all?) his abstracts rejected. Someone else (who I believe was involved in the selection process) said something to the effect that that is what happens with blind vetting sometimes. There were some Tiger Woods based metaphors as well. (There were over 100 comments in that thread and I am not about to go look them all up.) I think the parallel was drawn between Tiger Woods being denied a round at the local course. The reply was that maybe Tiger didn’t have it that day or wasn’t at his best and it could happen to anyone. Confusion over metaphors aside, I was very interested in what read to me like blind belief in the blind review system. Surely the system is only as good as the people doing it? I surely don’t wish to suggest that big name dudes need to be accepted every time but I also think the system can (and *should) be questioned. I don’t believe an organization can simply say “blind review” to evade any questions about the type of sessions accepted/not accepted at conferences. I do believe it might behoove organizations to consider what is happening when well-regarded presenters are denied. I think it is all too easy to attribute it to prospective presenters just having a bad day on the links. What are the “best practices” for setting up such committees? How can the vetters be vetted?

3) I don’t want to get too specific here but I have seen some accepted abstracts (at various conferences) that seemed loaded with jargon. Well, not only loaded with jargon. Not only loaded with jargon used in ways I’d deem unacceptable but also jargon used in ways I think most people I respect in the field would say is not the usual way it is used. How does this happen? Are vetters blinded by the jargon? Am I too easily turned off by jargon? Am I missing something?

4) I recently submitted an abstract for a conference in a country, indeed region, I have never been to. I was delighted when I saw they provided a set of criteria they’d be using to vet the proposals. I was even more delighted when I saw samples of what a successful abstract for certain types of sessions might look like. I thought this was a brilliant idea and found myself wondering how commonplace this is in the rest of the world.  If not, why not? When writing abstracts what information provide by the conference committee have you found helpful?

5) This might be just me but…as a potential presenter I sometimes find it extremely hard to know what I might be interested in talking about as much as 8 months in the future.I fully understand that conference committees want to get everything sorted but I also think there is something missing when people are not talking about what is most interesting to them at the moment. I don’t really have any great suggestions or anything here. Am I expecting too much? Is this a common issue for others? Is there anything that can be done about it?

Advertisements

6 comments

  1. Rob Dickey

    Hi Mike,

    Thanks for the BRILLIANT post!!! As a matter of fact, I am one of the KOTESOLgroup posters you’ve paraphrased above. You well capture an important concern for many folks. Conference proposal vetting.

    I have been a member of the vetting teams for a number of conferences in Korea and at TESOL International and some other bodies not related to TESOL. I would say there are at least three approaches, and many of those on the vetting teams share concerns about each approach.

    (1) Truly blind vetting with strict criteria and scoring rubric.
    (2) Truly blind vetting with lax (or no) criteria and/or scoring rubric.
    (3) Less-than-blind vetting.

    There is also, of course, the non-vetted session – either an invited session or something that comes in the vetting process but is quickly accepted prior to scrutiny. These last two are generally for “big name” speakers.

    Vetting abstracts is sortof like Oral Proficiency rating systems. Either you believe in it, or you don’t. I don’t. They are neither as precise nor impartial as claimed. (You can train people to concur on a score, but they will give differing reasons for how they came about it, and argue amongst themselves on those points, so that’s a crock!) Still, it is a fact of life. So, the first thing is – acceptance or non-acceptance of a proposal is not a very accurate measure of the quality of the abstract. (Think journals – in submitted papers at least they see the entire final package, yet one can still argue merits of the rating – but with conferences there is only a brief abstract!) Still, when looking at 200 scored abstracts, there are very few where the three or four raters score wildly differently. (Mid to high, or mid to low scores, but both high and low scores on the same proposal almost never happens.)

    Less-than-blind vetting suggests that scores can be “reconsidered” based on the reputation or biographical sketch of the presenter. So, in that case, “connections” can help.

    Still, there are a couple factors that don’t quite fit in the “3 vetting approaches” I’ve outlined above.
    (a) “Tracks” (or conference sub-themes) – how important are these, and is there any attempt to balance the number of sessions in each? I.e., are there rough quotas, might a higher-rated presentation be not accepted because there are “too many” in that theme? (This is very strong at TESOL International, where Interest groups are limited in the number of sessions, and proposals are directed to Interest Sections for sponsorship.)
    (b) Demographic aims – the most contentious of all factors. Is there an attempt increase (or decrease) the number of presenters who —
    … i. are from abroad
    … ii. have a history of good (or bad) presentations
    … iii. are relative novices to presenting (at least before this organization)
    … iv. belong to a particular nationality

    Tracks and demographics are something that the program team (program chair, probably in discussion with the conference executives) may determine after the vetting scores have come in. If your proposal form requires this kind of information, there’s a good chance it will be a factor.

    KOTESOL International Conference 2013 did “truly blind” vetting, and the National Conferences for 2013 (and upcoming 2014) similarly. Rubrics in KOTESOL are becoming increasingly similar to TESOL International designs, and that’s not an accident. Tracks are far less important in KOTESOL, though these can be a bit of a factor because too many proposals get similar middling scores and some must be selected while others not. As an example, with 110 slots one year, there were 80 proposals that were clearly “yes” and another 50 with roughly similar “mid-rank” scores. (And as I recall, about 50 or so that were clearly less favored.) Some of the other ELT organizations in Korea worry far less about ‘blindness’ in their conference vetting processes.

    I think whether or not any of the considerations listed above should be incorporated, or how strongly, is a fair question, and many organizations wrestle with them from time to time.

    note – should anyone wish to see the proposal rubrics from TESOL2014 or KOTESOL National Conference (not International) 2013, I have them on hand (at home) in PDF. If I looked really hard I might find others…

    The KOTESOL 2014 National Conference (May 31, Daegu) call for proposals is available at http://www.koreatesol.org/nc2014CallForPresenters

    • mikecorea

      Thanks for the kind words about the post, Rob. I have to say though, if anything is brilliant, it is these comments. Very informative, helpful and balanced.Thanks so much. I guess I don’t have too much to else to add except my thanks for taking the time and sharing your insights here. Very much appreciated. I

    • mikecorea

      More…
      A comment I saw on Facebook was, “They had a rubric? Really? Nobody told me. What is the point of having rubric if you don’t tell people about it when they are writing their proposals?”” I don’t have an answer for that.

      • Rob Dickey

        good point. and lots of places don’t post their rubric – some don’t design a rubric until after the call has been out for a while.

        The earlier point about jargon is well-founded, but on the other hand, a bit of appropriate light referencing to show you know what the hell you are talking about, that you have read the literature, is indeed important. Of course it needs to be specific and tied-in to what you are doing. Conference abstracts are like Thesis abstracts – there is a bit of a science to it.

      • mikecorea

        Thank you for the response Rob.

        Very interesting about designing the rubric after! This has all been very instructive for me and I am glad I wrote this. I am obviously coming from a lack of background on this whole thing (thus all the question marks at the end of sections).

        Regarding jargon…fair point… BUT I think using jargon in awkward/inappropriate ways is a very bad thing. (I of course realize this is subjective too) Maybe lots of people have different things that catch their eyes and this is one for me.

  2. mikecorea

    Someone (perhaps a frenemy) on Facebook wrote, “Funny how you don’t hear people who get accepted complain about the system. Actually, not that funny.” I had a hard time commenting on this but here are some thoughts. Perhaps I am just writing too many words about a throwaway comment/joke. If that is the case I am sorry.

    If the whole thing above read like a “complaint” then perhaps I missed the mark I was aiming for. If any and all sorts of questioning everywhere reads like “complaining” to specific readers then I think its on them to consider their own perspectives and biases.

    Regarding what is actually written in my post, I think my friend (with maybe 4 out of 5 acceptances) who was accepted more than not is still allowed to wonder if it was connections that both helped and doomed him.

    The Tiger Woods thing came from someone talking about someone else not being accepted by someone who might very well have been accepted. Yes, talking about someone else that wasn’t accepted.

    I have heard plenty of people wondering how certain things were accepted. Perhaps I just surround myself with more negative people?

    Actually I have heard plenty of complaints in my time by those that were both accepted and not.
    Literally within minutes of hitting publish on this blog post I got a message from someone lamenting the types of abstracts he felt he had to write in order to be accepted—which he was.

    I am not super proud of it but I believe I have made some disparaging comments about abstracts I saw in years when I was accepted to the vaunted halls of the KOTESOL International Conference. Does this disprove your theory? Surely it does, right? The abstract that I submitted this year to the KOTESOL IC was not accepted (sad face) but I honestly don’t think that changes my opinion of some of the abstracts I saw that looked nonsensical/strewn with inappropriately (IMHO) used jargon to me.

    Maybe I would need to have been accepted each and every time at every conference before I could share my questions and stories?

    I guess that is all I have to say about that for now.
    I’d be really sad if every time something is questioned it ends up reading like sour grapes.

    I’d welcome any guest posts on this very interesting topic!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s