|
After a rejection at a journal, I was asked by my PhD supervisor to make my article more conventional in its structure. I thought I was being conventional, but clearly didn’t have the knowledge of structural convention that was needed. So I took some articles recently published by my PhD supervisor, and one ethnography article (since the study I was trying to publish was an ethnography), and reverse outlined them. I used Jessica Calarco’s outline of a journal article (located here as of 21 Feb 2023: http://www.jessicacalarco.com/teaching-resources-1/2019/8/30/article-writing-101 ) as a basis for reverse outlining these articles (it is on her website, under Tips & Tricks > Teaching Resources [link at the bottom of the page]). So in a way, this paper asks how widely and how exactly Calarco’s outline is applied (with an admittedly small and homogeneous sample). What I found was that, for the 7 articles I coded, Calarco’s outline was never exactly applied. Of course, the authors of these articles probably weren't looking at her outline and thinking whether they wanted to apply it or not — but Calarco's outline is clearly not only hers but is a set of usually unstated assumptions in the field. But with that caveat, it is still the case that the outline isn't exactly what everyone does. Some articles ranged very far from the outline in particular sections. Sometimes they followed some of the outline and neglected others. Sometimes they did most of the things but mixed up the order. After doing this, however, I know what usually happens in each section (at least across these 7 articles). In the following sections of this post, I will italicize the codes I used (I used Atlas.ti to code these). I will put numbers in parentheses to mark how many times I coded a particular move in the articles. So this is not number of articles that had these passages, but instead how many times the move happened. So if a paper mentioned the data, then mentioned the findings, then mentioned the data again, in that sequence, there would be Data (2) and Findings (1). ABSTRACTS This is Calarco’s outline for abstracts: 250 words or less
The most common moves in abstracts were to talk about Findings (13) and to describe the Data (11). Then the next most common moves were to discuss the Contribution to Literature (9) and to discuss Existing Literature (8). The moderately common moves in the articles were to Identify an empirical case that has had relatively little attention in the literature (5), State the research question (even if it wasn’t exactly in the form of a question, something like, “This paper analyzes how X and Y relate”) (4), and State a fact about the world (such as, “ethical consumption is a popular form of everyday politics”) (4). Rare in abstracts were to explicitly articulate a Gap in existing literature (2), define a Theory/Concept (2), and mention a Contribution to understanding world / Explain why these findings are important (2). Abstracts ranged from 172–251 words. INTRODUCTIONS Calarco’s Sections (she says the introduction should be 3 paragraphs) with my edits Describe the puzzle or gap in the literature that you will address with your data
Identify your research question and explain how you answer it
Explain the importance of your findings
Introductions most commonly discussed What we Know (12) and Findings (12). They often discussed What do we not know? (8), discussed the contribution (7), described the data (7), stated the research question(s) (7), and discussed what the existing research suggests might be the answer to the research question (6). They sometimes discussed the general analytical strategy of the paper (4) and clarified a concept (3). The papers rarely mentioned a fact in the world (2) (e.g., “a growing cultural narrative that children are “at risk””), a problem in the world (1), explained theoretical concepts (2), mentioned an exciting new trend in research (1), signposted what was to come in the article (2), provided a data excerpt (1), and gave wider context for findings (1). Introductions ranged from 401–1,188 words. JUSTIFICATION/LITERATURE REVIEW Calarco’s Outline (She says it should be “1,000 words or less”)
Calarco calls this the “justification” section to highlight that “The point of a literature review is not actually to review all of the relevant literature …The point is to make the case for why your study is important.” As you can guess, the most common move in the justification section is to talk about What we Know (39). The next common move is to Restate the puzzle or gap in the literature that you will address (12). Then the most common move after that is Justifying choice of literature or concept(s) (5). Then there are several less-common moves, such as: Clarifying/Defining a Concept (4), Contribution(s)/Clarifying the relationship of the paper to what we know (3), mention the research question(s) (3), mention facts about the world (2) or problems in the world (1), mention a general question (“how queer people connect in the digital age”), coining a concept (1), and discussing a new trend in research (1). Justification sections ranged from 776 – 3,390 words. METHODS Calarco says these should be “4-6 short paragraphs”, and this is her outline:
Most common in methods sections was Characterizing the Sample / Describe the Data Collected (24). The articles were mostly qualitative, and the next most common codes were Interviewee/Focus Group Recruitment Methods (8), Qualitative Coding Method / Describe how you analyzed the data collected (7), a general discussion of analytical strategy (8) (e.g., “By studying the political talk of organizational leaders advocating eat-local food practices in three cities, we aim to capture a sense of the broader discourse animating eat-local activism in the Canadian context”.), describing interview questions asked (6), describing the research sites (6), describing the general strategy for data collection (5) (e.g., “This geographic range allowed us to access a diversity of perspectives that were not constrained to the specific environmental and regulatory conditions of a single region or province.”), and giving a brief study overview (4). Less common moves were to make a statement that all names reported were pseudonyms (3 – the other articles did this in the analysis sections), make a statement about the representativeness of the findings (3), provide some findings (2), discuss empirical or theoretical literature (1), discuss methodological literature (1), describe political jurisdictions in which the data was collected (1), defining concepts (1), and explaining unfamiliar methods (1) (latent class analysis was the unfamiliar method described). Methods sections ranged from 469–2,464 words. The article with the 2,464-word methods section was a mixed-methods study. Methods sections had 3–7 paragraphs. The methods section were called “Data and Methods” (3), “Data and Method” (2), “Methods and Data” (1), and “Study Design and Data” (1). ANALYSIS Calarco’s outline (she gives no word limit suggestion)
Most common in analysis sections was to Describe data that typifies the pattern under discussion (e.g., through a quote) (129), Generalizing Across Sample (64) (e.g., “For many mothers, the nutritional importance of meat for a growing child’s diet was a matter of common sense.”), and Briefly explain how an example represents a larger pattern/sub-theme (52). Calarco, in her outline, talks about the difference between “supporting points” and “patterns in the data that provide evidence for each supporting point”. I did not find that distinction intuitive. Instead, I talked about analysis sections as having an “overall theme” (i.e., the main message of the findings). Then all of the papers broke that overall finding into subthemes. I called those subthemes just “subthemes” or “First Order Subthemes”. Then if one of those subthemes had a subtheme, I called that a “Second Order Subtheme” (I’m getting the idea of first, second, etc. “order” from Greek Philosophy — I’m using the idea loosely, but I wanted to use this language instead of saying things like “Sub-sub theme” or “Sub-sub-sub Theme”; so in the way I am using it, a “Sub-sub Theme” is a “Second Order Subtheme”). And so on. Some of the papers ended up having Fourth-order subthemes, and one paper even went so far as to have Sixth-order Subthemes. So it just refers to nesting of subthemes within each other. So what I found in terms of that is that there were the overall theme was not mentioned that often, as might be expected (7 times), subthemes were mentioned a lot more (37 times), Second Order Subthemes were a main aspect of the articles (mentioned 45 times), as were Third Order Subthemes (used 58 times). Then the more nested categories beyond that were rare: Fourth Order Subthemes (19), Fifth Order Subthemes (3), and Sixth Order Subthemes (2). So as you can see, most of the papers had 3 levels of subthemes. Something like this: Overall message of paper
That was the typical kind of structure in these articles. Some of the other common moves that happened in the articles were discussing how the findings were consistent with previous literature (31), introducing a set of first/second/third/fourth/fifth/sixth order subthemes (14), and generalizing beyond the sample/contextualizing the findings (12). Less common moves were discussing a conflict between the data and existing literature (7), offering an explanation for the observed patterns (5), stating the research question(s) (4), stating that the respondents’ names were pseudonyms (3), stating an analytical strategy (3) (e.g., “Our focus group data allow us to paint a richer picture of this orientation to food”.), and defining or clarifying a concept (2). Analysis sections were 2,859 – 6,411 words (the highest one there is not the mixed methods study). DISCUSSION Calarco says that this should be 1,000 words or less. This is Calarco’s outline, with additions I made. In general, I found Discussion/Conclusion sections to be the most variable in terms of format. I numbered Calarco’s sections and added a “title” section. “TJB” are my initials, indicating where I added a section. 1 – TJB > Title of Section 2 – Summarize your findings 2.1 Remind readers of the puzzle/gap in the literature that you are trying to solve 2.2. Remind readers of the specific research question that you have answered 2.3. Review what you found 2.4. Explain what these findings imply about the answer to your research question 3 – Discuss the implications of your findings 3.1. TJB > State Implication(s) of Findings (without doing any of the sub-tasks Calarco outlines) 3.2. Explain how your findings solve the puzzle or fill the gap in the literature 3.3. Explain how the resolution of the gap/puzzle helps to clarify, challenge, or expand existing knowledge or theory 3.3. TJB > Explain how themes and/or subthemes, or the implication(s) of the findings, clarify, challenge, or expand exiting knowledge or theory 3.4. Using existing literature, explain why your findings are or are not surprising 3.5. TJB > Simply explain how previously published literature relates to your findings, without commenting on whether it is surprising relative to findings, and without saying that your data clarifies, challenges, or expands previous literature 3.6. TJB > Identify areas of social life where your argument likely applies 3.7. TJB > Draw out implications of your findings, without referring to literature 4 – Identify possible explanations for your findings 4.1. TJB > Identify a possible explanation for your findings without referring to existing research 4.2. Use existing research to discuss the most likely explanation for your findings 4.3. Consider alternative explanations for your findings and explain (using your data and/or other research) why these alternative explanations do or do not seem plausible 4.4. Conclude by reviewing why these findings (and the larger puzzle/gap they address) are important 5 – Apply Findings 5.1. TJB > Evaluate social change strategy held by respondents or scholarship activism more broadly by referring to literature and/or through argument referencing the study’s data 6 – TJB > Discuss Limitations 7 – TJB > Future Research 8 – TJB > Generalizing Beyond Sample without Referencing Literature The titles were pretty boring and consistent — either “Discussion” or “Discussion and Conclusion”. Most of the articles didn’t have a separate “Conclusion” section. So I just lumped them together. It seems, in general, that it is conventional enough to just forego the conclusion, unless you want a dedicated place to talk about future research. Future research isn’t in Calarco’s outline. The most common move in the Discussion was to review the findings (13). Then the general “implications” move was to 3.3. Explain how the resolution of the gap/puzzle helps to clarify, challenge, or expand existing knowledge or theory (8). Sometimes, though, this main implications move was done a bit differently, and I reworded it to 3.3. TJB > Explain how implication of findings, themes and/or subthemes clarify, challenge, or expand exiting knowledge or theory (5). Then the most common moves were:
OVERALL ASSESSMENT Overall, I did find Calarco’s outline to be a good general guide for understanding the structure of journal articles. But as you can see, the structures varied widely. This is not to say that they should have varied widely. Perhaps it would indeed be better if everyone kept to the exact structure that Calarco outlined. At least in some cases of course (mixed methods or purely quantitative data), you couldn’t exactly follow that template. But something I found in these articles is that they weren’t trying to deeply change theory. If you just looked at Calarco’s outline, you might think that every article had to have a major theoretical contribution. There is this article … Besbris, Max, and Shamus Khan. 2017. “Less Theory. More Description.” Sociological Theory 35(2):147–53. … where they worry about people trying to revolutionize thinking with every contribution. That is certainly something I have learned in graduate school: that you don’t actually have to make ground-breaking theoretical contributions every time or even ever. And if you do that, sometimes you are typecast as a “theory” person, as in, a particular subfield of sociology. There are many debates about the status of theory and I’m not going to give a literature review of that here (e.g., in terms of whether it should be ever considered a sub-category of sociology or whether everyone should engage with theory construction, among other issues). But when you read Calarco's outline and it says identify a “puzzle or gap in the literature” it doesn't mean a major theoretical gap. Replicating research is, of course, frowned upon in sociology (but not in e.g., biology etc.). So it isn’t that you don’t have to find some kind of a gap. You do. But the gap doesn’t have to be a major modification or take-down of a theory. It can be an empirical gap, and you can just imply that the gap is there, while constructing an argument with important implications. I created this post (and document, see end of post) so that I would have template outlines for my drafts. I added all the quotes for the various coded parts so that I could know what I meant by a particular code. I’ve left that all here, you can take it or leave it (or modify it, or whatever). It also led to me drafting an alternative outline to Calarco's (see below). Also, all the codings in the document (attached at the bottom of this post) is a first draft. As I work with this I may re-code something or refine some kind of this structure. The document has all the details of the coding, the kind of behind-the-scenes process that you usually don’t get in qualitative coding! And people don’t give it out because they don’t want to be proven wrong etc. (which some people, like Biernacki, find a way to do anyway! Biernacki, Richard. 2012. Reinventing Evidence in Social Inquiry. New York, USA and Basingstoke, UK: Palgrave Macmillan). So the document's full coding looks messy and I’m sure you can find a way to improve the coding structure there (I know I will!). But if you want to see what I see as the outline of these articles (in a structural way) I wanted to provide all the detail, which you may find useful. I outlined more abstracts than the full papers—at the abstract phase, I was trying to figure out which I wanted a full outline for. So if you open the full document, you'll see more abstracts than the other sections. ALTERNATIVE OUTLINE Given all of this, I have made an alternative outline, that I tried to simplify. There are, of course, many other things you can add. But from my reading, these are what are most important. ***Many of these are copied from Calarco’s outline; in some cases, moves that are distinct in her outline are combined here*** Abstract (150–250 words)
Introduction – Basically it is an extended version of the abstract / a mini version of your full paper
Justification
Methods and Data
Analysis FOR QUALITATIVE ARTICLES ONLY: Have a structure something like this. ***You do not have to have a certain number of subthemes, second order subthemes, third order subthemes, etc. – do what you actually have, whatever the nested structure is for your codes.*** ***When you are coding, make sure you nest the codes so that you can write this section with ease when you get to the writing stage.*** I realize the structure here is very repetitive, but I want to show the general structure. (SEE BELOW FOR A WORD DOC WITH COLOUR CODES, for this and other sections).
Discussion/Discussion and Conclusion
FULL DOCUMENT Below, you can download the document where I did this work. You can see how I coded each passage from each of the articles. I have this in the document because it helps me see how people dealt with particular issues, which I will have to reference as I continue to write journal articles. I hope this is helpful for you! Certainly for me, randomly finding that outline from Calarco and thinking through it has been an important part of learning how to write journal articles. 
ALTERNATIVE OUTLINE This is a document with only the outline. The analysis section is for a qualitative article.
0 Comments
Your comment will be posted after it is approved.
Leave a Reply. |
Tyler J. Bateman
Academic methods blog Archives
February 2023
Categories |
||||
RSS Feed