The LSGSC Planning Committee is committed to making our review process more transparent and supportive for both authors and reviewers. Our goal has been to foster a culture of constructive reviewing and thoughtful feedback in service of two primary aims of our conference: to build community among graduate students and to support professional growth.
This year, we asked authors to provide (1) notes on the status of their work and (2) guidance to reviewers about the nature of the feedback they are seeking. We are asking reviewers to take these statements from authors into consideration as they craft their reviews. We believe the review process can be a dialogue between scholars.
Below we offer suggestions and supports for reviewers that we hope will be useful—and will prompt further discussion amongst members of our community about a common aspect of our work as scholars. Like many of the abstracts submitted to our conference, the guidance we provide here is a work in progress. We welcome your feedback.
The review process is a learning opportunity for both authors and reviewers! Reviewers can return to key relevant literature, think with others in the field (including their advisors who may be resources for participating in a review process), and reflect on what they know regarding theory, methods, and argumentation. Further, by communicating their feedback to authors in writing, reviewers are practicing putting ideas down and crafting arguments. For authors, the review process can be an opportunity to hear from a reviewer how their ideas are being heard. Many of the submissions for this conference are works in progress; reviewing work at this particular stage should aim to offer guidance to improve the work with acknowledgement that analysis may not be final or complete. For additional thoughts on what a review process might achieve, see Wang’s 2018 editorial in Human Development.
Each corresponding author is asked to review approximately two abstracts for each item they submit. Our aim is to assign reviews on topics/methods that you’re comfortable with and knowledgeable about, such that feedback can be most valuable to the authors whose work you’re reviewing. We group abstracts by similar topic based on selections authors made in the EasyChair submission portal. Once we put abstracts into subgroups, we pay attention to keywords and titles, as well as read through abstracts. Although it’s difficult to ensure perfect matches, we do our best to assign abstracts to review based on your own abstract topics and/or methods. We do not assign reviewers from the same university to try to maintain anonymity.
The Reviews & Submissions Planning Committee, a group within the larger LSGS Conference Planning Committee, goes through all submissions and assigns two abstracts to each person who submitted an abstract of their own. This year, the Reviews & Submissions Planning Committee is comprised of graduate students at University of Wisconsin–Madison, University of Illinois Urbana-Champaign, and Northwestern University.
Ideally, the review process presents an opportunity for you to reflect on others’ work in addition to your own. Given the breadth of submission topics and methods, there is no one-size-fits-all approach to reviewing. Here are a few suggestions to get you started.
Give yourself enough time to read the abstract carefully and to offer constructive, actionable guidance to the author. (We recommend forty-five minutes, especially if you are new to reviewing abstracts for conferences.) Your review should reflect on:
When reviewing an abstract, it may be tempting to lead with and focus on criticism and critique. Instead, we suggest an approach common in ethnographic research that involves alternating between emic and etic stances, that is listening to understand an author on their own terms and, then, applying your own perspectives (e.g., Rogoff, 2003). For instance, it may be helpful to an author (and to you as a reviewer) if you try to reflect back what you think the author is communicating. Focus feedback on what you find to be the most salient aspects of the abstract. After summarizing for the author what you took from their piece, acknowledge both stronger and weaker points of the submission from your perspective as a reader, ideally with recommendations for how to strengthen the abstract.
While we are facilitating an anonymized review process, to help foster transparency, accountability, and a greater sense of community, you may choose to sign your review with your name so that the author(s) may contact you if they would like to follow up with any questions.
Please let us know by replying to the email you received via EasyChair! We will do our best to re-assign the abstract and find another abstract for you to review.
Below we have some example reviews and commentary on what kinds of helpful and less helpful feedback these reviews provide.
This abstract describes a qualitative study on students’ use of a computational model of climate change and makes the claim that use of the model enhanced students’ abilities to use NGSS scientific practices. The authors engage with appropriate literature but might also consider connecting their argument to theories of conceptual change (see papers by diSessa, Vosniadou and others) to align discipline-based research on scientific practices with more cognitive perspectives. More detail about the demographics of the student participants and school context would make the results more contextualized so that the generalizability of findings can be better assessed. The author made causal arguments that student behaviors were different after using the model but did not provide sufficient evidence on what student behaviors were like before using the model nor did they use a control group, which limits the usefulness of the study. Reducing the causal claims would provide more nuance and accuracy.
This review begins with a reviewer’s summary of the key purpose of the piece. The reviewer then offers constructive criticism that includes a suggestion for a useful concept from research literature and recommendations for further reading.
Also, the review focuses on the core contributions of the abstract, rather than only offering suggestions on word choice or grammar. While word choice or grammar are important in communicating ideas (and having a reader understand and trust you as an author), a reviewer should try to provide feedback that focuses on the most substantive contributions an author might make.
To improve this review, the reviewer might have reflected back to the author the novelty and contribution of the submission (or recommend clarifying this point).
The reviewer also offers advice on what could be added to the submission but does not offer any recommendations for what could be removed from the submission or why it could be removed; authors have limited space, so helping them prioritize what pieces to include over others is more beneficial than only requesting additional information.
Not a huge theoretical contribution but some new information on mathematical achievement and its relationship to student identity.
This review succinctly highlights how the submission is situated in the field from the perspective of the reviewer; however, it is very short and does not offer much in the way of actionable recommendations for the author. In general, rather than writing, “You don’t have…,” a reviewer may instead suggest, “You may want to consider...and here’s why…” This review could be improved by offering 1) the author’s summary of the abstract, to indicate that they carefully read and understood all parts of the abstract (and so the editor or review committee can quickly understand the abstract); 2) additional literature that the author could engage with and/or ways to better articulate the theoretical contribution; and 3) some discussion of how sound the methodology and analyses were and how the author might improve these further.
I appreciate that your work addresses equity. I do have some concern about how you write about “diversity,” however, and believe it has implications for how we as a field might work toward equity At times you label students as “diverse” and position diversity as a problem to be solved as you describe opportunities and challenges in scaling up an intervention. Stepping back, it seems that the central concern of the abstract is how to implement a top-down approach across a range of contexts to achieve a kind of sameness in literacy instruction. I appreciate that you are clear and consistent through your abstract about this goal and approach. However, I disagree with a framing and an overall approach that seems to view diversity as a problem to be solved. A number of scholars (e.g., Rosebery & Warren; Medin & Bang; Vossoughi & Espinoza; Calabrese Barton & Tan; Lee; Nasir; Philip; and others) have addressed this point far better than I might here. I hope you engage with this work and reflect further on the value of heterogeneity—especially in relation to visions for equity.
This review reflects use of emic and etic stances by acknowledging both the author’s intentions and the reviewer’s interpretation of the aim of the abstract. In order to suggest ways to improve the abstract, the reviewer recommends specific changes in framing of a research problem. The reviewer also respectfully expresses disagreement and recommends relevant literature; this is an example of a kind of dialogic interaction that a review can facilitate. In addition to addressing the framing of the piece, it would be helpful if the reviewer had also more explicitly commented on how this alternative framing might also affect the results of the study.