Should Language Acquisition Researchers Study “Grit”? A Cautionary Note and Some Suggestions

Grit is theoretically defined as the combination of perseverance and passion for long term goals. Both of these constructs are likely to be relevant for our understanding of how language acquisition occurs and for explaining between-person differences in the rate of language acquisition. Despite this relevance, there are methodological and theoretical reasons why language acquisition researchers should be cautious about studying “grit” as a construct that is predictive of or causally related to language acquisition. In this paper we discuss some of these reasons, with a specific focus on the problems associated with the aggregation of perseverance and passion into a single variable, and the lack of predictive validity for other important life outcomes. We also discuss and describe with examples other challenges involved in studying grit, passion, or perseverance. Finally, we offer suggestions for some potentially more fruitful ways in which perseverance and passion for long-term goals may be integrated into research on second/foreign language acquisition. For example, we discuss how the measurement of grit facets may need to be revised to be better aligned with the “persisting despite initial failure” theoretical definition of perseverance, and to also balance the negatively-worded and positively-worded item content of the scales. We also discuss how an examination of necessary-but-not-sufficient relationships between grit facets and language acquisition using Dul’s (2016) methodology may be particularly valuable. That is, perseverance and passion may both be required for successful language acquisition but be insufficient on their own because other variables also need to be present (e.g., opportunity to practice, feedback).


INTRODUCTION
Research in the area of second-language acquisition (SLA) has long featured a role for individual differences, including aptitude, motivation, and affect (Dörnyei & Skehan, 2003;Li et al., 2022).Recently, researchers interested in SLA predictors and processes have begun to examine variables that originate within the positive psychology movement.These include variables such as growth mindset, hope, optimism, and resilience (see MacIntyre, Gregersen, & Mercer, 2019 for a recent review), but has also extended to the well-known construct of grit (e.g., Khajavy et al., 2020), that was popularized by Angela Duckworth and colleagues (e.g., Duckworth et al., 2007;Duckworth & Quinn, 2009).
In this paper we aim to provide researchers in the psychology of language learning with a critical overview of grit in the hope that this informs the manner in which the grit construct is incorporated into future research.Our discussion falls into two broad sections.First, we briefly review the challenges and problems that have been identified by researchers who have examined grit in the broader psychological and educational context.Second, we offer our view of how the concepts that underpin grit may be most fruitfully applied to the general attempt to understand SLA.

GENERAL CHALLENGES
Grit is theoretically defined as a higher-order construct with two components: perseverance and passion (often also referred to as "consistency of interests").Researchers became interested in grit as a construct because of widely publicized claims (e.g., Duckworth, 2013, Scelfo, 2016), that grit represented a novel psychological construct and that it was highly predictive of success and performance in a wide variety of contexts ranging from the classroom, military settings, occupations, and even spelling bees.
For the most part, these claims about the structure of grit, the novelty of grit, and the predictive value of grit have not found empirical support (Credé, 2018;Credé et al., 2017).The claim that grit exhibits a higher-order structure is almost entirely based on a test of such a structure presented by Duckworth and Quinn (2009).Unfortunately, the model presented by these authors is statistically unidentified at the higher-order level (see Credé, 2018 for a full discussion of this issue) such that the results actually support treating perseverance and passion as two distinct constructs rather than a single unitary construct.That is, the inference that a higher-order grit factor exists is based on a statistical error.Unfortunately, this error involving an unidentified higherorder model has also been made in the SLA literature (e.g., Sudina et al., 2020), but researchers should be aware that there is no factor-analytic evidence in favor of a higherorder model.That is, perseverance and passion should be retained as distinct constructs in investigations of their role in SLA.
Researchers should also be aware that there is little evidence that grit represents a novel psychological construct.Meta-analytic evidence (Credé et al., 2017) and other investigations (e.g., Rimfeld et al., 2016;Schmidt et al., 2018;Steinmayr et al., 2018) have repeatedly demonstrated that grit itself-and also the perseverance facet-are largely isomorphic with the well-established personality trait referred to as conscientiousness (for an overview of consciousness as a personality trait, see Roberts et al., 2009).That is, grit explains almost no incremental variance in important outcomes such as performance in academic settings after controlling for conscientiousness.Metaanalytic syntheses of the grit literature have also shown that grit is a poor predictor of performance and success in its own right, and that it predicts success in academic and work settings far more poorly than other well-known predictors (see Credé et al., 2017& Credé, 2018 for discussions of these comparisons).Researchers should also be aware that some of the original published statements about the predictive value of grit made by Duckworth andcolleagues (e.g., Duckworth et al., 2007, Duckworth &Quinn, 2009) were based on the authors' misunderstanding of odds ratios and probabilities such that effects were at times exaggerated almost 30-fold.
One other, in our view, major problem with the way in which grit proponents have conducted research relates to the manner in which grit is measured.Duckworth et al. (2007) and Duckworth and Quinn (2009) described a 12-item grit scale and 8 item-grit scale respectively.It is our view that these scales are psychometrically of poor quality and should not be used or adapted.Two primary problems are evident.First, in both the regular and shortened version of the scale the content of the scale (i.e., perseverance versus passion) is perfectly confounded with the direction of item-wording.That is, all perseverance items are positively worded (e.g., "I finish whatever I begin"), while all passion items are negatively worded (e.g., "I often set a goal but later chose to pursue a different one").Negatively worded and positively worded items are well-known to result in artifactually distinct factors (e.g., Idaszak and Drasgow, 1987;Schmitt & Stults, 1985), and the use of these scales therefore makes it almost impossible to correctly estimate the size of the correlation between the two constructs.Second, the content of the items-particularly for the short scale-are very poorly aligned with the theoretical definition of the targeted constructs.Duckworth et al. (2007) define grit as "maintaining effort and interest over years despite failure, adversity, and plateaus in progress" (pp.1087-1088), but this core element of persisting despite failures and setbacks is not well represented in the items.By our count only one of the eight items in the short scale even attempts to reflect the definition ("Setbacks don't discourage me").The item is unsatisfactory because, in addition to the word 'setbacks' whose meaning might be problematic for some respondents, it contains a double negative statement which can be very difficult for respondents to answer in the intended direction, especially for novice or intermediate language learners.We therefore urge researchers to not simply use or adapt these scales but to rather try to develop new scales to ensure that the items reflect the intended construct.At a minimum, new measures of grit and perseverance should better assess the manner in which individuals respond to initial failures and difficulties.

RECOMMENDATIONS FOR SL RESEARCHERS
We recognize that perseverance and passion represent interesting and potentially important areas of inquiry for researchers interested in the processes that underpin SLA.Our prior work on more general performance processes allows us to offer some suggestions to researchers in this domain that we hope may be useful.In the following paragraphs we describe specific types of biases and potential threats to interpretating findings related to grit that researchers might wish to avoid, including survivor bias, Simpson's paradox, Berkson's paradox, and the specificity or contextualization of grit within the language domain.
First, we hope that future qualitative inquiries into the role of either grit facet (consistency of interest or perseverance of effort) on SLA processes should ensure that the perspectives and experiences of individuals who struggled or failed to acquire second-language skills are considered together with those who succeeded.Positive psychologists have sometimes only considered the experiences of those who succeeded at an endeavor, but such an approach is prone to survivorship bias.For example, those who successfully acquired second-language skills may rate themselves highly on perseverance and passion but this does not necessarily mean that these attributes are causally related to-or even predictive of-the development of language proficiency.Perhaps those who did not succeed also exhibited high levels of perseverance and passion, and their lack of success can be traced to other variables.The successful author J.K. Rowling is sometimes held up as a demonstration of the importance of having high levels of grit because she persevered through personal hardship and repeated rejections by publishers.However, many other aspiring authors who failed to become successful probably exhibited these same levels of perseverance.A failure to formally examine the characteristics, behaviors, and experiences of both those who succeed and those who fail to succeed may lead researchers to overestimate the value of variables such as perseverance.
We also urge SLA researchers to not combine data from participants who attempt to engage in SLA for substantially different reasons because a decision to combine such data may result in incorrect conclusions being drawn due to Simpson's Paradox.Simpson's paradox refers to the potential for statistical trends at one level (e.g., a diverse group of learners) to reverse themselves at another level (individual learners).Two hypothetical examples will hopefully serve to illustrate this concern.First, consider two groups of individuals attempting to learn Spanish.The first group is comprised of high-school students who are satisfying a language requirement at their school, while the second group is comprised of adults who have a passionate interest in the culture of a country in which Spanish is widely spoken, who have recently moved to that country and are engaged in full language immersion.It is perhaps not unreasonable to assume that the level of passion for SLA after some amount of time has passed is likely to differ dramatically between these two groups, and that the level of L2 competency between these two groups is also likely to be quite different after the same amount of time spent in language learning.Passion may be predictive of SLA in both settings but the size of the relationship may be dramatically exaggerated if data from the two groups is combined.We demonstrate this in Figure 1.Here, the correlation between passion and language skill is r =.39 in both samples, but the correlation rises to r =.84 when the two samples are combined.This much larger correlation is an artifact of the decision to combine data from the two groups.
A similar mechanism can also distort correlations in the opposite direction.Consider two new hypothetical groups.Again, the first is a group of high school students who are taking a Spanish class to satisfy a language requirement.The second group is comprised of high school students whose parents have just moved to a Spanish-speaking country such that these students are immersed in a Spanish language environment.Here the level of passion might be largely similar (and low) across groups while the language immersion is likely to result in faster SLA for the second group.Here too, aggregating across the two groups would result in inaccurate inferences, and we demonstrate this in Figure 2.Here too the correlation between passion and language skill is r =.39 in both individual samples, but the correlation falls to r =.20 when the two samples are combined.Again, the smaller correlation is an artifact of the decision to combine the two groups.
SLA researchers, who often utilize naturally occurring groups of learners, should also be aware of Collider Bias (sometimes referred to as Berkson's Paradox) when examining the facets of grit.Berkson's paradox occurs when spurious correlations are observed, or meaningful correlations are obscured, by conditional probabilities related to group membership.Imagine that a researcher is interested in the relationship between passion for the Portuguese language and the development of Portuguese language skills.To examine this, the researcher surveys a group of college students enrolled in a Portuguese language class on their level of passion and obtains the grades in the class from the instructor.The potential problem with this approach is that being a member of this highly selective college class is likely to be correlated with both passion for the Portuguese language and also with skill in that language prior to enrolling.The overall result is that the sample is likely to largely exclude students who have both little passion for Portuguese and skill in the language.This can not only result in an artifactual reduction in the size of the correlation but can actually result in observed correlations between passion and language skill that are in the opposite direction to what the real causal effect of passion happens to be.
There has been good evidence that contextualizing individual difference measurements may improve the predictive validity of such measurements.For example, Lievens et al. (2008) demonstrated that measuring personality with items asking respondents to indicate their personality at work results in personality scores that are far more predictive of job performance than scores obtained from a personality scale that does not include reference to the work context.SLA researchers have taken this insight to heart and examined the value of adding a languageacquisition context to the study of grit (e.g., Teimouri et al., 2020).Although we are generally in favor of such contextualized measurement we also wish to offer two concerns that SLA researchers may wish to consider.First, individuals early in their language learning process may not be able to accurately respond to questions about their level of perseverance or passion in a SLA context simply due to lack of experience.Second, there is some evidence (e.g., Lex et al., 2020) that passion for a particular topic or activity sometimes only develops after some level of expertise has been developed and success has been experienced.That is, individuals may not initially have passion for a secondlanguage but acquire consistency of interest and passion for second language learning as they progress, based on their orientations and mindsets for language learning (Khajavy et al., 2021).Language learning is a long-term, iterative process and it is an empirical question whether passion may be more strongly an outcome of developing expertise rather than a predictor of language development.SLA researchers who are interested in exploring the role of passion and perseverance may also wish to examine whether these two variables represent necessary-but-notsufficient conditions for language learning.Recent methodological advances, specifically the development of Necessary Condition Analysis (Dul, 2016;Dul et al., 2020), would allow researchers to examine such non-linear relations given the sample.The greater the area of empty space in the upper corner, the more X limits Y (see Tynan et al., 2020 for examples involving academic performance criteria).Plotting the relation between two continuous variables, X and Y, will produce a data pattern in which there is notable empty space in the upper left corner when X is necessary-but-not-sufficient for Y.The degree to which X is necessary-but-not-sufficient for Y is indicated by the area of the empty space in the upper left corner and the resultant effect size (d)-demonstrating that low scores on X not only make high scores on Y unlikely, but impossible, Consider Figure 3 as a hypothetical small sample size example.Here, the pattern suggests that moderately high levels of passion may be a requirement for high levels of language achievement, but high levels of passion do not guarantee high levels of learning.That is, there are no observed cases where passion was low and SLA was high.
In general, both low levels of passion or perseverance represent barriers or bottlenecks for moderate to high levels of language achievement.Alternatively, a number of other contextual, instructional, and individual difference variables may also represent such bottlenecks as plausible necessary-but-not-sufficient conditions (Li et al., 2022).Low levels of many such variables make language development more difficult, and high levels make learning more possible but not guaranteed unless minimal thresholds of other necessary-but-not-sufficient conditions are also met.

CONCLUSION
We are deeply skeptical of the claim that grit, as a unitary construct formed by combining scores on perseverance and passion, holds much value for researchers focused on SLA-or any other domain.We are more enthusiastic about the argument that perseverance and passion are psychologically interesting constructs and that they may hold particular value for researchers interested in predicting and understanding language learning.We hope that our paper has explained both our concerns about grit, some of the challenges that SLA researchers should consider when examining grit and passion, and some research approaches that may hold value for researchers who work in this and related areas.

Figure 1 .Figure 2 .
Figure 1.Demonstration of Artifactual Increase in Correlation between Passion and SLA due to Simpson's Paradox

Figure 3 .
Figure 3. Demonstration of Data Distribution Observed for a Necessary-But-Not-Sufficient Condition