Preview image by José M. Reyes.

Title image by Anastasiia Ornarin

“It is dangerous for a scholar even to imagine that he might attain complete neutrality, for then one stops being vigilant about personal preferences and their influences – and then one truly falls victim to the dictates of prejudice.”

Stephen J. Gould – The Mismeasure of Man

Keywords: Reflexivity, Methods, Philosophy of Science, Quantitative Science, Open Science, Rigour, Transparency

Abstract

While reflexivity is an established hallmark of good scientific practice in qualitative research, it is practically non-existent in predominantly quantitative fields. Here, I argue that an awareness and acknowledgement of researchers’ influence on their work could contribute substantially to the quality of quantitative research. Basing my argument on the widely held values of transparency and rigour and corroborating my further premises with evidence and reasoning from metascience and the history, sociology and philosophy of science, I conclude that reflexivity ought to be endorsed by quantitative researchers, too.

To illustrate the implementation of reflexivity, opportunities for its gainful application in a quantitative demonstration example are provided.

Moreover, I highlight potential challenges for reflexive practices arising from current incentive structures and epistemic injustice but also discuss how reflexivity should contribute to its dismantling.

Finally, I argue that the well-developed method of reflexivity offers a very convenient tool to quantitative scientists to strive for greater rigour and transparency.

Introduction

The open science movement has been the vanguard of methodological improvement in science. Embracing preregistration, registered reports, multi-centre replication studies, open methods and open data, it has fostered the adoption of practices enhancing the quality of research[1][2]. The movement pursues the values of rigour, transparency, openness, reproducibility, replicability, and accumulation of knowledge and has provided researchers with numerous tools to help them live up to these values [3]. Reflexivity is a well-established practice in qualitative science where researchers are expected to reflect upon how they shape the research process and to share important information on their standpoint[4]. In the following, I will argue that open science proponents conducting quantitative research should practice reflexivity as a measure for greater rigour and transparency, too. My argument will take the logical structure of a twofold modus ponens, deducing the endorsement of reflexivity from the premises that it enhances rigour and transparency and the commitment to these values.

To illustrate how reflexivity can add to a research project over its entire course, I put forward points of application by reference to a demonstration example. Furthermore, potential challenges to and opportunities for the beneficial use of reflexivity related to epistemic injustice and academic incentive structures are discussed.

Reflexivity

Reflexivity, a sensitivity to and acknowledgment of the ways in which scientists shape the collected data and research findings, is an established hallmark of scientific rigour in qualitative research[4][5]. Researchers are asked to reflect upon the impact of their ideology, values, role in society, life circumstances and identity on the subject and form of research[6]. Furthermore, researchers’ positionality relative to the subject of inquiry (e.g. having personally experienced the phenomenon) should be explored[7]. This encompasses both influences on the decisions taken by researchers and on the behaviour of participants[8]. For example, the identity of a researcher might impact a participant’s social desirability bias in answering a questionnaire or performance in an experiment[9]. Reflexivity is not exhausted in a single session of critical reflection. The objective of these measures is to generate research that is sensitive to the ways how it has been shaped by the above-mentioned factors[8]. In order for reflexivity to be effective, it ought to be a sincere and continuous process that incessantly monitors potential developments and changes of influencing factors over the course of the project. It is a deeply engaging process that motivates researchers to take action to broaden their perspective and engage with their positionalities. Moreover, reflexivity should not be exercised in private but transparently so that readers can comprehend the influences that contributed to the research[4].

In quantitative science, reflexivity is largely non-existent[10][11]. The only influential factor that quantitative researchers are expected to make transparent are financial conflicts of interest. Reflexivity fosters an examination of one’s positionality and its implications beyond mere financial interests.

The standpoint-dependence of all research

Before expounding how reflexivity could promote rigour and transparency, we should first examine why the aspects, that reflexivity invites us to reflect, are consequential to our scientific endeavours.

In 1962 Thomas Kuhn published “The Structure of Scientific Revolutions”. In this historic analysis, Kuhn demonstrated that scientific progress is determined, both in times of continuous knowledge accumulation and in times of rapid paradigm shifts, by the scientific community, a sociological group comprised of psychological beings. It is through social interactions within the community that established paradigms are maintained and new ones become accepted. Our understanding of the world results from a social process[12]. Thus, Kuhn provided the foundations for a sociology of science[13] that studied how scientific work, whether progressive or erroneous, is determined by societal factors and how this is reflected in distributions of convictions amongst scientists[14]. In the following, I will present the results of two landmark sociological studies that developed such an empirical epistemology.

Long before it had become central to the open science, Knorr Cetina discussed researcher degrees of freedom. She observed that every scientific endeavour required numerous choices, from the choice of an animal model, over arbitrations between different electron microscopes to decisions for a particular statistical method. The eventual scientific discoveries heavily depend on these choices[15]. For example, the effectiveness of glucocorticoids for spinal cord injury varies substantially depending on the choice of mouse model[16]. This leads to the question:

How are these choices made?

Based on her fieldwork in Berkeley, Knorr-Cetina argued that the context of discovery is crucial for the decisions and that this context is trans-epistemic and trans-scientific. It is trans-epistemic as factors other than truth-conducive forces are at play. Scientists strive for and are dependent on the appreciation of peers like journal editors, peer-reviewers or conference chairs. Therefore, they always have the community’s beliefs in mind when they are conducting research. What will the community find interesting? What will it consider a worthwhile problem, what an acceptable solution to that problem and which issues on the way to that solution must be addressed or are negligible? Similarly, researcher’s interest in the maintenance and advancement of their own career shapes how they evaluate the work of others. Studies are not appraised according to their merit alone but always relative to what opportunities or challenges they hold for the reader. Moreover, the context of discovery is trans-scientific as it is moulded by actors beyond the scientific community. Politicians and industry representatives have causes which they want to see addressed by science. Henceforth, they use their power over funding to negotiate how and on what scientists should work[15].

Even in meta-analyses, the supposed platinum standard of quantitative evidence, researchers face a plethora of choices: choices about which studies to include or not, choices about whole or sub-population averaging, choices between outcome measures, choices between divergent quality assessment tools and arbitrations on how to subjectively score studies on them[17]. The observed positive correlation between financial conflicts of interest and meta-analyses favouring the treatment in medical research [18][19] could very well be explained by this. Similarly, researchers’ allegiance to a psychotherapeutic school has been found to inflate the effect sizes of RCTs of their preferred school[20]. All publishing researchers are required to state financial conflicts of interest and allegiance to a therapeutic school can be discerned. Henceforth, the impact of these biases could be studied. The importance of other biases must be assumed.

Latour conducted his fieldwork at the renowned Salk Institute and analysed his observations with the sociologist Steve Woolgar. The conclusions they reached are broadly agreeing with those of Knorr Cetina. Woolgar and Latour particularly emphasised the antagonistic nature of scientific work and how this impacts the construction of scientific facts. Competing groups following conflicting research programmes are constantly working to immunise their claims against objections from their opponents. Therefore, research might be shaped by a particular method for the simple reason that it would be very expensive for others to contradict the claims made using this method. More generally, what is at all considered negative evidence falsifying a claim is not decided a priori in a Popperian manner but results from a post-hoc social negotiation in the scientific community. The degree of scrutiny that claims must undergo depends on the person of the claimant. The work of an esteemed collaborator is likely to receive a much less critical examination than that of a rival. Furthermore, Woolgar and Latour stressed that the acceptance of scientific facts arises from a multitude of social factors of which many are less salient than ideology. One such factor that both Knorr Cetina and Latour and Woolgar describe is the dependence of all novel advances in science on established methods and reified materials. Just as the contemporary innovations that take them for granted, these methods and materials have once been subject to scientific discourse and became facts through the social process of science[15][21].

The paramount importance of our reliance on these reified facts becomes clear when we bring the relevance of such background assumptions about methods and theory to hypothesis testing to our minds. Evidence contradicting a hypothesis is only contradictory if we accept the background assumptions and tests our experiment relied upon[22]. This can be illustrated with philosophy’s time-honoured example of the hypothesis that all swans are white. If we encountered a black swan successfully mating with a white swan, we could consider the hypothesis falsified. However, we could just as well argue that we should replace Mayr’s biological species concept with a morphological one as in palaeontology. This would make our hypothesis a tautology but keep it unfalsified.

At least with hindsight, it is easy to see how individual and societal ideologies shaped the choices of even the most renowned scientists. Two paradigmatic cases are Charles Spearman (Spearman correlation) and Paul Broca (Broca’s area). Charles Spearman’s work has been decisive for how intelligence is conceived. He was a fierce proponent of the concept of a general intelligence and justified his belief with principal component analysis (PCA). PCA is designed to identify one factor that explains a maximum of the variance (e.g. of cognitive test scores). Therefore, this methodological choice always produces results compatible with general intelligence. The availability and equivalence of other factor analytical approaches suggesting multi-dimensional intelligence did not change Spearman’s mind. Craniometrician Paul Broca can be seen as an early example of the abuse of researcher degrees of freedom. Broca neglected relevant control variables when his results affirmed his preconceived hierarchy of “races” and included them when uncorrected results were contradictory to it[23].

Researchers’ beliefs do not only affect their own behaviour. The impact of researchers’ expectations (or what participants suppose they are) on participants’ behaviours has been known and studied for decades and is amongst the most well-established psychological findings[24][25].

Moreover, aesthetic preferences influence the progress of science, too. Scientists are often convinced of a theory’s truth by its perceived beauty[26] but whether this is a reliable guide to empirical success is dubious[27].

In summary, scientific results are moulded by researchers’ choices and behaviours which in turn are influenced by values and convictions held by them, the scientific community and society at large. The work of others is evaluated relative to its impact on one’s own scientific programmes and career advancement and the truth-values of hypothesis tests depend on methodologies and concepts that we decided to accept at one point in history. Depending on the field of research, different influences will be of varying importance. While adherence to specific research programmes exerts influence in all fields, aesthetic preference have been identified as particularly influential in theoretical physics[27] and political commitments tend to be more relevant in social sciences (but also remember the example of “German physics” in theoretical physics[28]). The dependence of all research on the investigators’ standpoints is not to be understood as the result of nefarious activities of researchers. Rather, it is inherent to the ways in which knowledge is produced. Reflexivity asks us to carefully contemplate how this is reflected in our work. Gaining more awareness of such important factors should in itself be desirable to researchers. Reflexivity offers us a method to deal with emotionally challenging experiences in research and to gain insight into ourselves[29]. It has even been argued that situating oneself relative to one’s research is an ethical obligation[30]. I believe that it also makes our science more rigorous and transparent.

Rigorous science is reflexive

To argue for reflexivity’s contribution to scientific rigour, we should first clarify the term. A dictionary definition of rigour is: “the fact of being careful and paying great attention to detail”[31]. Similarly, in his list of qualities of good math, Terrence Tao describes rigorous research as work “with all details correctly and carefully given in full”[32]. The NIH defined the purpose of scientific rigour to be ensuring robustness and unbiasedness in scientific practice[33]. As a synopsis of these statements, we shall consider a method as rigour-enhancing when its careful attention to details leads to more robust and less biased research. This might not be a definition compatible with a qualitative ethos and methodology[34][35] but as this text is primarily directed at fellow quantitative scientists, this seems acceptable.

The brief outline of reflexivity given above should have expounded that reflexivity involves a great deal of careful attention to detail. What remains to be shown, is how this attention to researcher influence contributes to robustness or bias reduction.

Due to the inevitable empirical underdetermination of theory by data, scientists will always rely on values in arbitrating between theories to some degree[36]. Thus, absolute objectivity is unattainable. Bias reduction should be an objective, though. Philosopher Thomas Nagel argued that it is mistaken to consider objectivity and subjectivity as dichotomous. Instead, our judgements lie on a spectrum of varying degrees of dependence on our subjective perspective. Nagel further argues that if we strive for greater objectivity, we must learn how our perspective shapes our judgements and incorporate this insight into a more objective one. Relativity theory is an example of this possibility as it both explains why time and space are relative and why our more naïve perspective perceived them to be absolute[37].

We don’t need to have the ambition to leap to a revolutionary new theory of Physics to obtain greater objectivity. Practising reflexivity is exactly doing what Nagel suggests. When we understand the impact of background assumptions and reified materials in our work, we are better equipped to account for them. Reckoning with the situatedness of our thinking, we can incorporate this insight into our theories and will thereby attenuate the standpoint-dependence of our work. Reducing the biases of standpoint-dependence will make us move forward towards less subjective and more rigorous science.

Transparent science is reflexive

I have argued that reflexivity can contribute to rigour by reducing researchers’ biases. I am sceptical about its potential for total bias elimination, though and others might go as far as claiming that any attempts to reduce the impact of one’s own positionality are futile. However, even and particularly for those unconvinced by the rigour-enhancing potentials of reflexivity advertised above there is ample reason to adopt reflexivity.

Given the importance of researchers’ positionality, the contribution of reflexivity statements to transparency seems obvious. This insight helps readers to contextualise studies and opens up an opportunity for metascientific investigations into links between positionalities and research results beyond financial conflicts of interest. Yet, using Solomon's social empiricism, we can even more systematically illustrate why transparency about our positionalities is expedient.

Social empiricism acknowledges that arbitrations between competing theories in science are not made based on empirical merit alone. Additional to empirical decision vectors, Solomon identifies non-empirical decision vectors like ideology, peer pressure, deference to authority or elegance. These biases are not treated as inherently problematic, though. Only in the absence of epistemic fairness do they become a reason for concern. Thus, democratic science becomes a prerequisite to the pursuit of maximisation of empirical success and truth. Between competing theories, “Empirical decision vectors should be equitably distributed (in proportion to empirical successes)” while “Non-empirical decision vectors should be equally distributed”. Henceforth, journal editors, grant officers, science policy experts, etc. are charged with the task to balance decision vectors accordingly[38]. If scientists transparently exercised reflexivity, their work to ensure epistemic fairness would not only become substantially more manageable but non-empirical decision vectors, that otherwise would hardly have been identified, could be uncovered, too.

Reflexivity throughout the research process

To illustrate where reflexivity can enhance our research over the course of a project, I will now delineate opportunities for it. A study investigating how participants’ obesity is linked to cognitive performance via neural correlates in the field of dementia research shall serve as our demonstration example.

At the outset, a first question to reflect upon would be why we chose dementia research. When studying a phenomenon, we should state and consider our own experiences with the subject, like the role of loved ones that had suffered from dementia. Concerning the specific risk factor, our own body weight’s effect is to be contemplated: Will we be biased towards finding certain results? How susceptible are we to societal stereotypes? Furthermore, we could discuss why specific theories and research programmes attract us and in how far this results from our education and training, the scientific community or political and ideological commitments. In our example, we might ask why we chose to investigate an individual lifestyle risk factor rather than an interpersonal phenomenon like income inequality.

Wittgenstein pointed out: “The aspects of things that are most important for us are hidden because of their simplicity and familiarity. […] The real foundations of his enquiry do not strike a man at all.”[39] Some fundamental methodological or theoretical choices we learned in our scientific training have become so deeply engrained in our minds that we cease to question them. Our concept of what constitutes a disease will influence how we research dementia. Do we conceive of disease as a biostatistical deviation from a norm or is disease defined by the suffering inflicted on patients[40]? Reflexivity invites us to make ourselves aware of how these background assumptions are affecting our study design. Additionally, we could ponder how our methodological choices and faculties might lend themselves more to a research programme that believes relevant things to take place in the brain (rather than the gut or society) and what explanatory models may lie beyond the grasp of our toolkit. Refining our methodological reflexivity can thereby reveal new limitations and interactions of different systems to address.

Studies often have very unrepresentative samples leading to conclusions that are invalid beyond the limited scope of the sample[41]. When we aim for representativeness, we should also consider how our positionality affects recruitment and how undesirable effects could be attenuated. For example, being White, highly educated or wealthy might influence our recruitment strategies or reduce the willingness to participate of individuals that have been and are marginalised by people like us[42].

Additionally, we could consider how we influence data acquisition. In our example, a lean researcher at a prestigious academic institution investigating the role of obesity on cognitive performance might exacerbate internalised weight stigma[43] in obese participants during cognitive testing. Reflexivity could help us to become aware of this potential issue and in taking steps to ameliorate it. For example, consciously avoiding cues that could bring negative stereotypes to mind and creating a welcoming and safe atmosphere that counters internalised stigma could help us obtain data that is less affected by the distortion of internalised stigma and thus more valid.

Different researchers trying to answer the same hypothesis with the same data will make different reasonable processing and analysis decisions and thereby obtain often disparate results[44]. Such results from many-analyst approaches have been found in physics, physiology and social sciences[45][46][47][48]. The extent of our impact can be illustrated using a multiverse analysis in which the results of all combinations of choices are compared[49]. Preregistrations [50] are great tools to limit our ability to fiddle around with our data until we obtain a result we desire. Yet, even with this measure in place we should still be reflexive of the directions in which we explored the data and our decisions regarding pre-processing, priors, relevant effect sizes, alpha-levels, multiplicity control (Should we control FWER or FDR?), Bayesian vs. frequentist statistics, etc. as all these choices are still going to impact the results.

Then, numerous aspects of how we discuss our results should be examined reflexively. If we found an adverse effect of obesity on memory performance, we could ponder whether our positionality made us attribute this effect to stress due to discrimination, receiving worse healthcare or systemic inflammation. If we suggest solutions to maintain cognitive health, we could reflect upon why we propose societal (tackling discrimination), political (taxing unhealthy foods) or individual (dieting), biomedical (GLP-1 inhibitors) or psychological (behavioural therapy) interventions.

Throughout the research process, we can contemplate which language we use and how this affects us and our participants. Are we using the biomedical term obese or the activist self-designation fat?

Often, it is rather difficult to become aware of the standpoint-dependence of our actions on our own. Therefore, working in an interdisciplinary and diverse team that brings a multitude of perspectives to an issue is invaluable for reflexivity[51]. In research on issues ranging from HIV/AIDS over dementia and cancer to mental health, participatory research involving communities and individuals affected by the disease under study has been employed successfully. It helped identify needs and meaningful research questions, increase acceptability of trials and interventions amongst the target groups, determine appropriate outcome measures, ensure high ecological validity and uncover limitations in the existing literature[52][53][54][55][56][57]. Likewise, if we lack expertise in the lived experience of being fat, inviting experts in this could help us to improve our work.

Structural challenges to reflexivity

It has repeatedly been argued that we should expect researchers to implement open science measures. Similarly, many journals publishing qualitative studies require transparent reflexivity sections [58][59]. Throughout this article I have argued for the merits of reflexivity, but I am hesitant to endorse strict reflexivity section requirements.

The reason for my reluctance is epistemic injustice[60]. Evidence indicates its presence in citation patterns, peer review, postdoctoral fellowship applications and in the way how scientific innovations are taken up by peers[61][62][63][64][65][66][67][68][69][70][71]. Moreover, science has a long history of discounting and neglecting the advancements made by women (“Matilda effect”) and scientists from other marginalised groups like Charles H. Turner, Lise Meitner or Rosalind Franklin[72][73][74].

These injustices are grave ethical issues and concerning for purely epistemological reasons, too. The insufficient study of non-default populations is well documented in multiple fields of inquiry and frequently results in incorrect generalised inferences[41][75][76][77]. Moreover, Donna Haraway suggests that “there is good reason to believe vision is better from below the brilliant space platforms of the powerful” because an awareness of the situated nature of all science is more likely amongst those individuals whose identities are regularly disregarded. Therefore, they are less prone to believe in the existence of an unlocated perspective[78] and more likely to obtain representative samples[42]. Furthermore, Sandra Harding has argued that their perspectives are particularly fruitful as they provide problems of particular significance that have been neglected before and raise critical questions: questions about received belief[79]. The work of the female cytogeneticist Barbara McClintock provides an example of this. Her distinct methodology and holistic conception of the cell challenged the dominant central dogma of molecular biology with the DNA in the role of the master molecule. Thus, she was able to make significant contributions to her field based on an understanding of genetic mechanisms constituted of multi-molecular functional interrelationships[80]. In line with this, there is evidence showing higher rates of innovation in the work of PhD students from minority groups.[61]

Science benefits from diversity but scientific culture does not foster diversity’s flourishing.

As reflexivity necessitates an analysis of power[30], it should lead researchers to acknowledge epistemic injustice, the epistemic privileges of marginalised researchers[42] and the vitality of diverse, representative scientists and samples[81]. Emirbayer and Desmond argued that reflexivity is pivotal for scientists to enhance our thinking on and tackling of racial injustice[82] and Hadley et al. proposed reflexivity as a method to advance from merely paying lip service to a commitment to diversity[83].

However, epistemic injustice poses a challenge to reflexive practices, too. Given that discrimination is still evidently prevalent in academia, having to discuss one’s identity alongside one’s research could yield a greater potential of repercussions for researchers from marginalised groups. Furthermore, reflexive science is often considered “soft” by other researchers[30] which could be a particular concern for researchers experiencing sexist discrimination. Henceforth, while we should harness the potential of reflexivity to counter discrimination, all attempts aimed at establishing a reflexive culture in quantitative science must be conducted in a manner sensitive to it to avoid aggravating existing injustices.

Further challenges to reflexivity arise from the academic incentive structures. Reflexivity demands time and effort, resources that could otherwise be spent on producing more papers. Researchers competing for academic positions find themselves under immense pressure to publish as many articles as possible because they are primarily measured against this yardstick[84]. In contrast to this, reflexivity is more aligned with slow science that cherishes quality over quantity[85]. Moreover, reflexivity does not fit the textbook successful research article. We are supposed to tell a coherent story that linearly leads to the main conclusion and from which unnecessary details and distracting aspects have been expunged[86]. Reflexivity does not require bad prose, though. Great literature always has nuance. Learning about the research process can enrich any manuscript but it will not provide us with a simple and straightforward storyline. As with reflexivity, we often find that incentives for individual researchers do not align with societal interests in high-quality research[87]. Yet, recent innovations like the Hong Kong principles for assessing researchers that reward rigour and transparency[88] give hope that not only the research but also the researchers will benefit from going the extra mile of reflexivity.

Conclusion and Outlook

I have proposed reflexivity as a new tool to enhance rigour and transparency in quantitative studies. Given the evidence for the impact of researchers’ positionality on their work, I have discussed how the tool can help us to live up to these values. If one champions rigour and transparency and is convinced by reflexivity’s contribution to rigour and/or transparency, one should endorse reflexivity.

Reflexivity should be incorporated in open science toolkits because of the values it corresponds to and its potential synergies with more established open science methods. An awareness of our standpoint should encourage preregistrations and registered reports as pre-emptive measures against motivated reasoning in analysis. A multiverse analysis[49] might motivate us to practice reflexivity and an awareness of the importance of our analytical choices for our results in turn might prompt us to employ multiverse analyses or triangulations[89]. Adversarial collaborations can be a fruitful endeavour for researchers with conflicting standpoints[90] and reflexivity could initiate such cooperations. Furthermore, reflexivity can be considered a systematization and conflation of many more small-scale initiatives to reform science from calls for declarations of diets as conflicts of interest in nutritional research[91] and political commitments in psychiatry[92] to efforts to counteract biased citation patterns [68]. Yet, reflexivity is not merely a tool to synthesize and encourage the application of other open science methods. It goes beyond what has thus far been covered by open science. Regarding adversarial collaborations, Kahneman, who coined the term, pointed out that most such projects do not lead to changed beliefs as the adversaries still disagree about the interpretation of the experiments[93]. Here, reflexivity can be pivotal to help us understand the remaining differences. More generally, it addresses some fundamental questions: Which analogies are used to make sense of the data, which questions are being raised and answered and which theories are chosen to explain phenomena?

As policies demanding transparency about financial conflicts of interest are uncontested, it seems almost insulting not to discuss other influences on our work. It is as if we could only be motivated by greed.

Writing about reflexivity irreflexively would be silly. Henceforth, the interested reader can find a reflexivity statement in the appendix.

I have argued that reflexivity can contribute to bias reduction but a “view from nowhere” is likely to be unattainable. Rather we should conceive of reflexivity as an evolutionary process leading to improved but still standpoint-dependent perspectives [37] [93] . Therefore, it is important to keep in mind that practising reflexivity is essential to dealing with the standpoint-dependence of science but does not completely dissolve the issue[94]. While pursuing reflexivity for the sake of either rigour or transparency is worthwhile, pursuing both values simultaneously is synergistic. Being transparent helps us to handle the unfeasibility of complete objectivity and striving for rigour pushes us to convert our insights into improved theories and methods and thus prevents us from descending into mere self-indulgence[95].

A potential point of criticism of my argument is its conservative axiology committed to a value of objectivity strongly linked to unbiasedness. This is not in line with the value systems of many schools of qualitative research [34]. Furthermore, some quantitative scientists like the statisticians Gelman and Henning[96] or the epidemiologist Greenland[97] have called the value of objectivity into question, too. Yet, these alternative axiologies centred around virtues like transparency and awareness of context dependence and multiple perspectives can only embrace reflexivity more rhapsodically.

In the development of a quantitative reflexive practice, we should pay special attention to discrimination. Reflexivity can enhance our efforts to tackle injustices in and outside of academia but we should also be cognizant of the fact that being reflexive in public is demanding more from some than from others.

Remembering to tread carefully should not impede us from harnessing the potentials of reflexivity, though. As quantitative scientists, we have ample reasons to endorse reflexivity and moreover, this conclusion is also very convenient for us. We do not have to reinvent the wheel. While the open science movement has displayed extraordinary inventiveness in the development of more sophisticated methods for quantitative scientists, here we are offered the opportunity to improve our methodologies by simply adopting a tool that has already undergone decades of development and refinement. We only have to bite the bullet and accept that there is something to learn for us from our qualitative colleagues. Introductions for doing so in the qualitative literature cited above and in the primer by Jamieson et al.[98].

At some point, reflexive methodologies specifically tailored to the problems quantitative scientists are confronted with should be developed. For a start, we could simply take advantage of the experiences of qualitative science.

Appendix

My appreciation of the method of reflexivity has itself resulted from a reflexive analysis of how and why I choose and evaluate research tools.

Being socialised as a boy with the associated stereotype of mathematical aptitude[99], in a society and the discipline of Medicine that largely equate quantification with good scientific practice[100], and having some inherent inclination towards numbers, I was clearly steered towards applying quantitative methods to answer my research questions. When I ventured into my scientific career not so long ago, I did not only have an affinity to quantitative approaches but also held a derogatory view of all non-quantitative methodologies. The lack of statistics as a remedy against subjectivity made me consider qualitative approaches as deficient. I believed that using statistics would absolve me from my personal perspective and help me pursue science as committed to impersonal truth[101]. Becoming aware of researcher degrees of freedom[102] and engaging with the history, sociology and philosophy of science, I had to accept that this had been an illusion. In light of the limitations of my own methodology, I gained a greater appreciation of the practices of friends conducting qualitative science and became interested in how they dealt with researcher-dependence of their studies. I am still very fond of quantitative methods and consider them to be invaluable tools for science, but I also believe that they can be augmented by methodologies from other scientific traditions.

In line with my general political convictions, texts in feminist epistemology have been particularly influential in this process. Additionally, the input of my friends in qualitative research, patiently explaining their methodologies and the reasoning behind them to me has been of paramount importance to me.

In writing this manuscript, I have particularly struggled with two issues. Being very privileged, I have not encountered a great deal of discrimination in my scientific career and life in general. Therefore, I felt unease addressing related potential repercussions of reflexivity. Discussing the topic with numerous colleagues with different experiences of discrimination helped me to develop a better grasp of the issue and refined my treatment of the concerns in the text. Yet, I believe that a broader discussion of this issue with contributions from multiple perspectives is warranted.

Writing this very reflexivity section has been quite a challenge, too. I was afraid that exposing my process could make my argument less persuasive for some readers. Furthermore, having been trained in quantitative methods, I doubted my own reflexive prowess. Developing competence in reflexivity is an enduring task, and I am at the beginnings of it.

Acknowledgement

I would like to thank all friends and colleagues whose thoughtful considerations have helped refine my thoughts on this issue. I am most grateful to Frauke Beyer and Veronica Witte for their valuable comments on earlier versions of this manuscript.

Data availability statement

Data sharing is not applicable to this article as no new data were created or analyzed in this study.

Competing Interest Statement

The author declares no competing interests.

Funding

The author did not receive funding for this article.