International Journal of English for Academic Purposes: Research and Practice

Comparing contract cheating papers and L2 university student papers using lexical complexity analysis: An exploratory study

International Journal of English for Academic Purposes: Research and Practice 2021, 119–145.

Abstract

Instructors typically use widely available plagiarism detection tools to uncover overlapping vocabulary with other texts; however, they lack empirical evidence on measures differentiating between student writing and outsourced papers. Contract cheating is defined as purchasing custom-made assignments with the intention of submitting them for course credit. In this exploratory study, disparities between student papers and contract cheating papers are examined using a computational tool, the Lexical Complexity Analyzer (Ai & Lu, 2010) that automatically identifies twenty-five lexical complexity features (e.g., noun and verb variation). Results show statistically significant differences between L2 student papers and contract cheating papers in features of lexical density and lexical sophistication, supporting instructors’ intuitions about differences in the lexical make-up of papers written by students versus contract cheating staff. At the same time, tools such as the Lexical Complexity Analyzer provide valuable information for instructors through quantitative measures, which effectively reveal features of lexical variation that could not otherwise be obtained.

Comparing contract cheating papers and L2 university student papers using lexical complexity analysis: An exploratory study

Abstract

Instructors typically use widely available plagiarism detection tools to uncover overlapping vocabulary with other texts; however, they lack empirical evidence on measures differentiating between student writing and outsourced papers. Contract cheating is defined as purchasing custom-made assignments with the intention of submitting them for course credit. In this exploratory study, disparities between student papers and contract cheating papers are examined using a computational tool, the Lexical Complexity Analyzer (Ai & Lu, 2010) that automatically identifies twenty-five lexical complexity features (e.g., noun and verb variation). Results show statistically significant differences between L2 student papers and contract cheating papers in features of lexical density and lexical sophistication, supporting instructors’ intuitions about differences in the lexical make-up of papers written by students versus contract cheating staff. At the same time, tools such as the Lexical Complexity Analyzer provide valuable information for instructors through quantitative measures, which effectively reveal features of lexical variation that could not otherwise be obtained.

Introduction

The current research is motivated by direct classroom experience at a major urban university in Southern California where instructors have witnessed incidents of undergraduate English for Academic Purposes (EAP) students handing in written work vastly different from previously submitted writing in the same course. In these instances, instructors, relying on their intuition primarily based on the perceived lexical profiles of the two texts, suspect some form of academic dishonesty, specifically contract cheating. However, they lack empirical evidence beyond what plagiarism recognition software, such as Turnitin or Dupli Checker, are able to provide. As these software are only able to track overlapping vocabulary between the two papers, contract cheating, which is the bespoke completion of an assignment by an individual other than the student (Clarke & Lancaster, 2007), cannot be detected through these tools. Instructors need more sophisticated empirical data on patterns of lexical differences between the suspect pieces of writing and students’ prior work in order to support their teaching as well as aiding the conversations with students regarding this behaviour.

This paper presents an exploratory study that compares papers written by English as a Second/Foreign Language (L2) undergraduate university students and by workers at contract cheating sites using a freely available online lexical complexity analysis software that provides quantitative, empirical evidence of the differences. First, we briefly discuss studies that compare L2 and native English speaker (L1) writers as well as L2 writers of varying proficiencies, focusing on grammar and lexis. This is followed by a detailed description of contract cheating as an increasingly popular form of academic dishonesty. The subsequent sections describe the methodology and the software used for the comparison, and the findings. Finally, we highlight pedagogical implications, discuss limitations, and point to further directions.

Grammar and lexis in second language developmental writing

Examining the literature on L2 writing development is critical because instructors faced with possible cases of contract cheating must compare writing they have already seen from a given student with work they suspect has been completed by another individual. Studies of the past decades have compared L2 to L1 writing from multiple and various perspectives, examined L2 writers of varying proficiencies to determine markers of development, and examined errors in L1 and L2 writing.

In describing differences between L1 and L2 writing, Silva’s (1993) early work reviewing seventy-two studies is seminal, as it provides a summary of findings up to that point in time. One of the claims which can be made is that L2 writers demonstrate more difficulty in using background texts and paraphrasing, and that L2 writing is generally less complex, less mature, and less academic than L1 writing. Research up until the early 1990s also reported that L2 writer texts contain fewer modifiers and exhibit less lexical variety and less lexical sophistication.

Over a decade later, Crossley & McNamara (2009) found that L1 writers demonstrate higher levels of hypernymy (specificity of vocabulary) and polysemy (the use of words with multiple meanings). Both measures provide evidence for larger lexical networks, which means that L1 writers tend to have larger vocabularies than L2 writers. Furthermore, the connections among those words are richer and more complex, allowing for new lexical items to be learned more easily. Adding to this idea, they also reported that L1 writers generally use words with higher ‘meaningfulness scores’, meaning lexical items these writers tend to choose ‘have stronger associations with other words and concepts’ (p. 130). At the same time, L2 writers employ higher amounts of words that are generally more frequent than those of L1 writers, making their writing seem less lexically sophisticated. Given the findings of this study, it is clear that L1 and L2 texts can be successfully distinguished based solely on lexical measures. In a subsequent study, Crossley & McNamara (2011) found that in addition to hypernymy and polysemy, lexical diversity and stem overlap (the repetition of words sharing a stem, such as text and textual, from one sentence to another for cohesion purposes) also show differences between L2 and L1 writers, though these measures were similar among L2 writers with different L1 backgrounds.

Other, more recent studies, such as Staples and Reppen (2016), have examined the writing of first-year undergraduate L2 students with regard to lexical, grammatical, and lexico-grammatical structures (e.g., type/token ratio, pre-modifying nouns, noun/verb + that clause). The researchers looked at differences in students’ academic writing with varying L1 backgrounds and in different genres, and found that type/token ratios marking lexical density in a text varied between the L1 English and L2 English groups. In other words, L2 writers were found to repeat already used vocabulary items in their writing rather than employing different words or synonyms for the same concept, resulting in lower type/token ratios. This indicates that L2 writers used a smaller variety of vocabulary than L1 writers in their corpus of student writing.

Another group of researchers were interested in examining L2 student writing across English proficiency levels. Grant & Ginther (2000), for example, explored how lexical, grammatical, and clause-level features could distinguish among three proficiency levels of L2 university-level timed writing. They found correlations between higher L2 proficiency and higher lexical specificity, use of conjuncts, nominalizations, essay length, modals, verb tense variation, subordination, and passive constructions. Their conclusions indicate that higher proficiency L2 writers tend to exhibit more features associated with L1 writers than their less proficient peers.

As for errors in L2 writing, Silva (1993) reported on previous findings claiming that, overall, L2 writers make more errors than L1 writers mostly in morphosyntax, lexicosemantics, verbs, prepositions, articles, and nouns. Again, decades later, Eckstein & Ferris (2018) compared L1 and L2 writers in first-year university composition classes, comparing linguistic features as well as attitudes towards writing in English. They found that L2 writers commit more errors than L1 writers in terms of use of verbs, sentence structure, word form, and word choice. They also found that L2 writers have lower levels of lexical variation. Participating students were also surveyed regarding their attitudes towards their own writing. L2 writers expressed less confidence regarding their knowledge of English grammar and their ability to write grammatically. This attitudinal survey holds useful findings for those seeking to understand reasons for engaging in academic dishonesty: many students lack confidence in their own writing abilities, seeing writing (especially in an L2) as an insurmountable obstacle to academic success. This feeling of defeat can quickly lead to seeking out methods for avoiding the task altogether.

Academic dishonesty

Fishman’s (2009) definition of plagiarism provides a framework in which to discuss a particular type of academic dishonesty. According to her, plagiarism happens when one

  • uses words, ideas, or work products

  • attributable to another identifiable person or source

  • without attributing the work to the source from which it was obtained

  • in a situation where there is a legitimate expectation of original authorship

  • in order to obtain some benefit, credit, or gain which need not be monetary. (p. 5)

  • The submission of work not completed by the students themselves for course credit is a type of academic dishonesty that we have adopted for our discussion.

    While colleges and universities are independent entities with administrative authority, seventeen states in the United States have laws addressing third-party assignment completion including the intent of the contracted party, or knowledge of their work being used for academic credit or other inappropriate circumstances (Newton & Lang, 2016). At the same time, the results of the International Center for Academic Integrity’s extensive survey show 62% of undergraduate students and 40% graduate students admitting to some form of cheating on written assignments (McCabe, 2020).

    Specific to L2 student writers of dissertations and theses, an earlier study by Pecorari (2003) found instances of students using source language at rates of 40% and higher without attribution. However, in cases where students rely inappropriately on published sources, it is possible that students are not intentionally engaging in academic dishonesty, but rather, are making an unsuccessful attempt at paraphrasing, or do not fully understand citation conventions. The students in Pecorari’s study were mostly inexperienced writers, even in their L1. Behaviour such as this can serve as a teachable moment and is separate from an intentional attempt to deceive. In fact, targeted EAP instruction has been shown to increase university students’ writing competence and reduce inappropriate textual borrowing. In their study of undergraduate students and instructors in two EAP programs in China and in England, Zou & Jiang (2021) found that more than 60% of students surveyed reported having learned how to avoid potential plagiarism through training in accurate citation and paraphrasing, while also producing writing that is more organized, structured, and grammatically accurate.

    In order to identify copy-and-paste plagiarism, instructors often rely on commercially available computer programs matching student papers with other sources of text in a database. For example, if a student copies words from another source, the basic versions of widely used internet-based plagiarism detection programs, such as Turnitin, DupliChecker, or Copyleaks, will recognize the borrowed language, highlight the re-used language, and provide instructors with a percentage of the work identified as identical in other sources. This provides instructors with objective quantitative data which facilitates the identification of copy-and-paste plagiarism. Although more powerful varieties of some of these programs using forensic linguistics to detect authorship through textual differences are also available, universities typically do not invest in these versions. Hence, teachers are left with their intuition to determine whether academic dishonesty of this type has taken place.

    Studies have been conducted regarding students’ understanding of what constitutes plagiarism as well. Radunovich, Baugh & Turner (2009) indicated that many students do not always recognize plagiaristic behaviour as academic dishonesty. Hundreds of surveyed university students were asked to identify academically dishonest behaviours. The vast majority of respondents (96.9%) distinguished the clearest form of academic dishonesty: changing the name and submitting a paper written entirely by someone else. However, a relatively high percentage of students (21.2%) did not view it as plagiarism if one’s ‘mother offers to write an introductory paragraph’ and that ‘exact paragraph [is used] in the final paper’ (p. 31), indicating that there is some gray area in students’ minds when it comes to academic integrity matters.

    Contract cheating

    One form of academic dishonesty is contract cheating. Morris (2018) defines contract cheating as a student outsourcing their work to a third party: an essay mill, a friend, or an academic custom writing service. Currently, widely used plagiarism detection technologies are designed to ‘identify unoriginal content’ in order to ‘manage potential academic misconduct’ (Turnitin, 2020) but cannot detect forms of academic dishonesty in which the writing is original to the assignment, though not written by the student themselves.

    According to Newton’s (2018) review of sixty-five studies dating back to 1978, contract cheating is likely to be under-reported due to methodological issues such as use of convenience sampling and lack of participant assurance regarding the anonymity of their data. More specifically, studies often have low response rates, and it is logically postulated that students who have engaged in contract cheating are more likely to be concerned about data exposure. In fact, Lancaster & Clarke (2016) stated that there are ‘no complete and reliable results that confirm the true extent of contract cheating, and there is little evidence to support the fallacy that students who are contract cheating are being detected’ (p. 641). Despite such gaps in the research, based on anecdotal evidence from colleagues and evidence provided by Newton (2018), it does appear that contract cheating and academic dishonesty in general are increasing over time. In the aforementioned review of contract cheating research, Newton (2018) found a median response rate of 3.5% of participants admitted to participation in some form of contract cheating; however, the data with the highest reported rates of contract cheating (above 20%) were collected in 2009 or later. The increasing rate of this behaviour is especially concerning for academia because, in the case of the student use of a custom writing service or essay mill, ‘the act of payment makes contract cheating deliberate, pre-planned and intentional’ (Newton, 2018, p. 2) as opposed to a misunderstanding of academic norms as would be the case in some instances of plagiarism.

    Detecting contract cheating is difficult. Instructors have played the role of detective, combing agency sites for submissions of student work or setting up Google Alerts for when a web page is edited or created with reference to an assignment specific to a course (Lancaster & Clarke, 2016). This method is not part of the instructor’s job description, and yet time consuming and potentially fruitless, as it relies on students going to public online forums requesting for work to be completed. It has also been suggested that instructors compare word document author names with that of students as a method to determine text authorship (Lines, 2016). Additionally, contracted papers often contain language that is rather vague, does not necessarily follow directions within the prompt, and may be written using a variety of English different from that of the student, such as British English in an American university setting (Lines, 2016). Once an instructor’s suspicions have been aroused, Rogerson (2017) suggests holding in-person discussions with students, asking students to demonstrate how they located material referenced in their papers. If students are unable to locate texts they cited in their papers, then the references may not exist at all or may have been found by another writer.

    In a recent study, seven university instructors were given twenty psychology writing assignments to assess and were also asked to identify whether they believed the essays they were reading had been completed by a student or had been purchased from a contract cheating site. Six of the twenty papers had been purchased. Instructors successfully identified the contract cheating papers 62% of the time, citing that these papers generally flowed in a way that was unrelated to the prompt, the writing style was overly formal, and they were missing key sections (Dawson & Sutherland-Smith, 2018). If we consider the fact that instructors were specifically told to identify contract cheating, this rate of recognition is rather low. As a follow up to their previous study, Dawson and Sutherland-Smith (2019) found that providing essay markers with training in the detection of contract cheating improved the accuracy of their contract cheating detection from 58% to 82%. Additionally, the researchers acknowledge that indicators of contract cheating can vary widely across disciplines and assignments, and thus did not share what the markers found to be relevant clues that contract cheating may have taken place. That training can improve faculty’s ability to recognize this form of academic dishonesty is encouraging, although this complex method of identification arguably takes resources and time unavailable to many institutions. Therefore, many instructors are not considering this style of plagiarism when reading student work, and so they are detecting no contract cheating at all.

    Lancaster & Clarke (2016) suggest a computational approach to detection in which student writing is collected electronically and software is used to identify when a piece of writing diverges from past writing. Researchers have explored automated methods for determining authorship, such as stylometry, which serves to analyse attributes and writing styles of individual writers. Since language allows for a multitude of ways of expressing a singular idea, and writers have the freedom (and proclivity) to express things in certain ways (Juola, 2017), individual styles can be detected. This is referred to, by forensic linguistics, as idiolect; that is, a writer’s own distinctive version of spoken and written language. This distinctiveness can be identified through linguistic analyses, examining one’s grammatical style or phrasal preferences (Coulthard, 2004). Using freely available software, Ison (2020) explored the accuracy of stylometry tools to attribute written texts to a particular author. The researcher used 500-word blocks of text from a peer-previewed journal: five published articles written by one author and five published articles written by authors different from the first group and analysed them using three different pieces of stylometry software. The accuracy rates of the software varied widely, but one in particular was able to identify text as having been written, or not, by the author with an 80-90% accuracy rate (Ison, 2020).

    It has been found that engagement in contract cheating may be impacted by favourable circumstances. A study by Rigby et al. (2015), which calculated students’ levels of risk preference correlated with a willingness to contract cheat in hypothetical scenarios, revealed that 50% of participating students indicated a willingness to buy at least one essay, especially when the odds seemed in their favour for a good grade and low risk of being caught. Among the L2 English student population in the same study, over 80% indicated a willingness to buy at least one essay. Curtis & Clare (2017) found that 62.5% of students who have engaged in contract cheating have done so repeatedly. In fact, Clarke & Lancaster (2006) found that over 50% of the users placing contract cheating requests on the website RentACoder did so between two and seven times. Bretag et al. (2018) surveyed over 14,000 students at eight different Australian universities finding that 5.78% of the survey respondents reported engaging in some form of contract cheating. Here, too, L2 English speakers were found to be over-represented in contract cheating behaviour by a ratio of 1:1.9, and international students at a ratio of 1:2.

    Researchers have taken on the role of the student seeking to pay someone to complete an assignment, as reported in an investigation into the industry of bespoke academic essays in the UK (Medway, Roper, & Gillooly, 2018). Contract cheating site staff give assurances regarding lack of detection of their work by plagiarism software and the guarantee of a good grade on the assignment. Additionally, some professional ghost writers have disclosed using Turnitin on their own writing before passing it on to students to ensure a low textual match to other writing, thus attempting to lower the chances of the student submitting the contracted paper being accused of plagiarism (Sivasubramaniam et al., 2016).

    Limited research has been conducted, however, regarding who professional ghost writers are. One study looked at ghost writers operating via the micro-outsourcing site fiverr.com, finding that 43.4% of the academic writing providers surveyed were from Kenya (Lancaster, 2019). Many purport to be operating within the US (31.6%), though according to Lancaster (2019) many of these writers make L2-like errors in their writing. Ghost writers have been found to have advanced degrees from Western universities, including doctorates, who claimed that upon returning to their home countries they faced ‘a grim future as regards employment’ and the authors conclude that therefore ‘some of these students can and do start ghost writing services’ (Sivasubramaniam et al., 2016, p. 2). Ghost writers have been identified and classified into six distinct categories: essay writing services; friends, family, and other students; private tutors; copyediting services; agency websites; reverse classifieds (Lancaster & Clarke, 2016).

    When looking at the reviewed literature regarding L2 writing development as well as contract cheating detection, there seems to be an over-reliance on perceptual and manual methods as ways of identifying this behaviour. This exploratory study aims to examine the possibility of using automated linguistic methods to detect this popular form of academic dishonesty. There are noteworthy and quantifiable differences between the writing of L1 and L2 students, as well as between the writings of L2 writers of varying English proficiency levels. These differences can be exploited in order to differentiate between work that L2 students write and submit themselves versus work they submit written by contracted essay writers.

    The current study

    The current research was inspired by instructors’ experience with one type of academic dishonesty, contract cheating. Taken from this classroom experience, Example 1 below illustrates a paper written by a lower-division L2 university student. Example 2 is from a paper submitted by the same student, but this time, according to discussions held between the instructor and the student, the student submitted work written by someone else. Examples 3 and 4 illustrate the same pattern, respectively. The spacing errors in Example 3 are original to the submitted text.

    Example 1: Written by student

    As such, the risks of innocent and ordinary civilians being harm under the impression of being criminals increase with the implementation of the system.

    Example 2: Contracted paper

    In short, it is reasonable to conclude that this mother represents the majority of parents, who understand that technology is a necessary part of life but not necessarily one that all children, regardless of age, should own.

    Example 3: Written by student

    People has different thoughts about robots. Some of them think that robots are very helpful in our society, others think that they are harmful for our society, for example, some of the people think that the use of robot as a pet is not good for us, while others think that it’s good to improve people communication skill and how to chat with others, robots also are very so expensive so not everyone can have robot .

    Example 4: Contracted paper

    According to the article, the unprecedented developments in service robots has not met the initial promise of truly intelligent machines, and the human-robot interaction still possesses serious ethical concerns even when intelligence techniques have been implemented for the expression of emotion, face recognition, language interaction, and speech perception.

    Seeing differences in student work illustrated by the examples above as well as recognizing the need for empirical evidence to show differences between student papers and contracted papers motivated this study. The following research questions guided this study:

  • Does the lexical profile of L2 university student papers differ from that of contract cheating papers on the same assignment?

  • Which measures are the strongest indicators of the differences between these two sets of papers?

  • Methodology

    Data

    Two sets of papers comprise the data in our collection for this exploratory study. To start, a set of five papers written by international students in a university EAP composition course were included (6,507 words total). In order to be admitted to this large public Southern California university, L2 English students must submit a TOEFL score of 80 or higher on the internet-based test (iBT) or 550 or higher on the paper-based test (PBT), or a score of 6.5 or higher on the International English Language Testing System (IELTS) test; these scores are roughly equivalent to a minimum score of B2 on the Common European Framework of Reference for Languages (CEFR).

    Student work included in the study were essays written in the second of three required undergraduate EAP composition courses. Students must have passed the first level course with an overall grade of 70% or higher in order to take the second course. These papers were written for an assignment designated as the ‘exploratory synthesis’ paper, in which students were prompted to explore the topic of social inequality in the United States using three assigned articles as reference (see assignment prompt in the Appendix). Students in this course went through a two-draft process for each writing assignment. The papers in this collection are the first drafts, and students had ten days to complete it. Based on anecdotal evidence from discussions with fellow EAP instructors, when students submit contracted papers, it is often on the first draft of the paper, and so this is the key moment for an instructor to distinguish between a paper written by a student or a contract cheating worker. The student papers for this draft all received grades in the ‘B’ range (B-, B, or B+).

    Secondly, a set of five contracted papers was added to the collection (4,871 words total). For these papers, we visited contract cheating websites where anyone can pay to have essays written for them by ‘qualified experts’. In order not to arouse suspicion by placing multiple orders for the same paper on the same site, we placed orders for these papers at five different sites (samedayessay.com, papernow.org, payforessay.net, domywriting.com, and rapidessay.com). Websites were chosen simply based on what appeared in a Google search using the term ‘write my essay for me’. The requests relied on the same exploratory synthesis prompt and readings used by the students (Appendix) in the classroom.

    Contract paper sites generally have a similar customer experience: a greeting page assuring clients of the speediness of the service, promises of high grades and no plagiarism detection, and positive testimonials from other clients. As noted by Medway, Roper, & Gillooly (2018), the fine-print disclaimers found at the bottom of some of the pages claim purchased papers are intended only ‘for research purposes’ (p. 410) and not to be submitted for academic credit. Essay prices vary depending on the level of writing required (e.g., undergraduate versus doctoral programme), the required number of pages, the experience of the ghostwriter, and how quickly the writing needs to be completed (essays can be written as quickly as in one hour). For our purposes, papers were requested to be three pages long, at an undergraduate level, and completed in two weeks’ time. We recognize that our research is a small-scale, exploratory study that compares only five texts from each group afforded by our limited budget to purchase contract cheating papers. However, we believe that our innovative approach can pave the way for large-scale studies that will further test the viability of this approach to contract cheating as discussed in the final section.

    Analytical tools and measures of lexical complexity

    The Lexical Complexity Analyzer (LCA) (Ai & Lu, 2010; Lu, 2012) was applied in the attempt to distinguish between contracted papers and student papers with quantitative, empirical methods. It is a freely available program accessible through a website (https://aihaiyang.com/software/).

    The LCA was primarily developed to explore the relationship between the lexical richness of ESL learners’ oral narratives and the scores given by human raters. The program uses the British National Corpus’ ‘Top 2,000 Most Frequent Words’ list to judge word sophistication and provides twenty-five different variables regarding lexical usage. Table 1 presents a grouped summary of those features.

    Automated lexical complexity measures including lexical word variation (adopted from Lu, 2012)*

    Measure Code Definition/Example
    Word types WORDTYPES number of separate lemmas in text; different inflections of the same lemma (i.e., sit, sits, sat) are one word type
    Sophisticated word types SWORDTYPES number of different words (word types) not included in the list of top 2,000 most frequent words in the British National Corpus (BNC)
    Lexical word types LEXTYPES number of different lexical words, that is, number of different nouns, different adjectives, different verbs (excluding modals, auxiliary verbs be and have) and different adverbs with an adjectival base
    Standardized lexical word types SLEXTYPES a scaled number of different lexical words (defined above)
    Word tokens WORDTOKENS total number of words
    Standardized word tokens SWORDTOKENS a scaled total number of words
    Lexical word tokens LEXTOKENS total frequency of lexical words (defined above)
    Standardized lexical word tokens SLEXTOKENS a scaled total number of lexical words (defined above)
    Lexical density LD ratio between number of lexical words (defined above) and total words
    Lexical sophistication - I LS1 ratio between the number of sophisticated words (i.e., those excluded from the BNC top 2,000 frequent word list; defined above) and the total number of lexical words (defined above)
    Lexical sophistication - II LS2 ratio between the number of sophisticated word types (defined above) and the total number of word types
    Verb sophistication - I VS1 ratio between the number of sophisticated verb types (defined above) and the total number of verbs
    Verb sophistication - II VS2 ratio between the number of sophisticated verb types (defined above) squared and the total number of verbs
    Number of different words NDW number of different words
    Type - token ratio TTR proportion of the number of different words and the total number of words
    Uber index UBER measure of vocabulary richness similar to TTR but relying on vocabulary size
    Lexical word variation LV ratio between the number of lexical words (defined above) to the total number of words
    Verb variation - I VV1 ratio between the number of different verbs and the total number of verbs
    Verb variation - II VV2 ratio between the number of different verbs and the total number of lexical verbs (defined above)
    Noun variation NV ratio between the number of different nouns and the total number of nouns
    Adjective variation ADJV ratio between the number of different adjectives and the total number of adjectives
    Adverb variation ADVV ratio between the number of different adverbs and the total number of lexical adverbs (those with an adjectival base)
    Modifier variation MODV ratio between the number of different adjectives and adverbs and the number of nouns

    * For further details on these measures, see Lu (2012).

    The variables are categorized as lexical variation, lexical sophistication, and lexical density.

    Lu (2012) defines lexical words as nouns, adjectives, verbs (excluding modals and auxiliary verbs be and have), and adverbs with an adjectival base (including those that function as both adverb and adjective, for example, fast). Lexical density is the ratio of the number of lexical words to the number of words in a text. Lexical sophistication is measured by the ratio between sophisticated words (words that are excluded from the BNC 2,000 most frequent words list) and the total number of lexical words (LS1) or the total number of word types (LS2).

    Texts were first converted from .doc files to .txt, using the freely available AntFileConverter (Anthony, 2017) and were then uploaded to the LCA. Once the texts are uploaded, the analyser produces an Excel spreadsheet of the numerical data connected with the different markers of lexical complexity.

    Statistical methods

    Each lexical complexity feature in LCA is a dependent variable in our study measured on a continuous scale. Our independent variable has two categorical independent groups (student papers versus contract cheating papers). Each paper (observation) was written by a different person and each paper belongs to only one group, hence the observations are independent. The normal distribution of scores for each group was tested by skewness and kurtosis measures, histograms, and normal Q-Q plots available in the statistical program we used. Since our sample size is small, we were unable to meet some of the assumptions required to carry out a parametric test, such as the Independent Sample T-test, which would compare the mean difference between two groups. Instead, to compare the two groups on all variables produced by LCA (Table 2), a Mann-Whitney U test was applied using SPSS 26.

    Descriptive statistics for lexical complexity measures in contracted and student papers

    Measures/group descriptives Contracted papers Student papers
    Mean SD Mean SD
    Word types 351.60 40.19 331.40 41.11
    Sophisticated word types 103.60 20.13 75.80 19.15
    Lexical word types 286.60 39.56 260.40 44.89
    Standardized lexical word types 99.40 19.24 70.80 20.14
    Word tokens 994.40 80.99 1330.00 212.27
    Standardized word tokens 196.40 30.30 190.60 46.66
    Lexical word tokens 559.80 50.63 709.80 120.73
    Standardized lexical word tokens 168.40 22.27 149.20 39.80
    Lexical density .56 .02 .53 .02
    Lexical sophistication - I .30 .04 .21 .04
    Lexical sophistication - II .29 .03 .23 .03
    Verb sophistication - I .13 .04 .09 .04
    Verb sophistication - II 1.98 1.11 1.13 .61
    Number of different words .96 .30 .72 .25
    Type - token ratio .36 .03 .25 .04
    Uber index 19.95 1.77 16.31 1.42
    Lexical word variation .64 .02 .49 .132
    Verb variation - I 46.25 4.40 31.61 10.57
    Verb variation - II .13 .015 .09 .01
    Noun variation .49 .06 .36 .05
    Adjective variation .10 .02 .07 .02
    Adverb variation .04 .01 .14 .03
    Modifier variation .14 .03 .12 .02

    After determining statistical significance with the non-parametric test, Cohen’s d values were calculated using an Excel sheet to determine the effect size. Cohen’s d is a commonly used descriptive statistic in applied linguistic research (Crawford et al., 2018; Wei & Hu, 2019; Wei et al., 2019) that ‘expresses the mean difference between (or within) groups’ (Plonsky, 2015, p. 31). Effect size measures ‘provide an estimate of the actual strength of the relationship or of the magnitude of the effect in question’ (Plonsky, 2015, p. 31), affording qualitative measures for the researcher to describe the degree of difference between two groups. Cohen’s d is calculated by deducting the mean of one group from the other and dividing the difference by the two groups’ pooled standard deviation; more specifically

    While there is no cut-off point for Cohen’s d, researchers in applied linguistics typically consider ±0.40 a small effect size, ±0.70 a medium effect size, and ±1.00 and above a large effect size (Biber & Egbert, 2018; Crawford et al., 2018; Plonsky, 2015; Plonsky & Oswald, 2014). Cohen’s d effect sizes can take up positive or negative values depending on whether the larger or the smaller mean score enters the equation first.

    Findings

    In this section we first present the statistical results of the comparison between student papers and contract cheating papers. Second, we provide text samples illustrating some of the measures, and discuss them more in detail. Finally, we summarize the results.

    Our report here primarily concerns the statistically significant results as calculated by the Mann-Whitney U test. Table 2 shows descriptive statistics (means and standard deviations) for the measures studied for contracted papers and student papers, respectively.

    As Table 2 shows, contracted papers have higher mean scores on all but two lexical complexity measures (word tokens and lexical word tokens). Table 3 shows the Z scores gained through the Mann-Whitney U test, the p values, and the effect sizes (Cohen’s d) for each lexical complexity measure.

    Mann-Whitney U results and effect sizes for lexical complexity measures

    Lexical complexity/statistics Z p d
    Word types -1.358 0.175 0.48
    Sophisticated word types -1.567 0.117 1.41
    Lexical word types -1.358 0.175 0.62
    Standardized lexical word types -1.567 0.117 1.45
    Word tokens -1.984 0.047* -2.09
    Standardized word tokens -0.131 0.754 0.15
    Lexical word tokens -1.776 0.076 -1.62
    Standardized lexical word tokens -0.731 0.465 0.60
    Lexical density -1.803 0.071 1.36
    Lexical sophistication - I -2.417 0.016* 2.30
    Lexical sophistication - II -2.220 0.026* 2.23
    Verb sophistication - I -1.467 0.142 0.94
    Verb sophistication - II -1.149 0.251 0.94
    Number of different words -1.358 0.175 0.48
    Type - token ratio -2.619 0.009** 2.85
    Uber index -2.611 0.009** 2.27
    Lexical word variation -2.410 0.016* 1.67
    Verb variation - I 2.611 0.009** 1.81
    Verb variation - II 2.635 0.008** 2.79
    Noun variation -2.619 0.009** 2.25
    Adjective variation -1.838 0.066 1.41
    Adverb variation -0.565 0.572 -0.37
    Modifier variation -1.078 0.281 0.92

    * Statistically significant at p>0.05;

    ** statistically significant at p>0.01

    As Table 3 shows, papers written by students compared to the papers written by contract cheating workers demonstrated statistically significantly higher word token scores (Z = -1.984, p = .047) showing generally longer essays by students. A large Cohen’s d score (d = -2.09) indicates that a considerable difference is highly likely between the two groups on this measure.

    In contrast, contracted papers have more of the lexical density and lexical sophistication features. Of the lexical density measures, type-token ratio and the Uber index showed statistically significant results. Contracted papers compared to student papers showed better type-token scores (Z = -2.619, p = 0.009), with a strong difference between the two groups illustrated by a large effect size (d = 2.85). Again, contracted papers showed better Uber index scores (Z = -2.611, p = 0.009), a measure of vocabulary richness, with a substantial difference between the two groups illustrated by a large effect size (d = 2.27).

    Lexical sophistication measures, the use of less frequently used words, and lexical word variation measures (ratio of the number of lexical words to the total number of words) as they relate to verb variation (ratio of number of verb types to total number of verbs and total number of lexical verbs) and noun variation (ratio between number of noun types and number of noun tokens) yielded statistically significant differences. Specifically, contracted papers compared to student papers showed better lexical sophistication scores (Z = -2.417, p = 0.016 and Z = -2.220, p = 0.026) as well as lexical word variation scores (Z = -2.410, p = 0.016), with a strong difference between the two groups illustrated by relatively large effect sizes (d = 2.30, 2.23, and 1.67, respectively). More specifically, contracted papers compared to student papers showed higher variation on the ratio of number of verb types to total number of verbs (Z = -2.611, p = 0.009), with a strong difference between the two groups illustrated by a large effect size (d = 1.81) as well as on the ratio of the number of verb types to the total number of lexical verbs (Z = -2.635, p = 0.008), with a strong difference between the two groups also illustrated by a large effect size (d = 2.79). Additionally, contracted papers showed better scores on noun variation (Z = -2.619, p = 0.009), with a strong difference between the two groups illustrated by a large effect size (d = 2.25). These results highlight the fact that while the L2 student writing group may contain longer essays with higher words counts, the vocabulary is demonstrably less varied as compared with the papers completed by contract cheating site staff.

    Interestingly enough, although not statistically significantly different, large Cohen’s d values are apparent on other measures as well (Table 3) such as, sophisticated word types (d = 1.41), standardized lexical word types (d = 1.45), lexical density (d = 1.36), and adjective variation (d = 1.41). This finding indicates that there may be further, important variation between the two types of writing.

    Overall, the contracted paper group was found to be more lexically dense and more lexically sophisticated than the L2 student paper group; L2 students were more likely to rely on lexical items labelled by the analyser as being more common, or less sophisticated. Student papers, despite being more verbose, demonstrated less lexical variety as indicated by type-token ratios and Uber index. In addition, the variety of nouns and verbs was found to be higher in papers written by contract cheating site staff. The large effect sizes found through calculating Cohen’s d indicate that the writing produced by these two groups may be different suggesting that different individuals may have written the papers.

    The following examples illustrate these lexical differences:

    Example 5: From contracted paper group

    The critical themes brought out from the articles are, therefore, economic wrangles, probable solutions to societal problems, and social stratification’.

    Example 6: From L2 student group

    To conclude, social and income inequality is a big problem in America and it is still exist.

    Examples 5 and 6 originate from the first sentences of the conclusion paragraphs from the respective papers. To highlight differences in levels of lexical sophistication between the groups of papers, underlined words are included in the BNC 2,000; the bolded words are not found on this list and are treated by the program as more sophisticated. Example 6, from the group of L2 student papers, has no words labelled sophisticated, whereas Example 5 has three words judged by the program to be sophisticated. If this level of difference in lexical sophistication is evident from comparing individual sentences, it can be reasonably extrapolated that these differences are immense when comparing entire essays.

    Example 7: From contracted paper group

    Whereas economic inequality is experienced when the accumulation of wealth is unequally distributed, social inequality occurs when some people in the society are prohibited from obtaining the same quality of resources and services as the wealthy in society can access.

    Example 8: From L2 student group

    People in lower classes are having a lot of difficulties when it comes to success and joining universities, and also having their dreams crushed just because they are not in the level of society that let them act and do what they have in mind.

    Examples 7 and 8 exemplify the differences in the variety of verbs (in bolded italics) used by the writers. Example 7 from the contracted paper group, has greater verb variation than Example 8 which uses forms of ‘have’ three times in one sentence. Additionally, as noted previously, ‘have’ is not treated by the analyser as a lexical verb, thus further reducing the lexical verb variety according to the LCA.

    When looking at verb usage, it is natural to observe errors in usage such as subject-verb agreement, which are evident in the student sample though not in the contracted paper sample. The LCA does not account for lexical or usage errors.

    Example 9: From contracted paper group

    On the contrary, most average Americans have unchanged optimism and ambitions. Consequently, it is becoming even harder for the poorest 1% to get employment making them socially immobile and not changing their positions.

    Example 10: From L2 student group

    Success is everyone dream humans have a desire that they want to be successful and live the life that they want. Success is not only living by the values that you believe in but being in a way that you are recognized in the world.

    Examples 9 and 10 were taken from the first body paragraphs of their essays. In this case, nouns are in italics. In Example 10, the lemma ‘success’ is used three times in two sentences and a total of ten times in the paragraph from which the example sentences were taken. Evidence of this level of lemma repetition is not found in the contracted paper group, which provides evidence for the analyser’s finding that the contracted paper group exhibits higher levels of lexical variation. This contributes to our understanding of the type-token ratio and Uber index quantitative results from the LCA, as these are both indicators of vocabulary diversity and richness.

    In addition, the sample from the L2 student paper (Example 10) is more verbose at forty-five words versus Example 9’s contract paper sample (thirty-three words), highlighting the fact that while the L2 student writing group may contain longer essays with higher words counts, the vocabulary is demonstrably more limited as compared with the papers completed by contract cheating site staff.

    Pedagogical implications

    In this study, we have used a LCA tool with great success to identify differences between student papers and contracted papers. In terms of lexical diversity and lexical sophistication, contracted papers had higher type-token ratios and higher Uber index values. They had higher lexical variation scores in terms of higher noun variation as well as a higher ratio of verb types when compared to categories of verbs overall, and to lexical verbs alone.

    Among the pedagogical implications, first, we highlight the fact that often when instructors read a text that they doubt was written by the students themselves, one of the immediately apparent differences between the suspected paper and the student’s previously submitted work are the lexical items. According to the results of our exploratory study, contracted papers seem to be more lexically diverse and sophisticated than L2 student papers. Having this knowledge may be helpful for instructors in identifying when this behaviour has occurred; if a student who produces in-class writing at a completely different level of lexical sophistication as their major take-home assignments, this might raise questions for an instructor. Additionally, it is our hope that studies such as this one may be of use for teacher education programmes or professional development sessions as a way of increasing awareness of contract cheating as well as providing a basis for discussion on possible differences between student papers and contracted writing.

    Another implication relates to EAP and ESL teachers who may consider using this tool to aid in the actual detection of contract cheating. The LCA can provide quantitative values to lexical differences that instructors may find surprising or questionable when reading a possibly contracted paper submitted by an L2 student. In all, this tool can support already existing intuitions and perceptions when evaluating writing. That is, if an instructor suspects a student of contract cheating, this tool may provide empirical evidence on language differences between one paper and another, independent of the instructor’s intuition. Furthermore, the differences identified by the LCA are ones which would generally not be recognized through reading and assessing students’ writing. For example, a teacher would not usually count all of the nouns written in a given assignment and compare that to previous work. Given that this tool is freely available online, producing results in a matter of minutes, we believe that it merits an instructor’s exploration. When using the LCA, instructors may select certain measures for analysis which are easier to define and conceptualize, such as lexical word variation or noun variation, and use those measures to determine if there is a quantitative lexical difference between one piece of writing and another. In this way, instructors can be assured that they have an evidence-based, informed understanding of the differences that aids their explanation to a student or colleague.

    We note, however, that this tool could be improved to better serve practising teachers. Some of the measures included in the tool involve complex mathematical calculations and can be difficult to fully comprehend let alone explain to a student. Provided the developers’ interest in serving practising teachers and to help identifying differences between student papers and contract cheating papers, a comprehensive step-by-step manual providing the full and detailed description of the noted features in a more transparent way would be highly beneficial. Additionally, it would be helpful for instructors to gain support for their intuitions and have automated ways of quantitatively ascertaining if measured differences between the works are large enough to confirm suspicions of contract cheating. For the purposes of this research, we calculated significance and effect size in order to make these determinations. However, busy classroom teachers may not apply statistics to find substantial differences between papers. Hence, the question of how they decide if a difference between two pieces of writing is large enough to be meaningful on the given measures remains an area to be explored further.

    Finally, the LCA could be applied in L2 writing instruction outside of what was explored in this study. The measures related to lexical variety, sophistication, and density provided by this tool can add quantitative results to the vocabulary measures on rubrics instructors use to evaluate student writing. In fact, instructors can choose which measure(s) would best support their perception of lexical sophistication and lexical variety in student papers. Recognizing the importance of a varied vocabulary repertoire, instructors could incorporate more intensive and broader vocabulary teaching in their programme. For example, designing and implementing specific vocabulary learning activities that encourage students to employ a wider variety and more sophisticated set of words in their papers would result in positive learning habits. Tools such as the one explored in this study could then be used by instructors to gauge improvement in lexical variety and sophistication in students’ writing over the course of a semester or unit. The tool could even be introduced to students as a way of measuring these changes in their own writing, helping build autonomy in their language learning.

    Limitations and future directions

    This study also has limitations, such as the small number of files that were analysed. We also acknowledge the fact that the findings, while providing an important first step to additional work in this area, may not be unexpected. Future research using more L2 student and contracted papers may corroborate our findings or produce results different from our findings, perhaps with greater numbers of statistically significantly different measurements. Another important limitation is the fact that, within the L2 student paper group, it is impossible to say with absolute certainty that these papers were indeed written by the students themselves, and that none of these papers were contracted. This determination was made by one of the researchers who was also the teacher in that classroom. Without being present while work is in progress, it is nearly impossible to know for certain how students go about completing an assignment. It is also possible that the student would seek help at a campus writing centre prior to submitting their work and could obtain substantial support for their writing, especially on grammar and vocabulary choice.

    Looking to future directions, there is the possibility for conducting research in a similar manner but with different linguistic software, perhaps looking at different complexity measures (e.g., syntactic complexity). The software used here is not designed to identify syntactic or lexical errors as measures of complexity, though this is a clear difference between L2 student and professional writing. As seen in the limited research available on the subject, workers at contract cheating sites may be L1 or L2 speakers, hence, professional writing may be completed by L1 or L2 writers. Regardless, these contract cheating workers are generally writing at higher levels than the L2 students themselves and are thus making fewer errors in features such as word usage. Future investigation into error differences between contracted and L2 student papers might prove to hold interesting results.

    One of the goals of this research was to raise awareness of contract cheating. Understanding that this is taking place in classrooms around the world is the first step in detection. Using automated tools can be helpful in codifying detection and providing data to support instructors’ instincts. However, none of this replaces instructors knowing their students and their writing, as well as having conversations regarding academic honesty. Once irregularities in an assignment have been noted by an instructor, it is important to have a discussion with the student regarding the instructor’s questions and concerns. There may be contextual factors at play that the instructor cannot be aware of, or the student may need additional support that the instructor has not yet provided (Rogerson, 2017). An important concern in the detection of any type of academic dishonesty is the possibility of false positives, and thus of accusing students of misconduct who have not engaged in such an act (Juola, 2017). Additionally, scaffolding students’ writing development and giving ample time and instruction to complete assignments may dissuade students from turning to these professionals.

    References

    Ai, H., & Lu, X. (2010). A web-based system for automatic measurement of lexical complexity [Paper presentation]. 27th Annual Symposium of the Computer-Assisted Language Consortium (CALICO-10). Amherst, MA. 8-12 June. Google Scholar

    Anthony, L. (2017). AntFileConverter (Version 1.2.1) [Windows]. Tokyo, Japan: Waseda University. https://www.laurenceanthony.net/software. Google Scholar

    Biber, D., & Egbert, J. (2018). Register variation online. Oxford University Press. Google Scholar

    Bretag, T., Harper, R., Burton, M., Ellis, C., Newton, P., Rozenberg, P., Saddiqui, S., & van Haeringen, K. (2018). Contract cheating: A survey of Australian university students. Studies in Higher Education, 1-20. http://doi.org/10.1080/03075079.2018.1462788. Google Scholar

    Clarke, R., & Lancaster, T. (2006). Eliminating the successor to plagiarism? Identifying the usage of contract cheating sites. Proceedings of 2nd International Plagiarism Conference, Northumbria Learning Press, 19-21. Google Scholar

    Clarke, R., & Lancaster, T. (2007). Establishing a systematic six-stage process for detecting contract cheating. Proceedings of the 2nd International Conference on Pervasive Computing and Applications, Curran Associates, 342-347. http://doi.org/10.1109/ICPCA.2007.4365466. Google Scholar

    Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Sage. Google Scholar

    Coulthard, M. (2004). Author identification, idiolect, and linguistic differences. Applied Linguistics, 25(4), 431-447. Google Scholar

    Crawford, W., McDonouogh, K., & Brune-Mercer, N. (2018). Identifying linguistic markers of collaboration in second language peer interaction: A lexico-grammatical approach. TESOL Quarterly 53(1), 180-207. Google Scholar

    Crossley, S. A., & McNamara, D. S. (2009). Computational assessment of lexical differences in L1 and L2 writing. Journal of Second Language Writing, 18, 119-35. Google Scholar

    Crossley, S. A., & McNamara, D. S. (2011). Shared features of L2 writing: Intergroup homogeneity and text classification. Journal of Second Language Writing, 20, 271-285. Google Scholar

    Curtis, G. J., & Clare, J. (2017). How prevalent is contract cheating and to what extent are students repeat offenders? Journal of Academic Ethics, 15, 115-124. http://doi.org/10.1007/s10805-017-9278-x. Google Scholar

    Dawson, P., & Sutherland-Smith, W. (2018). Can markers detect contract cheating? Results from a pilot study. Assessment & Evaluation in Higher Education, 43(2), 286-293. Google Scholar

    Dawson, P., & Sutherland-Smith, W. (2019). Can training improve marker accuracy at detecting contract cheating? A multi-disciplinary pre-post study. Assessment & Evaluation in Higher Education, 44(5), 715-725. http://doi.org/10.1080/02602938.2018.1531109. Google Scholar

    Eckstein, G., & Ferris, D. (2018). Comparing L1 and L2 texts and writers in first-year composition. TESOL Quarterly, 52(1), 137-162. Google Scholar

    Fishman, T. (2009). “We know it when we see it” is not good enough: Toward a standard definition of plagiarism that transcends theft, fraud and copyright [Paper presentation]. 4th Asia Pacific conference on educational integrity: Creating an inclusive approach. University of Wollongong, 28-29 September. https://ro.uow.edu.au/cgi/viewcontent.cgi?article=1037&context=apcei. Google Scholar

    Grant, L., & Ginther, A. (2000). Using computer-tagged linguistic features to describe L2 writing differences. Journal of Second Language Writing, 9(2), 123-145. Google Scholar

    Ison, D. C. (2020). Detection of online contract cheating through stylometry: A pilot study. Online Learning, 24(2), 142-165. https://doi.org/10.24059/olj.v24i2.2096. Google Scholar

    Juola, P. (2017). Detecting contract cheating via stylometry methods. In R. K. Zeenath, H. Christopher and T. Foltýnek (Eds.), Plagiarism across Europe and beyond 2017 (pp. 187-198). Mendel University in Brno. Google Scholar

    Lancaster, T. (2019). Profiling the international academic ghost writers who are providing low-cost essays and assignments for the contract cheating industry. Journal of Information, Communication and Ethics in Society, 17(1), 72-86. Google Scholar

    Lancaster, T., & Clarke, R. (2016). Contract cheating: The outsourcing of assessed student work. In T. Bretag (Ed.), Handbook of academic integrity (pp. 639-654). Springer. Google Scholar

    Lines, L. (2016). Ghostwriting guaranteeing grades? The quality of online ghostwriting services available to tertiary students in Australia. Teaching in Higher Education, 21(8), 889-914. Google Scholar

    Lu, X. (2012). The relationship of lexical richness to the quality of ESL learners’ oral narratives. The Modern Language Journal, 96(2), 190-208. Google Scholar

    McCabe, D. (2020). Statistics. International Center for Academic Integrity, www.academicintegrity.org/statistics/. Google Scholar

    Medway, D., Roper S., & Gillooly, L. (2018). Contract cheating in UK higher education: A covert investigation of essay mills. British Education Research Journal, 44(3), 393-418. Google Scholar

    Morris, E. J. (2018). Academic integrity matters: Five considerations for addressing contract cheating. International Journal for Academic Integrity, 14(15), 1-12. Google Scholar

    Newton, P. (2018). How common is commercial contract cheating in higher education and is it increasing? A systematic review. Frontiers in Education, 3, 1-18. Google Scholar

    Newton, P. M., & Lang, C. (2016). Custom essay writers, freelancers, and other paid third parties. In T. Bretag (Ed.), Handbook of academic integrity (pp. 249-271). Springer. Google Scholar

    Pecorari, D. (2003). Good and original: Plagiarism and patchwriting in academic second-language writing. Journal of Second Language Writing, 12, 317-345. Google Scholar

    Plonsky, L. (2015). Statistical power, p values, descriptive statistics, and effect sizes: A ‘back-to-basics’ approach to advancing quantitative methods in L2 research. In L. Plonsky (Ed.), Advancing quantitative methods in second language research (pp. 23-45). Routledge. Google Scholar

    Plonsky, L., & Oswald, F. (2014). How big is “big”? Interpreting effect sizes in L2 research. Language Learning, 64, 878-912. https://doi.org/10.1111/lang.12079. Google Scholar

    Radunovich, H., Baugh, E., & Turner, E. (2009). An examination of students’ knowledge of what constitutes plagiarism. NACTA Journal, 53(4), 30-35. Google Scholar

    Rigby, D., Burton, M., Balcombe, K., Bateman, I., & Mulatu, A. (2015). Contract cheating and the market in essays. Journal of Economic Behavior & Organization, 111, 23-37. Google Scholar

    Rogerson, A. M. (2017). Detecting contract cheating in essay and report submissions: Process, patterns, clues and conversations. International Journal for Academic Integrity, 13(10), 1-17. Google Scholar

    Silva, T. (1993). Toward an understanding of the distinct nature of L2 writing: The ESL research and its implications. TESOL Quarterly, 27(4), 657-677. Google Scholar

    Sivasubramaniam, S., Kostelidou, K., & Ramachandran, S. (2016). A close encounter with ghost-writers: An initial exploration study on background, strategies and attitudes of independent essay providers. International Journal for Academic Integrity, 12(1), 1-14. Google Scholar

    Staples, S., & Reppen, R. (2016). Understanding first-year L2 writing: A lexicogrammatical analysis across L1s, genres, and language ratings. Journal of Second Language Writing, 32, 17-35. Google Scholar

    Turnitin. (2020). https://www.turnitin.com/. Google Scholar

    Wei, R., & Hu, Y. (2019). Exploring the relationship between multilingualism and tolerance of ambiguity: A survey study from an EFL context. Bilingualism: Language and Cognition, 22(5), 1209-1219. https://doi.org/10.1017/S1366728918000998. Google Scholar

    Wei, R., Hu, Y., & Xiong, J. (2019). Effect size reporting practices in applied linguistics research: A study of one major journal. Sage Open, 9(2), 1-11. https://doi.org/10.1177/2158244019850035. Google Scholar

    Zou, B., & Jiang, G. (2021). The impact of EAP skills on students’ academic study. International Journal of English for Academic Purposes, 1(1), 57-80. Google Scholar

    References

    Ai, H., & Lu, X. (2010). A web-based system for automatic measurement of lexical complexity [Paper presentation]. 27th Annual Symposium of the Computer-Assisted Language Consortium (CALICO-10). Amherst, MA. 8-12 June. Google Scholar

    Anthony, L. (2017). AntFileConverter (Version 1.2.1) [Windows]. Tokyo, Japan: Waseda University. https://www.laurenceanthony.net/software. Google Scholar

    Biber, D., & Egbert, J. (2018). Register variation online. Oxford University Press. Google Scholar

    Bretag, T., Harper, R., Burton, M., Ellis, C., Newton, P., Rozenberg, P., Saddiqui, S., & van Haeringen, K. (2018). Contract cheating: A survey of Australian university students. Studies in Higher Education, 1-20. http://doi.org/10.1080/03075079.2018.1462788. Google Scholar

    Clarke, R., & Lancaster, T. (2006). Eliminating the successor to plagiarism? Identifying the usage of contract cheating sites. Proceedings of 2nd International Plagiarism Conference, Northumbria Learning Press, 19-21. Google Scholar

    Clarke, R., & Lancaster, T. (2007). Establishing a systematic six-stage process for detecting contract cheating. Proceedings of the 2nd International Conference on Pervasive Computing and Applications, Curran Associates, 342-347. http://doi.org/10.1109/ICPCA.2007.4365466. Google Scholar

    Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Sage. Google Scholar

    Coulthard, M. (2004). Author identification, idiolect, and linguistic differences. Applied Linguistics, 25(4), 431-447. Google Scholar

    Crawford, W., McDonouogh, K., & Brune-Mercer, N. (2018). Identifying linguistic markers of collaboration in second language peer interaction: A lexico-grammatical approach. TESOL Quarterly 53(1), 180-207. Google Scholar

    Crossley, S. A., & McNamara, D. S. (2009). Computational assessment of lexical differences in L1 and L2 writing. Journal of Second Language Writing, 18, 119-35. Google Scholar

    Crossley, S. A., & McNamara, D. S. (2011). Shared features of L2 writing: Intergroup homogeneity and text classification. Journal of Second Language Writing, 20, 271-285. Google Scholar

    Curtis, G. J., & Clare, J. (2017). How prevalent is contract cheating and to what extent are students repeat offenders? Journal of Academic Ethics, 15, 115-124. http://doi.org/10.1007/s10805-017-9278-x. Google Scholar

    Dawson, P., & Sutherland-Smith, W. (2018). Can markers detect contract cheating? Results from a pilot study. Assessment & Evaluation in Higher Education, 43(2), 286-293. Google Scholar

    Dawson, P., & Sutherland-Smith, W. (2019). Can training improve marker accuracy at detecting contract cheating? A multi-disciplinary pre-post study. Assessment & Evaluation in Higher Education, 44(5), 715-725. http://doi.org/10.1080/02602938.2018.1531109. Google Scholar

    Eckstein, G., & Ferris, D. (2018). Comparing L1 and L2 texts and writers in first-year composition. TESOL Quarterly, 52(1), 137-162. Google Scholar

    Fishman, T. (2009). “We know it when we see it” is not good enough: Toward a standard definition of plagiarism that transcends theft, fraud and copyright [Paper presentation]. 4th Asia Pacific conference on educational integrity: Creating an inclusive approach. University of Wollongong, 28-29 September. https://ro.uow.edu.au/cgi/viewcontent.cgi?article=1037&context=apcei. Google Scholar

    Grant, L., & Ginther, A. (2000). Using computer-tagged linguistic features to describe L2 writing differences. Journal of Second Language Writing, 9(2), 123-145. Google Scholar

    Ison, D. C. (2020). Detection of online contract cheating through stylometry: A pilot study. Online Learning, 24(2), 142-165. https://doi.org/10.24059/olj.v24i2.2096. Google Scholar

    Juola, P. (2017). Detecting contract cheating via stylometry methods. In R. K. Zeenath, H. Christopher and T. Foltýnek (Eds.), Plagiarism across Europe and beyond 2017 (pp. 187-198). Mendel University in Brno. Google Scholar

    Lancaster, T. (2019). Profiling the international academic ghost writers who are providing low-cost essays and assignments for the contract cheating industry. Journal of Information, Communication and Ethics in Society, 17(1), 72-86. Google Scholar

    Lancaster, T., & Clarke, R. (2016). Contract cheating: The outsourcing of assessed student work. In T. Bretag (Ed.), Handbook of academic integrity (pp. 639-654). Springer. Google Scholar

    Lines, L. (2016). Ghostwriting guaranteeing grades? The quality of online ghostwriting services available to tertiary students in Australia. Teaching in Higher Education, 21(8), 889-914. Google Scholar

    Lu, X. (2012). The relationship of lexical richness to the quality of ESL learners’ oral narratives. The Modern Language Journal, 96(2), 190-208. Google Scholar

    McCabe, D. (2020). Statistics. International Center for Academic Integrity, www.academicintegrity.org/statistics/. Google Scholar

    Medway, D., Roper S., & Gillooly, L. (2018). Contract cheating in UK higher education: A covert investigation of essay mills. British Education Research Journal, 44(3), 393-418. Google Scholar

    Morris, E. J. (2018). Academic integrity matters: Five considerations for addressing contract cheating. International Journal for Academic Integrity, 14(15), 1-12. Google Scholar

    Newton, P. (2018). How common is commercial contract cheating in higher education and is it increasing? A systematic review. Frontiers in Education, 3, 1-18. Google Scholar

    Newton, P. M., & Lang, C. (2016). Custom essay writers, freelancers, and other paid third parties. In T. Bretag (Ed.), Handbook of academic integrity (pp. 249-271). Springer. Google Scholar

    Pecorari, D. (2003). Good and original: Plagiarism and patchwriting in academic second-language writing. Journal of Second Language Writing, 12, 317-345. Google Scholar

    Plonsky, L. (2015). Statistical power, p values, descriptive statistics, and effect sizes: A ‘back-to-basics’ approach to advancing quantitative methods in L2 research. In L. Plonsky (Ed.), Advancing quantitative methods in second language research (pp. 23-45). Routledge. Google Scholar

    Plonsky, L., & Oswald, F. (2014). How big is “big”? Interpreting effect sizes in L2 research. Language Learning, 64, 878-912. https://doi.org/10.1111/lang.12079. Google Scholar

    Radunovich, H., Baugh, E., & Turner, E. (2009). An examination of students’ knowledge of what constitutes plagiarism. NACTA Journal, 53(4), 30-35. Google Scholar

    Rigby, D., Burton, M., Balcombe, K., Bateman, I., & Mulatu, A. (2015). Contract cheating and the market in essays. Journal of Economic Behavior & Organization, 111, 23-37. Google Scholar

    Rogerson, A. M. (2017). Detecting contract cheating in essay and report submissions: Process, patterns, clues and conversations. International Journal for Academic Integrity, 13(10), 1-17. Google Scholar

    Silva, T. (1993). Toward an understanding of the distinct nature of L2 writing: The ESL research and its implications. TESOL Quarterly, 27(4), 657-677. Google Scholar

    Sivasubramaniam, S., Kostelidou, K., & Ramachandran, S. (2016). A close encounter with ghost-writers: An initial exploration study on background, strategies and attitudes of independent essay providers. International Journal for Academic Integrity, 12(1), 1-14. Google Scholar

    Staples, S., & Reppen, R. (2016). Understanding first-year L2 writing: A lexicogrammatical analysis across L1s, genres, and language ratings. Journal of Second Language Writing, 32, 17-35. Google Scholar

    Turnitin. (2020). https://www.turnitin.com/. Google Scholar

    Wei, R., & Hu, Y. (2019). Exploring the relationship between multilingualism and tolerance of ambiguity: A survey study from an EFL context. Bilingualism: Language and Cognition, 22(5), 1209-1219. https://doi.org/10.1017/S1366728918000998. Google Scholar

    Wei, R., Hu, Y., & Xiong, J. (2019). Effect size reporting practices in applied linguistics research: A study of one major journal. Sage Open, 9(2), 1-11. https://doi.org/10.1177/2158244019850035. Google Scholar

    Zou, B., & Jiang, G. (2021). The impact of EAP skills on students’ academic study. International Journal of English for Academic Purposes, 1(1), 57-80. Google Scholar


    Details

    Author details

    Bailey, Kathleen

    Csomay, Eniko