Objectives

1. Create a Spanish Grammatical Error Correction system that takes in text, and outputs grammatically corrected text

   Yo no persona famoso → Yo no soy una persona famosa

2. [Stretch] Classify grammatical errors as output spans and tags

   (2,2) Missing:SUBJ; (3,4) Replace:GENDER

Datasets

COWS-L2H

• ~3.5k essays from 1.3k students in Spanish courses at UC Davis
  • 4 essay prompts spread across 4 levels of Spanish learners
• Provide token-level annotations for specific error types (e.g. gender or number agreement)
• Provide 12.7k parallel sentences, both raw and corrected by doctoral students

Unlabelled Datasets

• CAES (learner Spanish)
• CEDEL2 (L2 Spanish)
• Spanish Wikipedia

Evaluation Metrics

ERRANT

(Paper; GitHub)

1. Performs alignment using a raw sentence and a corrected sentence (either reference or hypothesis)

   

2. By computing alignment between the raw sentence and both the expert-corrected and the model-corrected sentences, we can compute an $F_{0.5}$

   $$ F_\beta = (1 + \beta^2)\frac {P*R} {(\beta^2 P) + R} $$

   $$ P = \frac {TP} {TP + FP}; R = \frac {TP} {TP + FN} $$

   • TP - model and expert suggested the same edit
   • FP - model predicted an unnecessary edit
   • FN - model missed an expert edit