Search academic texts

Information × Registration Number 2121U003131, Article popup.category Препринт Title popup.author Tarnavskyi Maksym popup.publication 01-01-2021 popup.source_user Український католицький університет popup.source https://hdl.handle.net/20.500.14570/2707 popup.publisher Description In this work, we investigated the recent sequence tagging approach for the Grammatical Error Correction task. We compared the impact of different transformerbased encoders of base and large configurations and showed the influence of tags’ vocabulary size. Also, we discovered ensembling methods on data and model levels. We proposed two methods for selecting better quality data and filtering noisy data. We generated new training GEC data based on knowledge distillation from an ensemble of models and discovered strategies for its usage. Our best ensemble without pre-training on the synthetic data achieves a new SOTA result of an F0.5 76.05 on BEA-2019 (test), in contrast, when the newest obtained results were achieved with pre-training on synthetic data. Our best single model with pre-training on synthetic data achieves F0.5 of 73.21 on BEA-2019 (test). Our investigation improved the previous results by 0.8/2.45 points for the single/ensemble sequence tagging models. The code, generated datasets, and trained models are publicly available. popup.nrat_date 2025-05-09 Close