Polish Error Detection and Correction Test A
About
Subtask 1: Polish Morphosyntactic Error Detection and Correction
Introduction and motivation
Polish has a rich inflectional system in which word forms express grammatical case, number, gender and agreement relations. A small change in an inflectional ending can make a sentence ungrammatical, for example by disrupting adjective–noun agreement, subject–verb agreement or the case required by a preposition. Detecting these errors requires sensitivity to the grammatical structure of the entire sentence rather than to individual words in isolation.
This task evaluates two complementary abilities: determining whether a Polish sentence is morphosyntactically correct and, where necessary, producing a precise correction while preserving its original meaning.
Subtask description
For each sentence, the system must determine whether it is morphosyntactically correct and provide a correction when needed. The decision is expressed using one of two uppercase Polish labels:
POPRAWNE— the sentence is grammatically correct under the task criteria;NIEPOPRAWNE— the sentence contains a morphosyntactic error requiring correction.
For POPRAWNE, corrected_sentence must be an empty string. For NIEPOPRAWNE, it must contain the full corrected sentence. The system should change only what is necessary to repair the error or errors, preserving the original vocabulary, meaning, word order, spelling and punctuation wherever possible.
In particular, a stylistic preference does not justify a rewrite of a grammatically correct sentence. The task tests controlled correction rather than unrestricted paraphrasing or general text editing.
Dataset
The test-A dataset consists of 421 Polish sentences based on material from UD Polish-PUD. Morfeusz 2 was involved in preparing or verifying controlled inflectional modifications. The material covers agreement, inflection and government-related phenomena, including nominal, adjectival and verbal forms.
Some sentences require no changes; others contain one or more inflectional errors. Corrections are evaluated at sentence level, which means that a system must identify all relevant errors and avoid unnecessary edits.
Input format
Each input item contains the fields id and sentence:
{
"id": "03f6c5fc-c27b-4d54-82bb-5b1614af8d21",
"sentence": "Ten rozdźwięk pomiędzy danymi gospodarczymi a dyskursem politycznym są, a przynajmniej powinien być znajomy."
}
Model answer
The response is a JSON object with two fields:
answer:POPRAWNEorNIEPOPRAWNE;corrected_sentence: the full corrected sentence forNIEPOPRAWNE, or an empty string forPOPRAWNE.
For the input above, the expected output is:
{
"answer": "NIEPOPRAWNE",
"corrected_sentence": "Ten rozdźwięk pomiędzy danymi gospodarczymi a dyskursem politycznym jest, a przynajmniej powinien być znajomy."
}
For a grammatically correct sentence, return POPRAWNE with an empty string in the corrected_sentence field. Otherwise, return NIEPOPRAWNE and provide the complete corrected sentence, rather than only the modified words or a list of edits.
Evaluation
The main metric is full-task accuracy. An item receives credit only if both the grammaticality label and the corrected_sentence value are correct. Unnecessary rewriting, missing corrections and newly introduced errors result in an incorrect sentence-level prediction.
Submission format
Submit one JSON object per line, holding id (the input item's id) followed by the answer fields: answer and corrected_sentence. For the input example above, the line is:
{"id": "03f6c5fc-c27b-4d54-82bb-5b1614af8d21", "answer": "NIEPOPRAWNE", "corrected_sentence": "Ten rozdźwięk pomiędzy danymi gospodarczymi a dyskursem politycznym jest, a przynajmniej powinien być znajomy."}
The schema of these lines is output_schema.json, which is published beside inputs.jsonl. On the endpoint route the platform builds these lines from the model's answers, adding the id. A participant uploading a file must produce the same lines.
Campaigns
Inputs
hf download ipipan/llms4eu-benchmarks-inputs \ polish_error_detection_correction/inputs.jsonl \ polish_error_detection_correction/manifest.json \ --repo-type dataset \ --revision polish_error_detection_correction-1.0.0
Definition
An example item
{
"id": "01883dba-f1aa-46d6-ab59-74c162a3398f",
"sentence": "Wyjaśnienie pochodzenia muzyki zależy od tego, jak ją definiujemy."
}
Prompt template
Each item's fields fill the placeholders, and the result is sent as the only user message: there is no system message.
Jesteś ekspertem od gramatyki języka polskiego.
Twoim zadaniem jest ocenić, czy podane zdanie jest poprawne pod względem morfoskładniowym, a jeśli nie, poprawić je w minimalny sposób.
Zasady:
1. Sprawdź m.in.: zgodność rodzaju, liczby i przypadka; zgodność podmiotu z orzeczeniem; rząd przypadka po przyimkach.
2. Nie oceniaj stylu, prawdziwości treści ani naturalności zdania.
3. Jeżeli zdanie jest poprawne, ustaw "answer" na "POPRAWNE", a "corrected_sentence" na pusty ciąg znaków.
4. Jeżeli zdanie zawiera błędy, ustaw "answer" na "NIEPOPRAWNE", a w "corrected_sentence" podaj całe zdanie poprawione minimalnie: zmień tylko formy potrzebne do usunięcia błędów, nie zmieniaj doboru słów, liczby gramatycznej, szyku, interpunkcji ani budowy zdania.
5. Zwróć wyłącznie obiekt JSON w postaci: {"answer": "POPRAWNE" lub "NIEPOPRAWNE", "corrected_sentence": "..."}
6. Nie dodawaj wyjaśnień ani komentarzy.
Zdanie: {{ sentence }}
Generation policy
The request settings for every item, and the shape a raw output must have.
{
"generation": {
"temperature": 0.0,
"max_tokens": 512,
"stop": []
},
"output_shape": {
"max_words": 200,
"allow_empty": false,
"allow_newline": true
}
}
Example output
A submission line for an item like this: the id and the answer fields described by output_schema.json (published beside inputs.jsonl). The answer key is private.
{"id": "01883dba-f1aa-46d6-ab59-74c162a3398f", "answer": "NIEPOPRAWNE", "corrected_sentence": "Ten rozdźwięk pomiędzy danymi gospodarczymi a dyskursem politycznym jest, a przynajmniej powinien być znajomy."}