# Badini Kurdish AI Translation Test 2026

77 everyday spoken lines in English and Duhok Badini (Northern Kurdish, Arabic script). A native speaker corrected and approved every Badini line by hand. Each line comes with the answers of ChatGPT, Gemini and Claude, collected on 8 November 2026.

- Runs: one new private chat per assistant, default settings, same prompt (prompt-en2ku.txt, prompt-ku2en.txt). ChatGPT free default, Gemini free "Auto", Claude Sonnet 5.5 (free default) and Opus 5.5 (paid; English to Badini only).
- Files: data.jsonl (id, en, ku_badini_reference, en_to_ku, ku_to_en), raw/ (replies as returned), results.json, score.py.
- Reproduce: `python score.py data.jsonl raw/*.txt`
- Metrics: chrF (char 6-grams, beta 2) against the reference; sorani_leak_pct (a fixed list of Sorani forms that has a different Badini form; it flags 0 of the 77 reference lines); sorani_spelling_pct (lines using the letters U+06B5 / U+0695; the reference uses them in 1 line).
- Caveats: one run per assistant on a small set, so differences of a few points are noise. The Badini references were first drafted with Gemini and then corrected by hand (may favour Gemini on chrF). The English lines were drafted with Claude (may favour Claude on Badini to English).

License: CC BY 4.0. Credit: Serdar Dilshad, https://serdardilshad.dev/en/datasets/badini-ai-test-2026