threefold
Menu

Blog

Translated isn't localised: what the i18n readiness check looks for

· threefold · 5 min read

An app can have a translation file for every target language and still not be ready to ship in any of them. The keys can be out of step with the base language. A placeholder can be renamed in one locale, so a greeting renders as a literal {userName}. A price can be built as '$' + price. A date can be hardcoded as MM/DD/YYYY. Russian needs plural categories English does not have. A Mexican storefront can be written in the Spanish of Spain. A German checkout button can be labelled in a way that German law says does not count.

None of that shows up as "untranslated". All of it shows up to users.

threefold's i18n readiness check is one command that looks for these problems in your project:

npx threefold check . --markets es-MX,de-DE

It runs on your machine. Your source code is never uploaded; if you pass --upload, only the report is sent.

What it looks for

There are 37 checks, each with a source. They fall into a few groups.

Core string hygiene. Hardcoded UI strings, keys missing from a locale, orphaned keys, placeholder and ICU token mismatches, CLDR plural categories, currency and number formatting done by concatenation, hardcoded date patterns, and copy that is untranslated or in the wrong language.

Platform specifics. For iOS: String Catalog states (a string marked needs_review is not done), .strings and .stringsdict syntax, localised permission strings in InfoPlist.strings, and the development region. For Android: locale qualifier directories (values-es-rMX, not values-es-MX), missing translations, plural quantities and per-app language preferences. For the web: key parity across i18next, next-intl and FormatJS files, literal JSX strings, <html lang> and hreflang alternates. For Flutter: ARB parity, placeholder types and l10n.yaml. For React Native: Expo locales against native strings.

Store metadata. App Store and Google Play field limits per locale: an App Store name is 30 characters, keywords are 100 bytes, a Play short description is 80 characters.

Spanish variants. es-ES, es-MX and es-AR are different markets. The check warns on vosotros forms in Latin American copy, on tú imperatives where Argentine calls to action use voseo (descargá, not descarga), and on vocabulary that marks a variant, such as móvil and celular. Each is a warning with the matched text, not a silent fail.

Market legal items. Germany's Impressum, withdrawal-right copy, order-button wording and German privacy policy; Japan's 特定商取引法に基づく表記 page; Argentina's Botón de arrepentimiento; consumer terms for Spain and Mexico. These are always marked for human review. The check reports what it found and links the law. It never says "compliant", and the report says it is not legal advice.

Runtime checks go further when you let them: a pseudo-locale run of your web app in a real browser that flags clipped and overflowing text, and command builders for the iOS double-length pseudolanguage and the Android en-XA pseudolocale.

No readiness percentage

The report deliberately has no single blended score. A percentage that averages passes, failures and things nobody checked tells you nothing about whether the app is ready.

Instead, every check reports pass, fail, warn, unverified or not_applicable, with file, line and a fix hint. verified_coverage says what share of the applicable checks actually ran, kept separate from how many passed. The verdict for each market is one of three words. complete means every applicable static and runtime check passed and nothing required is unverified. complete_static means the static checks pass but the runtime checks did not run. incomplete means something failed. Human-review items sit next to the verdict as "still needs a person", so they are never quietly counted as done.

The localise-app skill has a coding agent run the check, fix the failures, translate with market resources, and run it again until the verdict is complete or complete_static, and it tells the agent never to invent pass results.

How we measured it

"If threefold says complete, it probably is" has to be measured. We ran the checks against a pinned corpus of four open-source projects, each recorded with the commit we checked out:

On top of the real projects we made copies with hand-inserted defects, one kind per check: an ICU token corrupted from {userName} to {invalidUser}, a price built as '$' + price, an Android directory named values-es-MX instead of values-es-rMX, an App Store name 68 characters long, a removed l10n.yaml, a fixed 40-pixel container that clips text in the browser.

The bar for a check to ship as production is 90% precision and 90% recall on at least five verified positives. Anything below it is labelled beta in every report until it improves.

The results

Of the 27 checks we could measure, 19 met the bar with 100% precision and 100% recall. Eight are labelled beta. Their recall was also 100%, but they raised false positives: the hardcoded-strings check had a precision of 71% (5 true positives, 2 false positives), and seven others, including missing keys, String Catalog state, Android missing translations, literal JSX strings, html lang and the store limits check, were at 83%. The iOS and Android runtime checks were not measured, because the run had no simulator or emulator.

Two caveats matter. Five positives per check is a small sample, so these numbers will move as the corpus grows. And seeded defects are cleaner than the mistakes people actually make, so 100% recall on them is a floor for what we have tested, not a promise that nothing slips through. The full table, with true and false positive counts for every check and the way each defect was seeded, is in the threefold repository at docs/research/i18n-benchmark.md.

Try it

Run it on your project:

npx threefold check . --markets es-MX,es-AR,de-DE,ja-JP

The i18n check page and the docs have the options, the CI gate and the full list of checks.

More from the blog · RSS