Fixing garbled text in Redmine attachments and repository files with the "Attachments and repositories encodings" setting
Redmine displays all pages in UTF-8. When you view a text file in Redmine, Redmine first converts the content of the file to UTF-8. The "Attachments and repositories encodings" setting specifies the character encodings that Redmine tries for this conversion.
If a text file is not written in UTF-8 and this setting does not contain the encoding of the file, Redmine cannot convert the content correctly. As a result, the text is garbled.
Content that this setting applies to
Redmine uses the setting when it displays the following content.
- The content of text files attached to issues, wiki pages, and other objects
- The content of files in a repository, including the "Annotate" view
- Diffs in a repository and attached diff files (
.diffand.patchfiles)
The setting does not apply to the following items. They have their own settings in the settings of each repository.
- Commit messages. Use "Commit messages encoding".
- File names and directory names in a repository. Use "Path encoding".
Redmine does not change the stored files. The conversion happens only when Redmine displays the content.
When garbled text appears
By default, the setting is empty. In that case, Redmine treats the content as UTF-8 and replaces every byte that is not valid UTF-8 with "?".
Therefore, the following files are displayed correctly even when the setting is empty.
- Files that contain only ASCII characters
- Files written in UTF-8
Files written in another encoding are garbled. Examples are source code with comments in Shift_JIS, a text file in Windows-1252 that contains accented characters, and a file in EUC-KR or GB18030. To display these files correctly, add their encodings to the setting.
How to configure
- Sign in as a user who is an administrator, and open the "Files" tab of the "Administration" → "Settings" page
- Enter the encoding names in "Attachments and repositories encodings". To specify more than one encoding, separate the names with commas
- Click the "Save" button
Use the encoding names that Ruby supports, such as UTF-8, ISO-8859-1, Windows-1252, Shift_JIS, EUC-JP, EUC-KR, GB18030, and Big5. Redmine ignores a name that Ruby does not recognize and continues with the next name.
Examples
| Files that your team uses | Value of the setting |
|---|---|
| UTF-8 and Western European languages (English, French, German, Spanish, and others) | UTF-8, Windows-1252 or UTF-8, ISO-8859-1 |
| UTF-8 and Japanese | UTF-8, Shift_JIS, EUC-JP |
| UTF-8 and Korean | UTF-8, EUC-KR |
| UTF-8 and Simplified Chinese | UTF-8, GB18030 |
| UTF-8, Japanese, and Western European languages | UTF-8, Shift_JIS, ISO-8859-1 |
The order of the encodings matters
Redmine tries the encodings from left to right. It uses the first encoding that can convert the content to UTF-8 without an error, and it does not try the remaining encodings. If no encoding in the list succeeds, Redmine treats the content as UTF-8 and replaces the invalid bytes with "?".
Redmine does not detect the language of the text. It only checks whether the bytes are valid in each encoding. For this reason, a wrong order can produce garbled text even when the list contains the correct encoding. Follow these rules.
Put UTF-8 first. Text in other encodings is rarely valid UTF-8. Therefore, UTF-8 at the start of the list does not prevent Redmine from trying the other encodings.
Put single-byte encodings last. In ISO-8859-1, every byte is a valid character. The conversion from ISO-8859-1 never fails, so Redmine never tries the encodings that follow it. Windows-1252 and other single-byte encodings accept almost every byte and behave in nearly the same way. For example, with UTF-8, ISO-8859-1, Shift_JIS, Redmine converts a Shift_JIS file as ISO-8859-1 and the text is garbled. With UTF-8, Shift_JIS, ISO-8859-1, Redmine displays both Shift_JIS files and ISO-8859-1 files correctly in most cases.
List only the encodings that you need. Some multi-byte encodings accept the same byte sequences. For example, many files in EUC-KR are also valid in GB18030. With UTF-8, GB18030, EUC-KR, Redmine converts a Korean file in EUC-KR as GB18030 and displays wrong characters. If your team uses files in several such encodings, put the encoding that you use most often first. Redmine cannot always display the others correctly.
Notes
- For a diff, Redmine converts each line separately. Each line uses the first encoding that succeeds for that line.
- In Redmine 5.0 and later, Redmine also uses this list to guess the default value of "Encoding" on the options page when you import a CSV file.
