MD094 - Invalid encoding¶
Aliases: invalid-encoding
Enabled by default. This rule reports bytes that are not valid UTF-8.
Why this matters¶
A file saved in a legacy encoding such as Latin-1 or Windows-1252, or one with a truncated multi-byte character, renders with replacement characters or fails to build in most Markdown tooling. Such a file is usually one stray character away from being valid, and the rest of it is still Markdown worth checking.
How it works¶
rumdl decodes the file, replacing each invalid byte sequence with U+FFFD, and lints the decoded text with every enabled rule. MD094 reports each invalid sequence at its line and column, naming the bytes:
The first 20 sequences are reported individually. The rest are summarized in
one finding, such as 5 more invalid UTF-8 sequences not shown. Sequences
hidden by a rumdl-disable comment are not counted.
A file with invalid UTF-8 is never written. The replacement characters stand
for bytes the file really holds, so writing the decoded text back would destroy
them. check --fix and fmt report the file and leave it byte-for-byte
unchanged, other rules' findings in it are reported without a fix, and
fmt --check and --diff print no diff for it. With --stdin, check --fix
and fmt echo the input unchanged.
check and check --fix exit 1 while the file has an unsuppressed finding,
unless --fail-on excludes warnings. fmt keeps its formatter-style exit
code 0.
Binary and UTF-16 files¶
Invalid UTF-8 that looks binary is not linted at all, because decoded binary
data produces a flood of meaningless findings from other rules. A file counts
as binary when it is not valid UTF-8 and either starts with a UTF-16 byte order
mark (FF FE or FE FF) or contains a NUL byte in its first 8000 bytes. It
gets one MD094 finding at line 1, column 1:
logo.md:1:1: [MD094] File appears to be binary; not linted
notes.md:1:1: [MD094] File appears to be UTF-16 encoded; not linted, convert it to UTF-8
A binary file is never written either. Because it is not linted, inline
rumdl-disable comments in it are not read: use configuration or per-file
ignores to silence it. Valid UTF-8 is never treated as binary, whatever bytes
it contains.
How to fix¶
Convert the file to UTF-8, for example with iconv:
Most editors can also reopen a file in its original encoding and save it as UTF-8.
Configuration¶
This rule has no options. The standard rule controls apply, including
disable, extend-disable, enable, severity overrides, inline disable
comments (except in binary files, see above), and per-file ignores:
Disabling MD094 hides its findings but does not make an invalid file writable:
check --fix and fmt still leave it unchanged.
Rust source files (.rs), whose doc comments rumdl lints, must still be valid
UTF-8. Invalid bytes in one are reported as a read error, since the Rust
compiler rejects such a file too.