MD084 - Invisible and discouraged Unicode characters¶
Aliases: invisible-characters
Disabled by default. This rule is opt-in: enable it explicitly with
extend-enable.
What this rule does¶
This rule detects:
- invisible Unicode characters that can cause copy/paste and rendering bugs,
confuse readers, or block
rumdlfrom applying auto-fixes. Those characters are typically introduced in your documents by copy/pasting contents from the Web, and are by nature very difficult to spot, until they create issues later on. - characters Unicode itself steers authors away from: the code points carrying the
Deprecatedproperty, and the ones UTR#20 lists as unsuitable for use with markup.
By default, to reduce the number of false positives, it reports only suspicious patterns:
- multiple consecutive invisible characters
- any invisible character at the start or end of a line
- invisible characters adjacent to any whitespace character
- deprecated and markup-unsuitable code points, anywhere they appear
It doesn't report any invisible character surrounded on both sides by visible characters, since that is how those invisible characters are typically used, so it might miss some findings.
It also doesn't report two kinds of character that are presentation rather than hidden content, because each is part of a glyph the reader actually sees:
- a variation selector that follows a base character.
U+26A0 U+FE0Frenders as the emoji⚠️, whileU+26A0on its own renders as the text glyph⚠. - a zero width joiner that fuses the characters on either side of it, as in the
rainbow flag
🏳️🌈(U+1F3F3 U+FE0F U+200D U+1F308) or👩❤️👨. Removing the joiner would split the emoji into separate glyphs.
Either one is still reported when it isn't doing that job: at the start or end of a line, next to whitespace, or with a character that draws no glyph where its base should be. A duplicated variation selector is reported, since only the first one has a base to modify, and so is one that follows an interlinear annotation delimiter.
Presentation characters and the interlinear annotation delimiters are never reported
or removed by those three triggers, but they draw no glyph either, so they still count
when looking for consecutive ones and nothing can hide behind them. The zero-width
space in ⚠️x is reported, and the auto-fix removes only the zero-width space.
With strict = true, it reports any detected invisible character anywhere.
The list of non-visible characters that this rule will trigger on is:
| Unicode Codepoint | Description |
|---|---|
| U+0000 to U+001F | C0 control characters |
| U+007F to U+009F | DEL + C1 control characters |
| U+00AD | SOFT HYPHEN |
| U+034F | COMBINING GRAPHEME JOINER |
| U+061C | ARABIC LETTER MARK |
| U+115F | HANGUL CHOSEONG FILLER |
| U+1160 | HANGUL JUNGSEONG FILLER |
| U+17B4 | KHMER VOWEL INHERENT AQ |
| U+17B5 | KHMER VOWEL INHERENT AA |
| U+180B to U+180E | Mongolian variation selectors + MONGOLIAN VOWEL SEPARATOR |
| U+200B to U+200F | ZWSP, ZWNJ, ZWJ, LRM, RLM |
| U+202A to U+202E | Bidi embedding/override controls |
| U+2060 to U+206F | WORD JOINER, invisibles, and bidi isolate controls |
| U+3164 | HANGUL FILLER |
| U+FE00 to U+FE0F | Variation Selectors (VS1..VS16) |
| U+FEFF | ZERO WIDTH NO-BREAK SPACE (BOM) |
| U+FFA0 | HALFWIDTH HANGUL FILLER |
| U+FFF0 to U+FFF8 | Reserved non-rendering specials |
| U+1BCA0 to U+1BCA3 | Shorthand format controls |
| U+1D173 to U+1D17A | Musical symbol format controls |
| U+E0000 to U+E0FFF | Tags block + Variation Selectors Supplement |
The deprecated Unicode characters, meaning every code point carrying the Deprecated
property. They still render, so the rule reports them without a fix: only the author
knows what the text should say instead.
| Unicode Codepoint | Description |
|---|---|
| U+0149 | LATIN SMALL LETTER N PRECEDED BY APOSTROPHE |
| U+0673 | ARABIC LETTER ALEF WITH WAVY HAMZA ABOVE |
| U+0F77 | TIBETAN VOWEL SIGN VOCALIC LL |
| U+0F79 | TIBETAN VOWEL SIGN VOCALIC LR |
| U+17A3 | KHMER INHERENT VOWEL SIGN AA |
| U+17A4 | KHMER INHERENT VOWEL SIGN AE |
| U+206A to U+206F | INHIBIT SYMMETRIC SWAPPING to NOMINAL DIGIT SHAPES |
| U+2329 | LEFT-POINTING ANGLE BRACKET |
| U+232A | RIGHT-POINTING ANGLE BRACKET |
| U+E0001 | LANGUAGE TAG |
The characters UTR#20 lists as unsuitable for use with markup that aren't already covered above. These are visible or structural rather than hidden, so removing one would drop content or leave a paired construct half-open. Only the two tone marks have a replacement that preserves the text exactly, and only those are auto-fixed.
| Unicode Codepoint | Description | Replacement |
|---|---|---|
| U+0340 | COMBINING GRAVE TONE MARK | U+0300 |
| U+0341 | COMBINING ACUTE TONE MARK | U+0301 |
| U+FFF9 to U+FFFB | Interlinear annotation anchor, separator and terminator | |
| U+FFFC | OBJECT REPLACEMENT CHARACTER |
U+206A to U+206F appear in both this rule's invisible list and its deprecated list. Where one of the invisible triggers already reports such a character, that finding wins, because it comes with a removal fix the deprecated finding cannot offer. Every character is reported at most once.
References:
Why this matters¶
Invisible code points are difficult to spot in reviews and can produce subtle behavior differences across editors, terminals, and renderers. Catching them early keeps Markdown content predictable and easier to maintain.
Examples¶
Incorrect (default mode)¶
Correct¶
Configuration¶
strict: Whentrue, report any invisible character occurrence. The deprecated and markup-unsuitable code points are reported in both modes. Default:false.allow: Allow-list of codepoints to ignore, written asU+XXXX,U+XXXXX, orU+XXXXXX(for exampleU+200B). Default:[].
Strict mode¶
Strict mode is deliberately literal, so it also reports variation selectors that
default mode treats as presentation. On content using emoji such as ⚠️ or ❤️
that means a finding per emoji, and the auto-fix would strip the selector and change
the rendered glyph. Allow-list the selector to keep those intact:
Allow-list example¶
Allow-list entries must use Unicode codepoint format. Invalid entries are rejected during config parsing.
Notes¶
- The rule checks all content, including code blocks and frontmatter.