Core Transforms¶

Generate a URL-safe slug from Unicode text.

Full pipeline: decode entities → transliterate → lowercase → strip non-alphanumeric → collapse separators → apply stopwords/max_length.

Shares python-slugify's core keyword parameters (separator, max_length, word_boundary, save_order, stopwords, lowercase, etc.), so slugify(text, ...) calls port directly. Note that disarm makes every parameter past text keyword-only, whereas python-slugify accepts some positionally.

Examples:

>>> slugify("Hello World!")
'hello-world'
>>> slugify("Straße nach München", lang="de")
'strasse-nach-muenchen'
>>> slugify("My Title", separator="_")
'my_title'
>>> slugify("The Big Fox", stopwords=["the"])
'big-fox'
>>> slugify("Very Long Title Here", max_length=10, word_boundary=True)
'very-long'
>>> slugify("🔥🔥🔥")
''
>>> slugify("🔥🔥🔥", default="n-a")
'n-a'
>>> slugify("🔥", default="N/A")  # default is sanitized, not returned raw
'n-a'

Unicode normalization.

Accepts a single string or a list of strings.

Examples:

>>> normalize("é", form="NFC")
'é'
>>> normalize(["é", "ño"], form="NFC")
['é', 'ño']

Replace Unicode confusable homoglyphs with target-script equivalents.

Uses Unicode TR39 confusables table. Characters without a confusable equivalent in the target script pass through unchanged (visual mapping only, not transliteration).

Examples:

>>> normalize_confusables("Ηello")  # Greek Η looks like Latin H
'Hello'
>>> normalize_confusables("раypal")  # Cyrillic р/а look like Latin p/a
'paypal'
>>> normalize_confusables("paypal", target_script="cyrillic")
'раураӏ'

Sanitize a string into a safe filename.

Transliterate → strip OS-illegal chars → collapse separators → handle reserved names (CON, NUL, etc.) → truncate respecting extension.

Examples:

>>> sanitize_filename("My Report (final).pdf")
'My_Report_(final).pdf'
>>> sanitize_filename("CON.txt")  # reserved on Windows
'_CON.txt'
>>> sanitize_filename("résumé.docx", lang="fr")
'resume.docx'

Remove diacritical marks while preserving base characters.

NFD decompose → strip combining marks → NFC recompose. Accepts a single string or a list of strings.

Examples:

>>> strip_accents("café résumé naïve")
'cafe resume naive'
>>> strip_accents(["café", "naïve"])
['cafe', 'naive']

Full Unicode case folding per CaseFolding.txt (Unicode 16.0).

Unlike str.lower(), this implements the complete Unicode Case Folding algorithm with all 1,557 status-C and status-F mappings. Covers Latin (ß→ss, ſ→s, İ→i̇), Greek (ς→σ, variant forms ϐ→β, ϑ→θ, ϕ→φ, ϖ→π, ϰ→κ, ϱ→ρ), Cyrillic, Armenian (ligature և→եւ), Georgian Mtavruli, Cherokee, Adlam, Deseret, Osage, Warang Citi, fullwidth Latin, and all Latin ligature expansions (fi→fi, fl→fl, ff→ff, ffi→ffi, ffl→ffl, ſt→st, st→st).

Equivalent to str.casefold() but executed in Rust via a compile-time PHF (perfect hash function) table. Pure-ASCII strings take a branchless fast path with no table lookup.

Examples:

>>> fold_case("Straße")
'strasse'
>>> fold_case("ΣΟΦΙΑ")
'σοφια'
>>> fold_case("find")
'find'

Normalize all Unicode whitespace variants to single ASCII spaces.

Optionally strip control characters and zero-width characters.

Examples:

>>> collapse_whitespace("  hello   world  ")
'hello world'
>>> collapse_whitespace("tabs\there\ttoo")
'tabs here too'
>>> collapse_whitespace("a\u200Bb\u200Bc")  # zero-width spaces
'abc'

Expand emoji sequences to their CLDR short-name text descriptions.

Output is always the bare CLDR short name as plain text.

Examples:

>>> demojize("I ❤️ Python 🐍")
'I red heart Python snake'

Set a global emoji provider for all demojize calls.

The provider must implement the :class:EmojiProvider protocol.

Pass None to reset to the built-in default (latest English CLDR).

.. note:: Sequence-length cap (#199). The provider's lookup() is offered a look-ahead window of at most 9 codepoints — the length of the longest built-in CLDR emoji sequence. A provider cannot match a sequence longer than 9 codepoints: the extra codepoints fall through to the built-in tables / per-codepoint handling. This cap is fixed (it sizes a stack-allocated scan window, so widening it would cost every demojize call); design custom mappings to key on ≤ 9 codepoints. Skin-tone and variation-selector modifiers trailing a matched sequence are consumed separately and do not count toward the 9.

Examples:

>>> set_emoji_provider(None)  # reset to default provider

Strip bidirectional override and formatting characters (UAX #9).

Removes: soft hyphen (U+00AD), Arabic Letter Mark (U+061C), LRM/RLM (U+200E/F), bidi embeddings/overrides (U+202A–U+202E), bidi isolates (U+2066–U+2069).

Examples:

>>> strip_bidi("hello\u200eworld")  # remove LRM
'helloworld'
>>> strip_bidi("hello\u061cworld")  # remove Arabic Letter Mark
'helloworld'
>>> strip_bidi("safe text")  # no bidi chars → unchanged
'safe text'

Strip excessive combining marks, preserving legitimate diacritics.

Caps the number of combining marks per base character at max_marks. Operates in NFD space and recomposes to NFC.

Examples:

>>> strip_zalgo("café")  # 1 combining mark — preserved
'café'
>>> strip_zalgo("Việt Nam")  # 2 marks — preserved
'Việt Nam'