safe_text 3.1.0

safe_text: ^3.1.0 copied to clipboard

Fixed #

• ~1,400 English entries could never match. The input text was normalized (e.g. v→u, 1→i, @→a) but the word list was not, so any entry containing one of those characters (vagina, pervert, 2g1c, …) was silently skipped. Such entries are now kept in a separate trie and matched exactly as written. They are deliberately not normalized: many are Arabizi or similar spellings where digits stand for letters, and normalizing them would have turned 3ars/@unt into false positives on ears/aunt.
• excludedWords is now case-insensitive and also covers leet-speak spellings. excludedWords: ['BadAss'] previously had no effect; excluding badass now also allows b4dass.
• extraWords containing v, digits or symbols now match (e.g. vodka, b@nana). Previously they could never match.
• The "same word in extraWords and excludedWords" assertion is now case-insensitive.
• PhoneNumberChecker.containsPhoneNumber no longer adds up every digit in the text. It now only combines numbers that appear next to each other, and any other word ends the current number. This fixes:
  • false positives on sentences with unrelated numbers ("I have 3 cats, 4 dogs, born 2019, flat 12");
  • missed detections when a message contained more than one phone number ("call 9876543210 or 9123456780" exceeded maxLength and returned false). Separated formats like 987-654-3210, (987) 654-3210 and +91 98765 43210 are supported.
• PhoneNumberChecker.containsPhoneNumber now works on the web. It previously threw Unsupported operation because it used Isolate.run, which is unavailable on the web.

Performance #

Measured with the new benchmark/safe_text_benchmark.dart (AOT, median of 7 rounds):

• Trie construction is now linear (it used List.removeAt(0) as a queue, which was quadratic) and uses dictionary suffix links instead of copying match lists into every node.
• Matching works directly on the already-prepared characters instead of converting and lowercasing the text again, and word-boundary checks have an ASCII fast path.
• containsPhoneNumber now runs on the calling thread instead of spawning an isolate on each call (spawning alone cost ~85 µs). It uses a hand-written tokenizer and takes ~2 ms even on ~87k characters, well within a frame. It still returns a Future<bool>, so no call sites need to change.

Behavior changes #

• A dataset word containing v is no longer matched when written with u instead (e.g. uagina). Leet substitutions of plain-letter words (fvck, f@ck) are still caught.
• containsPhoneNumber may return false for text that previously only "matched" because unrelated numbers were added together.

Data #

• English dataset cleanup: Removed 331 false positives from lib/data/en.dart — everyday, non-abusive words incorrectly flagged as profanity, including:
  • Ordinary common words (cool, sad, mad, ugly, crazy, naked, vibe, style)
  • Classic "Scunthorpe problem" substring matches — real words that merely contain a bad substring (cockpit, cockatoo, cockroach, shuttlecock, niggle/niggardly, sparse/coarse/hoarse, cummings, rehearse, scunthorpe itself)
  • Clinical/anatomical terms (urea, uterus, urinal, vasectomy, placenta, spermatozoa)
  • Brand and place names (adidas, budweiser, kmart, tampax, burma, canadian, hitchcock)
  • Neutral identity/demographic terms (lgbt, lgbtq, pansexual, cisgender, latina, latinx)

Thanks to @ChrisMICDUP for reporting this in #45.

Documentation #

• Clarified that 3.0.0's lazy auto-initialization loads the full Language.english word list (lib/data/en.dart, ~13k words) on first use — not the small legacy constants/badwords.dart list (~1.7k words) that 2.x fell back to when init was never called. This is a real behavior difference for apps upgrading from 2.x without calling init (e.g. "adult" now matches, where it previously did not).
• Corrected the "Migrating from v1.x" table, which incorrectly attributed the optional/lazy-init behavior to v2.0.0 instead of 3.0.0.
• Added a "Keeping the legacy word list" section to the README showing how to opt back into the old constants/badwords.dart list via useDefaultWords: false + extraWords: badWords.

Thanks to @ecolab-neil for reporting this in #52.

• Converted from a Flutter package to a pure Dart package. The flutter SDK dependency has been removed; safe_text now works in any Dart project (native, web, server, CLI) with no Flutter requirement.
• SafeTextFilter.init is now synchronous (void instead of Future<void>). Remove the await from existing call sites:

  // Before (2.x)
  await SafeTextFilter.init(language: Language.english);
  // After (3.0.0)
  SafeTextFilter.init(language: Language.english);
  

• SafeTextFilter.containsBadWord is now synchronous (bool instead of Future<bool>). Remove the await from existing call sites.
• Lazy auto-initialization. SafeTextFilter.init is now optional — if you never call it, the filter lazily initializes with Language.english on first use (filterText / containsBadWord). Explicit init(language: ...) still takes precedence when you need a specific language.

Added #

• Package Lifecycle API:
  • SafeTextFilter.isInitialized getter — check whether word lists have been loaded and the Trie is built.
  • SafeTextFilter.reset() method — clear loaded Trie word lists and reset initialization state, useful when dynamically switching languages or releasing memory.

Documentation #

• Rewrote the pub.dev package description and keywords with terms developers actually search for, to improve discoverability.
• Removed the stale "What's New in 2.0.0" section from the README — already covered by Features/How it Works and the CHANGELOG.
• Added a pub.dev like / GitHub star callout to the README.

Data #

• Marathi dataset cleanup: Removed ~55 false positives from mr.txt — everyday words incorrectly flagged as profanity (e.g. देव God, शिशी bottle, शीर्षक title, सह with, सुगंध fragrance, उत्साही enthusiastic, दु:ख sorrow, दृश्य scene, गर्भाशय uterus, परीक्षक examiner, घाट riverbank, and more).
• Removed 2 duplicate roman-script entries (jhavadya, zawadya).
• Fixed trailing punctuation on one entry.
• Added missing common Marathi profanity: halkat / हलकट, haramzada / हरामजादा, yeda / येडा, randwa / रांडवा, लुच्चा, बेशरम, नालायक, झवाडी, धंदेवाली.

Fixed #

• Unicode rune correctness: Replaced all codeUnits (UTF-16) usage with runes (Unicode code points) throughout SafeTextFilter and AhoCorasick. This fixes incorrect indexing and potential missed/false matches for characters outside the Basic Multilingual Plane (e.g. emoji, supplementary CJK characters).
• Word-boundary checks, match-range offsets, and string reconstruction now operate on rune indices, ensuring accurate filtering for all Unicode input.

Documentation #

• Added full dartdoc coverage to AhoCorasick and TrieNode — includes algorithm overview, usage examples, and per-member descriptions.
• Added dartdoc to the Language enum and LanguageExtension — documents fromString, fileCode, and all usage patterns.
• Added inline migration table and per-method docs to the deprecated SafeText class.

Added #

• 5 new languages: Bengali (Language.bengali), Gujarati (Language.gujarati), Punjabi (Language.punjabi), Swahili (Language.swahili), and Urdu (Language.urdu).
• Language count now reaches 82 supported languages (including Language.all).

Changed #

• Refactored FullMask and PartialMask to extend a shared ObscureSymbolStrategy base class, eliminating duplicated obscureSymbol field logic.
• Added assertion that obscureSymbol must be exactly one character, giving a clear error message on misuse.

Added #

• MaskStrategy API: New sealed class with three strategies for controlling how profanity is masked:
  • MaskStrategy.full({obscureSymbol}) — replaces every character (e.g. "badass" → ``"******"`).
  • MaskStrategy.partial({obscureSymbol}) — keeps first (and last, for 4+ char words) visible (e.g. "damn" → "d**n").
  • MaskStrategy.custom({replacement}) — replaces the entire word with a fixed string (e.g. "[censored]").
• strategy parameter on SafeTextFilter.filterText and the deprecated SafeText.filterText to accept a MaskStrategy.

Changed #

• Legacy fullMode and obscureSymbol parameters on filterText are now deprecated in favour of MaskStrategy. Existing call sites continue to work unchanged.
• MaskStrategy is exported from the top-level safe_text.dart barrel file.

Documentation #

• Rewrote README with pub.dev badges, full API reference tables, supported languages list, migration guide from v1.x, How it Works section, and Limitations section.
• Fixed CHANGELOG entries for 1.0.5–1.0.7: corrected Added category to Fixed/Changed for bug fix and refactor entries.

Added #

• Aho-Corasick Engine: Replaced legacy regex loops with a high-performance Trie-based search engine for O(N) search complexity.
• Extreme Performance: Up to 20x faster for large pattern sets and mixed language inputs.
• Flexible Initialization: SafeTextFilter.init now supports either a single language or a list of languages for custom combinations.
• Multilingual Support: Support for 75+ languages with full names (e.g., Language.english, Language.hindi).
• Modular API: Separated functionality into SafeTextFilter for profanity and PhoneNumberChecker for phone detection.
• Single-Pass Replacement: Optimized string building avoids multiple copies during filtering.

Deprecated #

• SafeText Class: The monolithic SafeText class is now deprecated. Use SafeTextFilter and PhoneNumberChecker directly for better performance and modularity.

Added #

• Added support for detecting phone numbers with repeated digits using multiplier words (e.g., "double five" → "55", "triple four" → "444").

Fixed #

• Improved time complexity of the phone number detector.

Fixed #

• Fixed the filtering and matching function.

Changed #

• Moved the phone number checker into an isolate to avoid blocking the main thread.

Added #

• A method to check whether the given String contains any phone number or not both in words and digits and in mixed format also.

Added #

• A method to check whether the given String contains any bad words or not. Process will run in new thread to avoid blocking of main thread.

Added #

• Made some changes in formatting.

Added #

• Made some changes in documentation.

Published #

• Initial release of the BadWordFilter package.
• Implemented the SafeText class with the filterText method for filtering out bad words from text inputs.
• Provided customizable options for filtering bad words, including extraWords, excludedWords, useDefaultWords, fullMode, and obscureSymbol.
• Included an example Flutter app demonstrating the usage of the BadWordFilter package.
• Added a README.md file with instructions on getting started, usage, example, and author information.