fork download
  1. import re
  2.  
  3. text = "the asbestos fiber <unk> <unk| is < unusually <unk once it enters the <<unk>$% with "
  4. pattern = r"(?i)(?<!\S)(<unk>)(?!\S)|[^\w\s]+"
  5.  
  6. print(re.sub(r"\s{2,}", ' ', re.sub(pattern, r"\1", text)).lower().strip())
Success #stdin #stdout 0.03s 9668KB
stdin
Standard input is empty
stdout
the asbestos fiber <unk> unk is unusually unk once it enters the unk with