Skip to content
UPEEC

Arabic, Chinese, Japanese, Korean and Thai NLP

Languages without spaces or with complex morphology need special tokenizers.

Advanced
Comments 0

jieba (Chinese), PyThaiNLP (Thai), CAMeL Tools (Arabic), MeCab/SudachiPy (Japanese).

Spotted a mistake or want to add detail? Suggest an improvement — Content Contributors' edits are reviewed by editors before they go live.

Discussion (0)

Sign in or join free to ask questions and discuss this topic.