Local Model Implementation for Text Translation

<span className="text-emerald-500">Introduction</span>

This blog aims to support multiple languages from the start, so I have been thinking about how to translate Chinese into Japanese and English locally. While APIs can quickly fulfill my requirements, I wanted to experience it myself and see how reliable the text translations generated by local models could be.

Before diving into the details, let me say

<span className="text-amber-500">Quick Conclusion: "The translation quality from the local model is disappointingly poor"</span>

# NLLB-200 + CTranslate2（macOS, Apple Silicon）離線翻譯環境

> 適用：MacBook Pro (M-series)，含 M4 / 48GB RAM  
> 方向：日文 `jpn_Jpan` ↔ 繁中 `zho_Hant`（FLORES-200 語碼）

---

## 0) 前置需求
- macOS 13+（Apple Silicon）
- Xcode Command Line Tools（若未安裝：`xcode-select --install`）
- 可用儲存空間 ~10 GB（模型 + 轉換輸出）

---

## 1) 建立 Python 虛擬環境
```bash
python3 -m venv ~/.venvs/nllb && source ~/.venvs/nllb/bin/activate
python -m pip install -U pip wheel


python -m pip install --only-binary=:all: "torch>=2.6,<2.7"
python -m pip install -U ctranslate2 transformers sentencepiece huggingface_hub safetensors


python - << 'PY'
import torch, transformers, sentencepiece, ctranslate2
print("TORCH =", torch.__version__)
print("TRANSFORMERS =", transformers.__version__)
print("OK")
PY


# FP16（先驗證流程用）
python -m ctranslate2.converters.transformers \
  --model facebook/nllb-200-distilled-600M \
  --output_dir nllb-200-distilled-600M-ct2-fp16 \
  --quantization float16

# INT8（省 RAM/較快）
python -m ctranslate2.converters.transformers \
  --model facebook/nllb-200-distilled-600M \
  --output_dir nllb-200-distilled-600M-ct2-int8 \
  --quantization int8

mkdir -p ~/translate && cd ~/translate

curl -L -o flores200_sacrebleu_tokenizer_spm.model \
  https://opennmt-models.s3.amazonaws.com/nllb-200/flores200_sacrebleu_tokenizer_spm.model

#（可選）字典
curl -L -o dictionary.txt \
  https://opennmt-models.s3.amazonaws.com/nllb-200/dictionary.txt

The above shows my written Chinese, and the output from the local model. Essentially, the translation is poor, and it did not translate all my Chinese sentences.

At that point, I naively thought that I could seamlessly translate my blog's Chinese into Japanese and English through a local model, but it seems that is not feasible.

Out of necessity, I now have to rely on the AI API to generate other language versions for the blog.

For a blog translation of around a thousand words, I checked the dashboard; it is not expensive, around $0.0x, much cheaper than images.

My initial idea was that after inputting Chinese locally, I could effortlessly click a button to generate translations into Japanese and English, as well as audio files in three languages. <span className="text-rose-500">In terms of results, this was achieved, but the translation quality is poor, so now I need to click two buttons, which is frustrating.</span>

The current process is: typing in Chinese -> using the API to translate into Japanese and English and saving it to the database -> clicking to generate voice (the audio files are generated based on the text).

Initially, I even designed a FastAPI for text translation:

However, the results were too poor, forcing me to give up in tears.