MoroLMDocumentation
Technical documentation for the MoroLM Moroccan Darija language-model project.
Work in progress: this page describes MoroLM-56M v0.6 and will change as the project moves.
Start here. This documentation records what has actually been built, what failed, what was fixed, and what remains experimental.
Documentation map#
Project map#
Repository structure, pipeline map and milestone matrix.
Architecture#
Model internals and full local stack.
Tokenizer#
24k custom Unigram vocabulary and special tokens.
Data & provenance#
Training mixture, holdouts and source policy.
Training history#
v0.1 → v0.6, including the double-shift bug.
Evaluation#
Loss curves, smoke tests and generation behavior.
Roadmap#
What comes after the v0.6 base.
Current status#
56,607,232parameters
24,000vocabulary
12,500v0.6 optimizer updates
5.2239final fixed validation loss
The current base model has demonstrated recognizable English, French, Arabic-script Darija and Latin Darija generation. It is not yet a chat model and should not be presented as factually reliable.