MoroLMProject map
A map of the MoroLM repository, research pipeline and milestone status.
Work in progress: this page describes MoroLM-56M v0.6 and will change as the project moves.
Research pipeline map#
1. Source auditlicense + provenance
2. Cleaningdedupe + quarantine
3. Tokenizer24k frozen UGM
4. Packing512-token blocks
5. Pretrainingnext-token LM
6. Evaluationfixed + external
7. ExportHF → GGUF
Repository map#
atlas-local-ai/
├── tokenizer/
│ ├── atlas_tokenizer.model
│ └── atlas_tokenizer-v0.1.model
├── model/
│ ├── configuration.py
│ └── atlas_model.py
├── training/
│ ├── pretrain_v2.py
│ ├── datasets.py
│ └── checkpoint.py
├── configs/
│ └── morolm56m-base-v0.6-cleanstart.yaml
├── data/
│ ├── staging/
│ │ └── v0.5/train_candidate.jsonl
│ ├── evaluation/fixed/
│ └── evaluation/external/
├── checkpoints/
│ ├── historical v0.1–v0.5 artifacts
│ └── morolm-56m-base-v0.6-cleanstart/
├── models/
│ └── exported HF / GGUF artifacts
├── Go backend (planned/in progress)
└── browser UI (planned/in progress)Internal paths still preserve the historical Atlas naming. Public documentation uses MoroLM.
Milestone matrix#
| Area | Status | Notes |
|---|---|---|
| Custom tokenizer | Working | 24k SentencePiece Unigram, frozen. |
| 56M architecture | Working | Llama-like decoder, GQA, RoPE, RMSNorm, SwiGLU. |
| Correct causal objective | Verified | Alignment smoke + 500-step memorization proof. |
| Clean base model | v0.6 | 12,500 updates from random weights. |
| Held-out evaluation | Partial | Fixed historical + external sets exist; natural Darija suite is next. |
| Instruction model | Not yet | Earlier SFT probes predate the clean base lineage. |
| GGUF runtime | Pipeline proven | Parity work exists; v0.6 export is a future release step. |
| Production app | Future | Go + llama.cpp + SQLite + streaming web UI target. |
Clean lineage rule#
Base checkpoints v0.1 through v0.5 must not be used as parents for the clean MoroLM base lineage because they were trained with the faulty two-token-ahead objective. They remain useful only as historical/debugging artifacts.