rsvalerio/starencoder-coreml

starencoder — CoreML (.mlpackage)

CoreML conversion of bigcode/starencoder for native Apple Neural Engine / GPU inference on macOS and iOS.

Files

File	Description
model.mlpackage/	CoreML model (FP16, flexible shapes)
tokenizer.json	HF fast tokenizer

Details

• Architecture: BERT (encoder-only, trained on The Stack code dataset)
• Precision: FP16 (native ANE precision)
• Compute units: .all — CoreML schedules across ANE, GPU, and CPU
• Input shapes: batch=1..512, seq_len=1..1024 (flexible range)
• Embedding dimension: 768

Usage with cai

cai index --embed-backend swift --embed-model "rsvalerio/starencoder-coreml"

The Swift backend downloads the .mlpackage from this repo, compiles it to .mlmodelc on first run (~30-60s), and caches the compiled model for subsequent runs.

Conversion

Converted using rsvalerio/models CI pipeline with coremltools.

pip install coremltools transformers torch
python convert.py