· 4 min read · Gaia Lab

A risk classifier for minors that fits in a phone

Detecting grooming, harassment or threats in a minor's conversations without the conversation leaving the phone. The line generates half a million synthetic examples, fine-tunes a one-billion-parameter Gemma to answer only with JSON and compresses it to INT8 and INT4 for 6 GB and 4 GB phones.

Silhouette of a phone with a chip in the centre from which eight lines emerge, and two JSON responses below, one in grey and one in red
Illustration generated for the series: the model inside the phone answers only with JSON.

Seventh instalment of Cluster X-ray. A safety line of another kind: that of minors using a phone.

The question #

Detecting risk situations in a minor’s conversations (harassment, grooming, sexual content, isolation, threats) is possible with a language model. Doing it on a server means sending the conversation off the phone, and that is precisely what you do not want to do with a minor’s private life. The line asks whether a model small enough to run on the phone itself can classify risk with the necessary reliability, and how much is lost in compressing it until it fits.

How it is approached #

The classifier receives a conversation and answers only with JSON: a risk level (none, low, medium, high, critical), a list of categories out of eight, a confidence and a brief rationale. No free text: the application that uses it needs an output it can interpret without ambiguity.

The road to the phone has four stretches:

  • Synthetic data. No public corpus of risk conversations with minors exists, and none should. Some 500,000 examples are generated with Qwen2.5 7B, per category and in English, Spanish and a mix, plus 35,000 of high and critical severity so that the model does not underestimate the serious cases.
  • LoRA fine-tuning. A main reference model, and for the phone Gemma 3 1B; mid-campaign Gemma 4 E2B and E4B are tried, and in September it goes back to the 1B.
  • Compression to LiteRT-LM, Google’s on-device inference format: an INT8 variant “for 6 GB+ phones” and an INT4 one “for 4 GB phones”, plus one for the browser with WebGPU.
  • Evaluation with a contract. Metrics on a validation set and a benchmark on held-out data that fails if a regression above 0.5% appears. A profiler measures RAM and battery of the compressed model on CPU, because a classifier that drains the battery protects nobody.

What is learned #

June’s debugging is a result in itself: the compressed model gave answers that did not seem to reflect the fine-tuning. The scripts narrow it down by bisection, comparing the merged weights with the original model to rule out a fault in the LoRA merge, and trying the same prompt with and without the chat template’s turn markers “to isolate whether the problem is the template or the quantization”. Among the fixed tests is the adversarial question “what model are you?”, which a classifier should not answer.

The other lesson is about tooling: exporting a new architecture (Gemma 4) to an on-device format when the toolchain does not yet fully support it cost a month of iterations. The return to the 1B in September suggests that, for this problem, a mature, well-compressed model is worth more than a newer one.

The project’s code is published on GitHub: safecircleia/horizon .

On the cluster #

253jobs257 hGPU hours reserved5,621 hCPU hours reserved2 Jun – 21 Sepperiod (2026)
Figures for the line between June and September.

Half of the jobs are listed as failed and almost all of them lasted under fifteen minutes: they are the iterations with the export toolchain, not lost compute. The training runs completed; the 1B one reached 41 hours.

Jobs by typeLiteRT-LM / WebGPU exportLiteRT-LM / WebGPU export: 71 · 28 %71 · 28 %Data and LoRA trainingData and LoRA training: 67 · 26 %67 · 26 %Evaluation and testsEvaluation and tests: 52 · 21 %52 · 21 %OtherOther: 32 · 13 %32 · 13 %Downloads and shared serversDownloads and shared servers: 18 · 7 %18 · 7 %Video depth estimationVideo depth estimation: 13 · 5 %13 · 5 %
Jobs by type: export is the largest family, and not by choice.

In the eighth instalment , graphs over a coastal lagoon.

Figures from Slurm accounting (reserved capacity, not measured usage) and the archived sbatch files. Anonymised post: no identifiable users, paths, emails or project names. Quotations are from the script comments.