LazuliQ 1.2 Photon
A 3.1B-parameter model, fine-tuned to see how much reasoning fits into something this small. It won't match the large assistants, and that isn't the point. Small models that run anywhere are where a lot of this is heading.
Early beta. Photon is an experiment and will get things wrong.
What the fine-tune changes
Photon is built on the Qwen2.5-3B-Instruct base. The fine-tune changes how it thinks through an answer rather than how much it knows. The base model's ceiling is still the ceiling.
Steadier step-by-step answers
Trained on reasoning examples so it works through problems in visible steps instead of jumping to an answer. Within its depth, the difference is clear.
Calmer, cleaner tone
Alignment tuning gives replies a more neutral voice and cleaner structure than the raw base model.
More knowledge per parameter
Careful data curation pulls more recall out of a 3.1B footprint than the base model manages, though at this size retrieval matters more than memory.
A custom agentic harness
LazuliQ doesn't just chat. It runs inside a full custom agentic harness that turns a request into a visible plan, then does the work: it edits files, creates tasks, checks its own output and builds real files you can download. The model writes the content; the harness decides what happens next.
LazuliQ 1.2 PhotonBetaHi, I’m LazuliQ.
A small research model with an agent harness: I plan, write, check my own work and build real files. Still small and often wrong, so check anything that matters.
- !Plan the site
- !Write the content
- !Check the draft
- !Design and build
- !Final check
- !Change the palette
Slow coffee, made properly.
Small-batch roasts and fresh pastries at Harbour Street 12.
See the menuMenu
- Flat white€3.40
- Filter coffee€3.00
- Almond croissant€3.20
Edits files
Websites, PDFs, Word files, slides, Markdown, JSON, CSV, CSS, JS and Python. Follow-ups change the latest version, and each edit is kept as its own revision.
Creates tasks
Every request becomes a short plan shown as a timeline, with the line the model is writing right now and what each check found.
Checks its own work
Loops, placeholders, topic drift and broken files are caught by deterministic checks. Small fixes are made locally; the rest go back to the model with the exact complaint.
Uses tools
An exact calculator, Wikipedia search with citations, file reading and workspace commands. The harness does the deciding, so the small model only has to write.
Still small and often wrong, so check anything that matters. See it work in the demo.
StartumRAG
A retrieval layer that runs server-side and offline, giving a small model access to facts it was never big enough to hold. No external tool calls, no waiting on the network. It's the most experimental part of Photon, and the most interesting.
Offline and private
Runs inside isolated, internet-free server environments, so nothing leaves the machine. On-device deployment is next.
Ultra-low latency
With no external web APIs in the loop, retrieval adds very little time on top of inference.
Fewer invented answers
Grounding answers in real sources cuts down the guessing a small model would otherwise do when it hits the edge of what it knows.
Under the hood
A modern decoder-only transformer, inherited from the Qwen2.5-3B-Instruct base and built for efficiency over scale.
- Total parameters
- 3.09B
- Context length
- 128K
- Layers
- 36
- Vocabulary
- 151.6K
Grouped-query attention
16 query heads and 2 KV heads. Cuts VRAM usage sharply and speeds up inference, which is what makes a model this size worth running at all.
RoPE embeddings
Rotary positional embeddings for long context. The architecture supports the full 128K window, though quality is strongest well inside it.
SwiGLU and RMSNorm
Standard components in current LLMs. They keep training stable and inference fast.
Tied word embeddings
Keeps the footprint small, at 2.77B non-embedding parameters, without giving up linguistic nuance.
About this release
LazuliQ 1.2 Photon is a student project: a fine-tune of an open-source 3B base model, built and maintained by one developer. It's an experiment in how capable a very small model can be, not a replacement for the large assistants you already use. Expect mistakes: at this size the model can be wrong, invent details, or lose the thread of a long conversation. Everything here is beta, StartumRAG most of all, so check anything important before relying on it.
DominikWalser
AI & Technology Enthusiast
Startups & Innovation
LazuliQ is an independent project, designed, researched and developed by a single student developer with a passion for AI, LLMs and cybersecurity. Open to collaboration, research partnerships and new opportunities.