A development issue needs the right kind of fix. An on-call engineer needs to know whether an incident is still affecting customers. A support agent needs to recognize when someone has asked for a human to take over.
These are small decisions, but they shape how work moves between people. They connect the teams that build software, keep it running, and help customers use it: the software development lifecycle, or SDLC.
Jev and Laya offer a way to turn that context into structured answers an application can use. To look beyond the product descriptions, we ran both on the same 18 synthetic cases, divided equally across development, DevOps, and support.
Jev matched our expected answers on every question in this small suite. Laya’s results varied by question type and model configuration. The useful details are in those differences: identifying a category did not always mean correctly recognizing the current situation or its impact.
Three questions an application can use
TypeSafe’s Jev offers three question types: Choice selects from supplied options, Score rates something against an ordered scale, and Noul estimates whether a statement is true. TypeSafe recommends breaking complex judgments into specific questions and combining the answers in code. TypeSafe introduction
For an incident, those questions could identify the established cause, check whether customer requests are currently failing, and rate how much work is blocked. Each answer serves a different purpose. Knowing the cause does not establish whether the incident is still active.
Laya exposes these same question types through downloadable models, with separate versions for English, multilingual input, and typed decisions. Its model card lists Apache 2.0 licensing. Shared output vocabulary does not make the models’ internals or behavior identical. Laya model card
The operating choice differs, too. Jev is available through a hosted API; our test used OpenRouter’s decisions endpoint. We ran Laya locally on CPU through its Python library. Hosted inference puts serving infrastructure with the provider. Running downloaded weights gives the operator control over the local environment and model revision, along with responsibility for dependencies, resources, and maintenance. Jev quick start, Laya repository
What we tested
We wrote six cases for each domain and fixed the expected answers and scoring rules before running either model. Each case contained one Choice, one Noul, and one Score question. That gave each configuration 18 answers of each type.
The same case text, questions, instructions, options, and option order went to all three configurations. We ran each case once, with no answer retries, prompt tuning, or selection of a different checkpoint after seeing the results.
The questions stayed deliberately narrow:
| Domain | Choice | Noul |
|---|---|---|
| Development | Which kind of change is needed: implementation, tests, documentation, or unknown? | Is a required release check currently failing? |
| DevOps | Is the established cause application, network, capacity, or unknown? | Are customer-facing requests currently failing? |
| Support | Does the problem belong to access, billing, product, or unknown? | Does the customer explicitly request human or manager takeover? |
Every case also received the same impact scale: 0 for no workflow blocked, 1 for a blocked workflow with a working workaround, and 2 for a blocked workflow without one. This measures the stated blockage, not subjective business severity. The development cases test routing from descriptions, not the ability to review source code.
We requested Jev version typesafe/jev-1.13; the responses identified typesafe/jev-1.13-20260917. For Laya, we explicitly selected the English and typed-decisions configurations at the same model-repository revision. They are reported separately throughout. This did not test Laya’s automatic router.
Results, one question type at a time
We counted exact Choice matches. For Noul, values of 0.5 or greater counted as true. For Score, we rounded to the nearest scale position, with exact half steps rounded upward. These rules were fixed before execution.
| Configuration | Choice matches | Noul matches | Rounded Score matches | Score mean absolute error, 0–2 scale |
|---|---|---|---|---|
| Jev | 18/18 | 18/18 | 18/18 | 0.0000 |
| Laya English | 16/18 | 16/18 | 14/18 | 0.3535 |
| Laya typed-decisions | 18/18 | 16/18 | 15/18 | 0.3349 |
All configurations returned valid, available answers to every question. The mean absolute error uses the unrounded Score values, so it retains differences hidden by rounding. We did not combine the three question types into an overall ranking statistic.
These are matches against our labels on 18 short, explicit, English-language synthetic cases. The suite was neither representative of production work nor blinded. Jev’s complete match is a result worth reporting, but it does not establish perfect reliability or general superiority. This test does not measure calibration, multilingual handling, adversarial robustness, or operational value. Local CPU and hosted execution also differ too much here to support a latency ranking.
Development: the category was only part of the answer
In dev-03, a setup guide contained an obsolete command, a required documentation check was failing, and new engineers could not complete setup and had no working workaround.
All three configurations selected documentation. Jev and Laya English also matched the expected failing-check and impact answers. Laya typed-decisions returned a Noul value of 0.4226, below our true threshold, and an impact score of 1.4223, which rounded to 1 instead of the expected 2.
That difference matters if a team uses separate fields to decide where work belongs and how urgently someone should inspect it. A correct category cannot validate the other fields.
DevOps: a resolved incident still needs a current-state answer
In ops-05, yesterday’s network fault had been repaired. The case explicitly said customer requests were no longer failing and nothing was blocked.
All three configurations identified network as the established cause. Jev and Laya typed-decisions matched both remaining answers. Laya English returned 0.7926 for current request failures and 0.5162 for impact, crossing our thresholds for true and impact level 1.
There were clear agreements, too. In ops-02, a confirmed DNS fault was disrupting a portal, but a secondary portal supplied the same reports. All three matched network, current failures, and impact level 1. The presence of a working workaround was a meaningful part of the question.
Support: uncertainty and escalation are separate judgments
In support-04, the customer said something had gone wrong and requested a human, but supplied no symptom or product area. They explicitly confirmed that no workflow was blocked. Our expected answers were unknown, true, and 0.
Jev matched all three. Laya English selected product, recognized the takeover request with 0.6068, and returned impact 0.5983, which rounded to 1. Laya typed-decisions selected unknown but returned 0.2722 for takeover and impact 0.9782, also rounding to 1.
The two Laya configurations therefore missed different parts of the same case. Switching configurations improved some answers without making every field dependable.
The complete results, case definitions, and evaluation protocol include every case, expected label, and reported result.
Where this could fit in a team’s workflow
A useful first application is a suggestion someone can inspect: route a development issue, flag an incident summary for attention, or highlight an explicit support escalation request. These remain proposed integrations; we tested model responses, not deployed workflows or time savings.
For team leads, the question is whether a suggestion reduces handoff time without adding costly mistakes. For practitioners, it is which inputs, thresholds, and checks make that suggestion useful. Where existing metadata already establishes an owner or alert status, use it directly.
Our next step would be to evaluate one candidate workflow against a separate set of real historical cases, labeled before inspecting outputs. Include incomplete descriptions, resolved incidents, and the consequential exceptions people currently catch. Check every field that will influence an action.
Then record suggestions alongside the existing process before letting them change assignments or escalations. Measure missed cases, unnecessary reviews, and actual time saved. Keep merging, rollback, and account actions under their existing controls.
A numerical output also needs interpretation. TypeSafe documents Jev’s Choice and Score confidence as derived from the returned distribution; Noul has no separate confidence field in that API. A concentrated distribution is not evidence that the answer is correct. Our thresholded matches do not establish that either model’s probabilities are calibrated. TypeSafe confidence documentation
Jev matched this suite fully, while Laya exposed differences worth investigating before integration. The practical decision still depends on the workload, the cost of mistakes, and who will operate the system. Start with one recurring handoff and measure whether the model helps the people responsible for it.
Run scope: Laya 0.3.4, source commit 42626c348753fbb17572a813127df2278a1ec527, model revision 1c5edc17a7acd8701df6fc341c0d179f1c62c982, CPU execution. The linked protocol and results describe the matched suite; earlier installation smoke checks are not included in these match counts.