9
declared evals
25
pass
33
fail
8
degraded
66
total verdicts
Coverage — declared evals + latest verdict
| Eval | Target capability | Trigger | Assertions | Latest | Score |
|---|---|---|---|---|---|
| Eval.CatalogExtract.CaravanTyre | Workflow.CatalogExtract.CaravanTyre | Workflow | 5 | none | |
| Eval.CatalogExtract.SolarPanel | Workflow.CatalogExtract.SolarPanel | Workflow | 7 | Pass | 100 |
| Eval.CatalogResearch.CaravanTyre | Workflow.CatalogResearch.CaravanTyre | Workflow | 11 | none | |
| Eval.CatalogResearch.SolarPanel | Workflow.CatalogResearch.SolarPanel | Workflow | 13 | none | |
| Eval.DailyBrief | Workflow.DailyBrief | Workflow | 9 | Pass | 100 |
| Eval.DeepResearch | Tool.DeepResearch | Tool | 3 | Fail | 67 |
| Eval.HealthReview | Skill.HealthReview | Workflow | 4 | Pass | 100 |
| Eval.ModelRnDReport | Workflow.ModelRnD | Workflow | 9 | none | |
| Eval.ResearchFinding | Workflow.Research | Workflow | 10 | Fail | 90 |
Recent verdicts
| Eval | Target | Tier | Outcome | Score | Failed assertions |
|---|---|---|---|---|---|
| Eval.CatalogExtract.SolarPanel | Workflow.CatalogExtract.SolarPanel | Quality | Pass | 100 | |
| Eval.CatalogExtract.SolarPanel | Workflow.CatalogExtract.SolarPanel | Quality | Fail | 86 | OutputIsContentType: contenttype '' != 'ContentType.SolarPanel' |
| Eval.CatalogExtract.SolarPanel | Workflow.CatalogExtract.SolarPanel | Quality | Fail | 29 | OutputIsContentType: no row landed (output-is-contenttype cannot be checked) | RequiredFields: no row landed (required-fields cannot be checked) | FieldInEnum: no row landed (field-in-enum cannot be checked) | FieldInEnum: no row landed (field-in-enum cannot be checked) | FieldInEnum: no row landed (field-in-enum cannot be checked) |
| Eval.DailyBrief | Workflow.DailyBrief | Quality | Pass | 100 | |
| Eval.ResearchFinding | Workflow.Research | Quality | Fail | 90 | OutputIsContentType: no row landed (output-is-contenttype cannot be checked) |
| Eval.DeepResearch | Tool.DeepResearch | Quality | Fail | 67 | MinCount: found 0 of 'http', need >= 3 |
| Eval.HealthReview | Skill.HealthReview | Quality | Pass | 100 | |
| Eval.DailyBrief | Workflow.DailyBrief | Quality | Pass | 100 | |
| Eval.ResearchFinding | Workflow.Research | Quality | Fail | 90 | OutputIsContentType: no row landed (output-is-contenttype cannot be checked) |
| Eval.DeepResearch | Tool.DeepResearch | Quality | Fail | 67 | MinCount: found 0 of 'http', need >= 3 |
| Eval.HealthReview | Skill.HealthReview | Quality | Pass | 100 | |
| Eval.DailyBrief | Workflow.DailyBrief | Quality | Fail | 89 | Forbid: forbidden '[Echo agent' present |
| Eval.ResearchFinding | Workflow.Research | Quality | Fail | 80 | Forbid: forbidden '[Echo agent' present | OutputIsContentType: no row landed (output-is-contenttype cannot be checked) |
| Eval.DeepResearch | Tool.DeepResearch | Quality | Fail | 67 | Forbid: forbidden '(synthesis unavailable)' present |
| Eval.HealthReview | Skill.HealthReview | Quality | Degraded | 0 | OutputIsContentType: endpoint degraded: SqlException - Execution Timeout Expired. The timeout period elapsed prior to completion of the operation or the server is not responding. |
| Eval.DailyBrief | Workflow.DailyBrief | Quality | Fail | 89 | Forbid: forbidden '[Echo agent' present |
| Eval.ResearchFinding | Workflow.Research | Quality | Fail | 80 | Forbid: forbidden '[Echo agent' present | OutputIsContentType: no row landed (output-is-contenttype cannot be checked) |
| Eval.DeepResearch | Tool.DeepResearch | Quality | Fail | 67 | Forbid: forbidden '(synthesis unavailable)' present |
| Eval.HealthReview | Skill.HealthReview | Quality | Degraded | 0 | OutputIsContentType: tool-loop provider degraded to the Echo stub mid-run (real provider 504/timeout) - infra, not subject |
| Eval.DailyBrief | Workflow.DailyBrief | Quality | Fail | 89 | Forbid: forbidden '[Echo agent' present |
| Eval.ResearchFinding | Workflow.Research | Quality | Fail | 80 | Forbid: forbidden '[Echo agent' present | OutputIsContentType: no row landed (output-is-contenttype cannot be checked) |
| Eval.DeepResearch | Tool.DeepResearch | Quality | Fail | 67 | Forbid: forbidden '(synthesis unavailable)' present |
| Eval.HealthReview | Skill.HealthReview | Quality | Fail | 75 | ToolTrajectory: 'Tool.FetchKnowledge' not called in the declared order; actual [] |
| Eval.HealthReview | Skill.HealthReview | Quality | Degraded | 0 | OutputIsContentType: tool-loop provider degraded to the Echo stub mid-run (real provider 504/timeout) - infra, not subject |
| Eval.HealthReview | Skill.HealthReview | Quality | Degraded | 0 | OutputIsContentType: tool-loop provider degraded to the Echo stub mid-run (real provider 504/timeout) - infra, not subject |
| Eval.HealthReview | Skill.HealthReview | Quality | Degraded | 0 | OutputIsContentType: tool-loop provider degraded to the Echo stub mid-run (real provider 504/timeout) - infra, not subject |
| Eval.DailyBrief | Workflow.DailyBrief | Quality | Fail | 89 | Forbid: forbidden '[Echo agent' present |
| Eval.DeepResearch | Tool.DeepResearch | Quality | Fail | 67 | Forbid: forbidden '(synthesis unavailable)' present |
| Eval.HealthReview | Skill.HealthReview | Quality | Degraded | 0 | OutputIsContentType: tool-loop provider degraded to the Echo stub mid-run (real provider 504/timeout) - infra, not subject |
| Eval.HealthReview | Skill.HealthReview | Quality | Fail | 75 | ToolTrajectory: 'Tool.FetchKnowledge' not called in the declared order; actual [] |
| Eval.DailyBrief | Workflow.DailyBrief | Quality | Fail | 89 | Forbid: forbidden '[Echo agent' present |
| Eval.DeepResearch | Tool.DeepResearch | Quality | Fail | 67 | Forbid: forbidden '(synthesis unavailable)' present |
| Eval.HealthReview | Skill.HealthReview | Quality | Degraded | 0 | OutputIsContentType: tool-loop provider degraded to the Echo stub mid-run (real provider 504/timeout) - infra, not subject |
| Eval.HealthReview | Skill.HealthReview | Quality | Fail | 75 | ToolTrajectory: 'Tool.FetchKnowledge' not called in the declared order; actual [] |
| Eval.HealthReview | Skill.HealthReview | Quality | Degraded | 0 | OutputIsContentType: tool-loop provider degraded to the Echo stub mid-run (real provider 504/timeout) - infra, not subject |
| Eval.DeepResearch | Tool.DeepResearch | Quality | Fail | 67 | MinCount: found 0 of 'http', need >= 3 |
| Eval.HealthReview | Skill.HealthReview | Quality | Fail | 75 | ToolTrajectory: 'Tool.FetchKnowledge' not called in the declared order; actual [] |
| Eval.DeepResearch | Tool.DeepResearch | Quality | Fail | 67 | MinCount: found 0 of 'http', need >= 3 |
| Eval.HealthReview | Skill.HealthReview | Quality | Fail | 75 | ToolTrajectory: 'Tool.FetchKnowledge' not called in the declared order; actual [] |
| Eval.HealthReview | Skill.HealthReview | Quality | Fail | 75 | ToolTrajectory: 'Tool.FetchKnowledge' not called in the declared order; actual [] |