VEDKUBER / TRANSPARENCY / CALIBRATION REPORT
CONFIDENCE CALIBRATION REPORT — MONTHLY, MEASURED ONLY

How honest are our confidence numbers?

Every AI prediction carries a confidence percentage. This page asks one question of that number: when the system says 70%, is it right about 70% of the time? The answer is measured from graded predictions — nothing estimated, nothing invented. Where measurement is thin or not yet possible, we write "n/a" instead of guessing.

Current calibration snapshot

ECE — RAW → CALIBRATED
—
Source: /api/ai/confidence-calibration, holdout evaluation. Lower is closer to honest; 0 is perfectly calibrated.
BRIER SCORE — RAW → CALIBRATED
—
Source: same holdout. Measures mean squared error of stated probability vs outcome (0 = perfect, 0.25 = coin flip).
90% PREDICTION-BAND COVERAGE
—
Source: conformal band measured on the same holdout. Target: ≥ 90% of outcomes fall inside the published band.

Reliability curve — raw confidence vs reality

Each point below is a bucket of graded predictions. The horizontal position is what the raw model claimed; the vertical position is how often those predictions were actually right. A perfectly calibrated system sits on the dashed diagonal. Ours does not — and we show that.

loading measured data…

Solid line: raw confidence buckets (claimed → measured). Dashed diagonal: perfect calibration. Dots: individual buckets, dot size roughly proportional to how many graded predictions sit in the bucket.

CLAIMED (RAW)MEASURED HIT RATEGRADED N
loading measured data…

Before and after calibration

Calibration does not make the model smarter — it makes the number it prints honest. The measured effect of remapping raw scores through the isotonic map (fit on graded history, verified on a holdout the mapping never saw):

METRICBEFORE (RAW)AFTER (CALIBRATED)WHAT IT MEANS
Expected Calibration Error (ECE)——Raw confidence claimed far more than it delivered; the calibrated number sits close to measured reality.
Brier score——Overall probability quality; 0.25 is what a pure coin flip scores. The calibrated number is honest about that.
Band coverage (target 90%)——Share of outcomes that fell inside the published uncertainty band on the holdout.

The measured truth, from the calibration engine: raw confidence carried almost no signal — buckets claiming 60–90% were right roughly half the time. Calibration makes the printed number honest (it now says "close to a coin flip" when that is what the data supports); the trading edge comes from the model, not from the confidence label.

Monthly history

Month-by-month measured record of calibration quality. History begins when this measurement pipeline started publishing; months without enough graded rows are written as n/a. Per-month band coverage is n/a by definition of the aggregate — conformal coverage needs row-level pairs and is only measured on the holdout snapshot above.

MONTHGRADED NHIT RATEECE RAWECE CALIBRATEDBRIER RAWBAND COVERAGE
loading measured data…
Chart: monthly ECE, raw (wide line) versus calibrated (narrow line) — data: monthly aggregates of outcome-graded prediction logs.

Methodology — in our own words

FAQ

What is confidence calibration?
It is the check that a stated confidence matches reality: predictions labelled 70% confident should be right about 70% of the time. Calibration remaps the stated number against graded outcomes so it stops overclaiming.

What is ECE?
Expected Calibration Error: group graded predictions by claimed confidence, measure the gap between claim and reality in each group, average the gaps. Zero is perfectly honest; our raw number was not, and this page shows the measured fix.

Why does VedKuber publish its calibration error?
Because an unverifiable confidence number is marketing, not measurement. Publishing the error monthly — including unflattering months — lets anyone check whether our numbers deserve the trust we ask for.

Why is some coverage "n/a"?
Band coverage needs each individual prediction's outcome compared against its own published band. Monthly aggregates cannot reconstruct that row-level comparison, so we publish n/a rather than an invented number. It is measured on the holdout snapshot, where it is 92.78% against a 90% target.

Disclaimers

Related: Monthly Transparency Report · AI Transparency & Live Reports · Track Record · Rating System

NAVIGATE EVERYTHING

Explore every corner of VedKuber

Every public page, floating in one place. Tap a bubble to jump — no digging through menus.

Home About Academy AI Engine AI Transparency API Status Architecture AutoPilot Learn Stock Market Calibration Report Careers Comparisons Compliance Contact Daily Report Docs FAQ Features Global Markets हिंदी How It Works Infrastructure Legal AI Disclosure Broker Disclosure Legal Privacy Refund Policy Risk Disclosure Security Practices Terms of Service Market Terminal Market Updates Monthly Report NVIDIA Inception Paper Trading Paper Trading for Students Pricing Privacy Ratings Real Auto Risk Management For Schools Security Support System Status Teacher Mode Trading Modes Trading Terminal Transparency VedSAAS Virtual vs Real Weekly Report What Is Paper Trading World Economy India Impact World Report