Set the standard
You draft the borderline-candidate description from our template; I review it; your governing body agrees the procedure in writing before ratings begin.
Standard setting for certification programmes
Where did your pass mark come from?
A documented standard-setting study for accredited and accreditation-seeking certification programmes. Designed, facilitated and signed by a psychometrician. Your certification examination has a pass mark. An assessor, an appeals panel or a candidate's lawyer will ask how it was set. "Seventy per cent, as before" is a convention, not an answer. A defensible cut score is the output of a documented procedure: a panel of subject experts, a defined standard, judgements reconciled by a named method, and a threshold reported with its own margin of error.
What you receive
One examination, one panel, one signed report.
The report states the following. It is organised so that each element an assessor expects to find in the documentation of a cut score has its own named section.
The procedure and why it was chosen, fixed in advance and agreed by your governing body before any rating
The panel, its qualifications and any limits its composition places on the result
The performance standard the panel agreed: who the borderline candidate is
Every rating round, and how far the judges agreed
The recommended cut score with its standard error
The standard error of measurement at the cut, from your own item data
A Hofstee reasonableness check
An impact table: the pass rate at the cut, at one standard error either side, and one whole mark either side
A statement of limits, and a signed verdict page
How it works
Three weeks, fully remote.
Method: modified Angoff, two rounds, with item data shown between rounds. Hofstee is a check, never the cut.
Round 1, alone
Each of your 8 to 12 experts rates every item in a protected workbook, by item identifier only. Exam content stays on your secure platform.
One live 90-minute online session
The panel sees where it disagrees and how candidates actually performed on each item, and discusses the contested items. Nobody is named.
Round 2, alone
Judges revise any rating they choose.
Analysis and report
Cut, standard error, agreement, Hofstee check, impact. I read the whole report and sign it.
Packages
Fixed price, fixed scope.
One examination form, 20 to 200 scored dichotomous items, a panel of 8 to 12, and item-level results from at least 150 candidates. Outside that, I quote before the clock starts.
Design, analysis and report
2,900 EUR ex VAT
You convene and run the panel to my written protocol. I design the study, supply the workbook and briefing, analyse the ratings, write and sign the report.
Facilitated study
5,400 EUR ex VAT
Everything above, and I run the briefing call and the live 90-minute session.
Additional cut score
1,200 EUR ex VAT each
On the same examination and panel, in the same session.
Invoiced by AdriaMont Consulting DOO once the scope is confirmed, payable before the three weeks start. VAT treatment is stated on the invoice for your country.
Founding study
The first programme to commission a study is offered a reduced fee in exchange for permission to publish an anonymised case from it. The founding rate is 30 percent off either package: 2,030 EUR or 3,780 EUR ex VAT.
The sample
A complete report on a synthetic examination.
Nobody should buy this without seeing what arrives, and no client's examination is ever used to show it. The sample is a full technical report on a fictional credential and a synthetic examination, with synthetic judges and candidates, produced by the same analysis I use for clients.
6 pages. A 40-item examination, 10 judges, 600 candidates. All synthetic.
Limits
Limits, stated because they are true.
The study recommends. Your body decides. I do not adopt your standard, and I do not tune it to a pass rate.
No outcome is guaranteed. The study makes the procedure defensible. It cannot promise that an accreditor will accept the result or that an appeal will fail. The report states the standard error of the cut so the size of the uncertainty is on the page.
A weak panel gives a weak cut. The panel is yours. The report says who sat on it and says plainly when its composition limits the result.
I do not see your exam. Items stay on your platform. I receive item identifiers, ratings and item-level results with pseudonymised candidate IDs.
No pre-employment selection cut scores for hiring individuals.
No track record is claimed. This is a documented method with a worked sample, offered to a first programme on founding terms.
Relationships and confidentiality
Declared before you pay.
Relationships are declared
If I have any relationship with your organisation, a competitor, or a candidate, it is recorded before work starts and stated in the report. A relationship I cannot manage, I decline. I do not review the same programme for an accreditor, regulator or competing body afterwards: that recusal is recorded at intake and does not expire.
Your exam content stays yours
Exam items stay on your secure platform. Judges read them there, never in an emailed file, and rate by item identifier. Item text, stems, options and keys never reach me. Item-level data is deleted 30 days after delivery, or earlier on request; ratings and results are kept six months; a record of the engagement is kept permanently so that the recusal above can be honoured.
Who is behind it
Dr Milos Kankaras, psychometrician.
Measurement and psychometric research, instrument evaluation and test-quality audits. The service is provided by AdriaMont Consulting DOO.
Ask
Describe your examination.
Tell us your name, organisation and programme, the accreditation context, the number of items, how many candidates have item-level results, the size of your panel and the weeks you would prefer. Please do not send exam content.
Or write directly to
milos@adriamont.meMeasurement Quality Audit: evidence that your examination measures what it should, before you set its pass mark.