A documentation release. The README now documents the handoff to nomologR in its own section, matching the one the nomologR README carries for the handoff from contentvalidR, so either front page explains how the two packages connect. Released on GitHub and R-universe; CRAN keeps 0.4.0 until the joint 1.0.
No computed value changes. Across 25 analyses spanning every workflow, every value 0.10.0 returns is identical in 0.10.1, and no function, argument, or handoff field changed.
Tenth public release, and the last planned before the joint 1.0 release
candidate on 2026-10-17. v0.10.0 adds figures for papers, posters, and
teaching: the item flow diagram and the item evidence profile of
content_evidence(), and the full distribution of a panel's ratings, each in
APA gray or in color. It also states the handoff contract nomologR reads, makes
this package's walkthrough the joint one, and separates the Delphi printout's
caveats from its teaching.
contentvalidR is on CRAN: 0.4.0 was published there on 2026-09-28. CRAN asks for updates no more often than every one to two months, so 0.10.0 is released on GitHub and R-universe, and the next CRAN submission is the joint 1.0.
The package continues to declare Imports: stats only.
No computed value changes. Across 25 analyses spanning every workflow,
every value 0.9.0 returns is identical in 0.10.0. Relevance fits gain
details$ratings. Some printed wording changes: the Delphi printout's method
paragraphs move into its key, anova_content() prints within 80 columns, the
printed handoff advises passing the whole handoff to nomologR, and the
citation a handoff carries for stability = "percent_change" reads
"Scheibe et al. (1975/2002)".
No breaking changes. The plot methods gain arguments after their existing
ones: type, apa, and labels for expert-panel fits, and
which = "distribution", apa, and labels for Delphi fits.
agreement_summary() remains deprecated and is removed in 1.0.0.
Each new figure draws what the fits and handoffs already decided, and says whether it follows a published form or is this package's own design.
content_evidence() (new) brings the handoffs from several review
stages together, such as a relevance panel and then an item sort. It prints
one row per item with each stage's decision beside the statistic its rule
read, and draws two figures:
plot(x, type = "flow"), the item flow diagram, modeled on the PRISMA
2020 flow diagram (Page et al., 2021): what each stage reviewed, what it
held back and the number behind each decision, and what went forward;plot(x), the item evidence profile, this package's own design: one
panel per stage with each statistic, its interval, and its criterion, so
agreement and disagreement between methods can be seen item by item.plot(fit, type = "distribution") for a relevance panel and
plot(fit, which = "distribution") for a Delphi study, one bar per round. It
shows what the I-CVI cannot: two items at 1.00, one rated relevant with 4s
and the other with 3s. Relevance fits now keep their ratings in
details$ratings to draw it.apa switches these figures between gray (TRUE, the default, as an
APA figure is printed) and a colorblind-safe scheme for slides and posters
(FALSE): teal for evidence that met its criterion, brown for evidence under
review. Symbols carry the decision either way, so no reading depends on
color.walkthrough_relevance.csv,
gives the twelve items a second source of content evidence.
vignette("reporting-examples") uses it to show both figures, with a sample
caption. The ratings are constructed, and a separate script,
data-raw/build-walkthrough-panel.R, says what each is built to show. The
three walkthrough files that nomologR ships are unchanged.vignette("handoff-to-empirical-validation") now
pass the whole handoff to nomologR, nomo_screen(data, items = handoff),
rather than handoff$items, so the keying and the reasons for anything held
back travel with the items. The vignette no longer calls the handoff reader
in nomologR future work.install.packages("contentvalidR") first, carries a CRAN badge, and
explains why CRAN's version can trail the newest release on R-universe:
CRAN asks packages to update no more than every one to two months.options(contentvalidR.show_key = FALSE). The
critique printed for an alternative stability method still always prints.
The example printout goes from 100 lines to 92, or 58 with the key hidden.anova_content() prints within 80 columns. The strongest competitor, which
htd() and the construct-rating workflow already show, stays in the
strongest_competitor column.stability = "percent_change". The 1975 printing's pages could not be
checked.Asked for by nomologR before handoff schema 1 becomes a 1.0 promise. The schema and every handoff are unchanged.
?content_handoff has a new section, "Reading the decisions": take each
item's decision from carried and status, never re-derive it by
comparing value with criterion or by branching on the producer version.
The example is the nine-expert correction in 0.8.0, where an item's I-CVI
stayed at .778 while its decision changed. The section also restates that
keying of NA means unknown, never forward-worded.?content_handoff says an item sits in at most one scale, the one its
item_evidence$scale names. A test holds the sort, rating, and congruence
workflows to it, and checks that each refuses an item with two targets.vignette("one-item-set-both-stages") is the joint walkthrough for
contentvalidR and nomologR, as the maintainer decided for the joint 1.0.
It now says so, passes the handoff itself to nomo_screen() and
nomo_run() rather than h$items, and links to the section of the
nomologR guided workflow that continues from content review. Two stale
sentences are corrected. The handoff reader in nomologR, which the vignette
called future work, has shipped. And the nomologR guided workflow starts
from the same handoff but simulates its own responses, so its screening
numbers differ from these.Ninth public release. v0.9.0 is about how the package presents itself before 1.0. Every reference list is in APA 7 form; printed output is shorter and says what it means; the component functions and report tables print in APA style; the figures show the criterion and the uncertainty; and the README is a front page rather than a manual.
The package continues to declare Imports: stats only.
No computed value changes. Across 25 analyses spanning every workflow, every value 0.8.0 returns is identical in 0.9.0. Some wording changes: the scale-level summaries of the item-sort and construct-rating workflows name the Colquitt band of the weaker index, counts read "1 item" or "3 items", and the note when judge severity cannot be estimated no longer blames near-total agreement when one dissenting judge causes it.
Breaking changes.
content_report() writes an APA table by default. For the numeric table
that used to be the default, pass format = "data.frame"; its p values
now keep three decimals whatever digits is.compute_psa(), aikens_v(), cvr(),
and eleven others) gain a class, so they print as formatted tables. Every
value is unchanged and as.data.frame() returns the plain data frame; only
code that tests class(x) == "data.frame" exactly would notice.Deprecated. agreement_summary() warns and will be removed in 1.0.0; use
panel_agreement(), which takes raters in rows like every other ratings
function.
New: content_report(format = "apa"), and csv_binom_test() returns its
inputs (n_target, N, p0, alpha) beside its result.
Published on GitHub and R-universe. 0.4.0 is still in CRAN's review queue, and CRAN policy asks that no further version be submitted while one is pending.
vignette("design-and-reporting") gains the planning examples the README
held: expert_power(), with why a fourth or fifth expert lowers the chance
of clearing Lynn's criterion, and gtheory_content(), with how many judges a
design would need. Its method list now names all six workflows, and it has a
reference list.No computed value changes; this is about what the figures draw.
main is given, and each restores the
graphics settings it changes. Item rows run top to bottom in the order of
the results table, and scale means are triangles, clearly distinct from
retained items.No computed value changes; this is about what is printed.
contentvalid_glossary() keeps the full definitions and now lists every
workflow's decision words. The item-sort example in the README prints 58
lines instead of 71.compute_psa(),
compute_csv(), htc(), htd(), anova_content(), aikens_v(), cvr(),
ioc(), interpret_colquitt(), colquitt_benchmarks(), csv_binom_test(),
signal_detection(), reproducibility_phi(), and similarity_from_sort()
printed raw data frames and lists, such as a p of 1.941920e-16 beside
seven-digit decimals. They now print as cvi() already did: a titled table
in APA style with a short note. anova_content() reports its tests as
F(1.71, 32.57) = 108.55. The values are unrounded; each result only gains a
class, as.data.frame() returns the plain data frame, and workflow results
are unaffected. csv_binom_test() also returns its inputs.content_report() writes APA tables by default. The new
default format = "apa" gives readable headings, counts such as 18/20,
p to three decimals, and [LL, UL] intervals, and prints without row
names; format = "markdown" writes the same table and prints as the
table. For the numeric table that used to be the default, pass
format = "data.frame"; its p values now keep three decimals whatever
digits is. The construct-rating table shows the contrast p beside the
omnibus p, since a Review item can have an omnibus p below .001.panel_agreement().
agreement_summary() is the only function that takes items in rows; every
other ratings function, panel_agreement() included, takes raters in rows,
so one matrix passed to both was read two different ways.
panel_agreement() also reports an interval and needs no other package.
agreement_summary() still works in 0.9.0 and warns when called. Removing
it at 1.0.0 leaves nothing deprecated past 1.0, as the stability policy in
?contentvalidR sets out.REFERENCES.bib held 13
of the works the package cites, and one it no longer cited; it now holds all
53, generated from the reference list in the README, with author names and
details checked against the Crossref record for each DOI. A test keeps the
two in step. The README calls it a reference list rather than a
bibliography.citation("contentvalidR") prints the package reference in APA form, with
its version and "[Computer software]".No computed value changes.
Eighth public release. v0.8.0 is about reading the output and the literature correctly. Every printed result now reads as a report in APA style, every source is cited where its method is used, and the earlier published rules can be shown beside each decision for teaching, each with its shortcomings stated. One criterion now matches its source, and the last deprecation is completed, so nothing deprecated is carried into 1.0.
The package continues to declare Imports: stats only.
Results that change. Values computed by 0.7.0 and 0.8.0 were compared on 25 analyses spanning every workflow. The only differences are these:
cvi_criterion now shows each panel size's exact cutoff, such as .875 for
eight experts, where it showed .78 throughout. Beyond ten experts, where her
table stops, the package holds her 7 of 9. That lowers the requirement by
one expert at every multiple of nine (14 of 18, 21 of 27, 28 of 36, and so
on) and at some panels above 58. Verdicts at other panel sizes are
unchanged.judge_validity()'s influence check. It removes one judge at a time, so
a ten-judge panel was checked at nine, where 0.7.0 applied the wrong
criterion. On a ten-judge panel this could flag every judge as influential.
It now applies Lynn's.Too predictable, not Erratic. The status is Review either way.anova_content() no longer returns posthoc_pass;
read contrast_pass, which holds the same value.One display default changes. Print methods now default to digits = 2, as
APA reports statistics; print(x, digits = 3) restores the previous
precision. Stored values are unaffected.
New arguments: sort_validity(legacy = , n_constructs = ),
expert_validity(legacy = ), and legacy in their print() methods.
Published on GitHub and R-universe. 0.4.0 is still in CRAN's review queue, and CRAN policy asks that no further version be submitted while one is pending.
anova_content() no longer returns posthoc_pass. It
was a duplicate of contrast_pass, documented as deprecated in 0.7.0. The
written policy requires one minor release of notice and allows removal in a
minor release only before 1.0, so this is the last release that can remove
it without waiting for 2.0. Read contrast_pass, which holds the same
value. It was the last column, so no other column moves.?contentvalidR records both of anova_content()'s completed cycles.For teaching, the way a methods text reports eta-squared beside omega-squared,
sort_validity() and expert_validity() now compute the earlier published
rules beside their own. legacy = TRUE prints them under "Earlier methods, for
comparison (not used for the decision)", and print(fit, legacy = TRUE) shows
them for any fit. They are stored in details$earlier_methods and never change
a decision. Each follows its source as published, and the tests check each
against the numbers its source prints.
1/k + .05,
which is their .30 at four. It prints in its own column marked * and is
described as "a contentvalidR extension, not a published rule". The new
n_constructs argument states how many constructs judges were offered;
by default the package counts the ones they used, and says so.z / sqrt(N), and Lawshe's
content validity index, the mean CVR of the items his table retains. A panel
size Lawshe did not tabulate gets no minimum rather than an interpolated one.
When a CVR prints equal to a cutoff it misses, such as 8 of 9 (.778) against
Lawshe's .78, the printout says so.?expert_validity and the
expert-panel vignette all set out:
vignette("reading-output"), which explains the APA number rules,
cites the Publication Manual. Yao et al. (2008) joins the README references.No statistic changes. Only printed output changes, and results still holds
every value at full precision.
95% CI [.70, .97] in a column headed with their level
(Section 6.43). digits now defaults to 2 in every print method.summary() methods show flagged items in the same columns
as print(), where the expert-panel summary used to print its whole results
table. An explanation shared by several items, such as a
Delphi note or a judge's flag, is printed once with every item it applies
to.expert_power() and
sort_power() put each assumed probability in its own column, so ten panel
sizes take ten rows instead of thirty.results.n_target and n print together as judges (18/20),
severity(pts) as severity, differentiation as scale use, and the
recommendation column as decision.delphi_validity() prints the degrees of freedom beside a chi-square, as APA
reports it. judge_validity() prints infit beside outfit, since either can
flag a judge, and states the rules that flag one.judge_validity() no longer calls a too-predictable judge "Erratic". A
fit mean square above the range means ratings noisier than the model expects;
one below it means ratings more predictable than it expects, the opposite
problem. Both were labeled Erratic. The second is now Too predictable,
with its own interpretation. The status is Review either way, and which
judges are flagged does not change. Code that matched recommendation == "Erratic" to find every misfitting judge should also match "Too predictable", or use status.rule text read
I-CVI >= 0.8333333. It now reads at least 5 of 6 experts rate the item relevant (I-CVI >= .83; Lynn, 1986), since the decision compares counts, and
the handoff now cites Lynn (1986). Rule text is prose for a person and
outside the frozen schema, which is unchanged.expert_validity() verdicts, expert_power() planning
probabilities, and judge_validity()'s check of whether one judge moves an
item.cvi_criterion now shows Lynn's cutoff for each panel size, such as .833
for six experts and .778 for nine, instead of .78 throughout. Verdicts at six,
seven, eight and ten experts are unchanged, but the printed criterion now
matches the verdict beside it: an item is supported exactly when its I-CVI
reaches the number shown. A test checks that for every panel of 3 to 15
experts and every count of agreeing experts.qfactor_content().cvi() and expert_validity() cite Lynn (1986), the source of the I-CVI
and of the panel-size criterion expert_validity() applies, and Polit and
Beck (2006), who named S-CVI/UA and S-CVI/Ave. ?cvi now explains where
each index comes from and why both scale-level versions should be reported.cvr() cites Wilson, Pan and Schumsky (2012) and explains why the exact
values replace Lawshe's original table: Wilson et al. found the table
unexplained, non-monotonic at eight experts, and closer to a two-tailed
than the one-tailed test it was labeled as.agreement_summary() credited nobody. It now cites Fleiss (1971) for the
statistic and the irr package's authors for the code it calls.sort_power() and simulate_csv_power() cite Howard and Melloy (2016), whose
test they plan for. Two incomplete entries are completed.print() on a handoff,
vignette("handoff-to-empirical-validation"), and the README all told
readers to pass the handoff object to nomologR::nomo_screen(), which cannot
read it until nomologR#46.
All three now pass handoff$items.Seventh public release. v0.7.0 is about being safe to depend on. It says in writing what code can rely on from one version to the next, freezes the handoff object that another package reads, and settles the one deprecation that a 1.0 release would otherwise lock in. It also adds a worked example that carries a single item set from an expert panel into response data, and it corrects the stated basis of the Delphi kappa interval.
The package continues to declare Imports: stats only.
One breaking change. anova_content() no longer accepts posthoc, and
passing it is an error. The argument has had no effect on any result in any
release, and it has warned in every one since 0.1.0. See below.
No default changes, and no computed value changes. Across all six workflows,
every Delphi stability method, and the bootstrap intervals, every value 0.6.0
returns is identical in 0.7.0. Handoff objects gain four columns (note,
keying, response_min, response_max), all appended within schema
version 1, so a reader written for 0.6.0 still reads them.
New arguments: content_handoff(reverse_keyed = , response_scale = ). New
vignette: vignette("one-item-set-both-stages"). New example data: three
walkthrough_*.csv files in inst/extdata.
Published on GitHub and R-universe. 0.4.0 remains in CRAN's review queue, and CRAN policy asks that no further version be submitted while one is pending.
anova_content()'s posthoc argument is removed
(#60). Passing it is now
an error rather than a warning.
anova_content() has always run. The argument has had no effect on any
result in any version.alpha, so anyone
who passes arguments by position past target_map should check their call.posthoc_pass is now documented as deprecated. It is a duplicate of
contrast_pass and has been returned silently since the first release,
without ever being marked. It is still returned and still correct, since
the policy requires a release in which reading it keeps working; read
contrast_pass instead. It is the last column, so removing it later cannot
move another.NEWS.md records a deprecation and its removal with the replacement; the
posthoc deprecation was never recorded at all until now. ?contentvalidR
now cites posthoc as the completed cycle and posthoc_pass as the one in
progress, and the test suite checks that both are in the state the policy
claims.csv_binom_test() moves from Tier 3 to Tier 2. Tier 3 describes helpers
"kept for continuity with older analyses", which are "not recommended
workflows" and "may be deprecated and removed". That is the wrong description
of the Howard and Melloy (2016) exact test that sort_validity() runs on
every item. The tiers become promises at 1.0, so a load-bearing function had
to stop being filed under the one that promises least. No code changes.vignette("one-item-set-both-stages") follows twelve items from an
expert panel through the handoff and into response data
(#55), so the pair of
stages is demonstrated rather than asserted. Every step runs: the second half
uses stats only, and the nomologR call is shown but not run, since
neither package depends on the other.EF4 is sorted to effort
regulation by eighteen of twenty judges and carries almost no common
variance; TF4 is sorted to one facet and loads on both. A panel cannot
see either, which is the reason for running the second stage.EF3 is the reverse, and the sharper case: an empirical screen flags it
and it is worth keeping. Nearly everyone endorses "I finish the assignments
that count toward my grade" — 96% answer in the top two categories — so its
variance is a third of every other item's and it cannot correlate strongly
with anything. Its correlation looks as bad as EF4's, and only the
distribution tells them apart. It is also the only item covering the
completion of required work, so dropping it would narrow the domain the
panel defined. Added at the nomologR maintainers' suggestion, so that both
packages' articles make the same point about it.EF2 and TF2 are written the other way round, and the walkthrough
passes that to content_handoff(reverse_keyed = ) along with the
respondents' one-to-five scale. It then shows why the handoff has to carry
both: before recoding, TF2 correlates at -0.51 with its own facet, which
looks like the strongest evidence against an item that item analysis
produces and is in fact a coding error. Recoded with the keying and the
scale limits the handoff carries, it is +0.51. Added together with the
keying field, at the nomologR maintainers' request, so that no reader ever
receives reverse-worded data without the means to know it.inst/extdata: walkthrough_items.csv (the items, their
facet, their stems, what each one was built to do, and whether it is
reverse-worded),
walkthrough_sort.csv (twenty judges), and walkthrough_responses.csv (400
respondents, twelve items, a two-level cohort variable, and the rejected
items still present). They are simulated. No participant was involved and
no real instrument is reproduced.data-raw/build-walkthrough-data.R writes all three and states the
generating model in full: the two correlated facets, every loading, the
response thresholds, and the single cohort shift. It uses base R only and is
deterministic. nomologR mirrors the response file from the same script, so
the two packages cannot drift.EF4 is the weak one and
TF4 the ambiguous one, that EF3 is flagged for restricted variance rather
than for lacking common variance, and that TF6 is the only item that shifts
by cohort. If the data are regenerated and a claim stops holding, the suite
fails rather than the vignette quietly becoming wrong.?content_handoff gains two sections. What version 1 freezes names every
field and column a reader can rely on, states that each keeps its name,
position, and type, and lists what is deliberately not frozen: which rows
appear, the labels in the statistic column, the prose in note, rule,
recommendation, and citation, the contents of settings and design,
and the printed output. If the schema ever changes gives the version-2 path
and the two-line version check a reader should gate on.NA in all four interval columns and
a workflow with no panel coefficient returns a zero-row panel_statistics
with the full column set. Handoffs from different workflows therefore bind
without reconciling them first.nomologR confirmed the frozen list covers everything its reader consumes,
and named two gaps, each blocking a specific computation rather than
expressing a preference. Both are added.solomonR was asked too
(solomonR#32): its
fit_solomon_sem_latent() takes item names as character vectors, which is
exactly what a handoff carries.content_handoff() gains reverse_keyed and response_scale, recorded
per item in three new item_evidence columns: keying (1 forward, -1
reverse-worded), response_min, and response_max.
lo and hi
are the scale the experts rated relevance on, usually 1 to 4, not the scale
respondents answer. Copying them across would hand a reader the wrong
limits, and it would then reject every legitimate top-category answer. So
both default to NA, meaning unknown, and a test guards against the fit's
scale ever leaking into these columns.reverse_keyed = character(0) records that someone checked and no item is
reversed, which is different from not having said.response_scale warns, since the
item cannot be recoded without the scale's limits. A reader that recodes
would otherwise refuse one step later, where the omission is harder to
trace. Suggested by the nomologR maintainers.note column described below. All are
appended after the original columns, as the additive rule requires.?delphi_validity gains Reading the kappa interval, which
says what the interval is and gives the two things a reader needs in order to
report it honestly: Klar et al.'s measured coverage for a nominal 95%
interval was about 83% with 20 units, 89% with 25, 91% with 30, and reached
94% only from 40 up, so a panel-sized interval is narrower than its label;
and their simulations used an unweighted kappa on two categories, so the
ordinal weighted case has not been simulated. The printed output carries the
short form of both.panel_agreement() keeps its extension warning for Gwet's AC1. Zapf et al.
(2016) evaluated the bootstrap for Fleiss' kappa and Krippendorff's alpha
only, and do not discuss AC1 at all, so applying their interval to it is an
extension. That caveat now travels with the number: an AC1 interval in
content_handoff() carries it in panel_statistics$note, where before it
reached only the console. (content_report() tabulates item-level evidence
and never shows a panel coefficient, so there is nothing to carry there.)B = 0, delphi_validity() still described
intervals it had not computed and still printed empty low and high
columns; it now omits both. The printed-claims test that should have caught
this used a statistic that has no interval at all, so it passed vacuously.?contentvalidR now states what code can rely on across versions
(the 1.0 criteria on
nomologR#53):
item_statistics and panel_statistics gain a note column, carrying the
producing function's own sentence for why a value or interval is absent or
degenerate, such as "Kappa is undefined: every rating fell in the same
category in both rounds."
NA: "" means there is nothing to say.NA cases applies stays
readable from value and proportion unchanged, and that remains the
supported way to decide anything.Documented the two reasons a Delphi stability statistic can be NA
(#48), which a reader
can tell apart from the object alone:
proportion unchanged is also NA: the item has no pair of consecutive
rounds, because it was rated in one round only;proportion unchanged has a value: a pair exists, but the statistic is
undefined for that data.The second case is easy to misreport. An undefined kappa beside
proportion unchanged of 1 is perfect stability that kappa cannot
express: kappa is chance-corrected, so when every paired rating in both
rounds falls in one category, the disagreement expected by chance is zero
and kappa is 0/0. Saying only "not estimable" would describe a defect that
does not exist.
The row is always present, so an NA is a statement about the data rather
than a missing record. Raised by the nomologR maintainers from the 0.6.0
fixtures.
Sixth public release. v0.6.0 finishes the Delphi workflow: a Delphi study can
now be handed off to empirical validation, carrying for each item the round it
settled in, and plot() shows the two trends a Delphi is about. It also adds
a test layer that checks the package's printed claims against what the code
actually computes.
The package continues to declare Imports: stats only.
No default changes. Existing calls return the same values as in 0.5.0.
New method: plot() for delphi_validity(). content_handoff() accepts a
Delphi fit.
Published on GitHub and R-universe. 0.4.0 remains in CRAN's review queue, and CRAN policy asks that no further version be submitted while one is pending.
plot() for delphi_validity()
(#44), so the Delphi
workflow matches the others in having one.
which = "consensus" (default) draws each item's share of experts
agreeing, round by round. A line that stops early belongs to an item that
settled and was set aside, so the picture shows when each item left the
study.which = "stability" draws the stability statistic per pair of rounds,
with the share of experts who kept their rating as open circles.compare_rounds() claims comparability only when the settings match;.print_key() now fails on an unknown term instead of dropping it. A typo
in a print method's key used to remove a column's explanation in silence.content_handoff() now accepts a delphi_validity() fit
(#42). It used to
refuse one, and its message did not even mention Delphi.
round now holds that round's index, so it varies between
items. This is the first workflow where it carries real information.expert_validity() fit, with intervals: I-CVI against the
consensus threshold, Aiken's V, and modified kappa. Modified kappa
carries no criterion, because a Delphi decides on the consensus
threshold rather than on the 0.74 rule.proportion unchanged plus the statistic that ran, named for its
method, such as weighted kappa (quadratic) or Goodman-Kruskal lambda. The chi-square methods add stability p_value. The names are
qualified so a downstream report cannot misread them
(nomologR#46).round cannot be set by hand for a Delphi fit, since the fit supplies it.Fifth public release. v0.5.0 adds a workflow for Delphi studies, where the same expert panel rates items over successive rounds. It reports consensus and stability as separate questions, and every published stability method is available with its limits printed. The handoff to empirical validation now carries each statistic's interval, so a downstream analysis can see how much panel evidence stands behind a number.
The package continues to declare Imports: stats only.
No default changes. Existing calls return the same values as in 0.4.0.
content_handoff() objects gain four interval columns in item_statistics
and a panel_statistics table, within schema version 1.
New function: delphi_validity(). New article: vignette("delphi-rounds").
This release is published on GitHub and R-universe. 0.4.0 is still in CRAN's review queue, and CRAN policy asks that no further version be submitted while one is pending, so 0.5.0 will be submitted to CRAN later as an update.
delphi_validity() for an expert panel rated over successive
Delphi rounds (#20).
It reports two questions separately. Consensus: whether enough experts
agree now. Stability: whether experts are still changing their ratings.
Consensus is the share of experts rating an item at or above
agree_cut, which on a relevance scale is the I-CVI. It is judged
against a consensus_threshold fixed before the study. There is
deliberately no default: without a threshold, results are descriptive
(Diamond et al., 2014).
Stability always reports the share of experts who kept their rating. Beside it goes one statistic:
Each alternative's limits are printed. The two chi-square tests read in opposite directions, and the printout says which is which.
No verbal kappa labels. Landis & Koch (1977) call their divisions arbitrary, and kappa falls as a panel converges. When a round is unanimous, kappa is 0 however many experts kept their rating; the output flags this and points to the share unchanged.
A bootstrap interval for kappa, resampling experts, marked as this package's extension.
Round-by-round relevance evidence. Each round is also fitted with
expert_validity(), so compare_rounds() can read the whole process.
vignette("delphi-rounds"), walks through a three-round
panel for a graduate reader. Its worked example covers:
contentvalid_glossary("delphi") defines the new columns.content_handoff() now carries each statistic's interval
(#33), so a reader can
tell a unanimous four-judge panel from a unanimous twenty-judge one. A
unanimous I-CVI from four experts has a Wilson 95% lower limit near 0.51.
item_statistics gains lower, upper, interval_method, and
interval_level. They hold the Penfield-Giacobbi score interval for
Aiken's V, and whichever proportion_ci method was chosen for I-CVI and
Psa.panel_statistics table carries the panel agreement coefficient
and its bootstrap interval, which are not per item.NA in all four columns means the statistic has no interval, never
missing data.Fourth public release. v0.4.0 connects content validation to what comes after it: a documented handoff carries the items that survived content review, and the evidence behind each decision, into empirical scale development. It also adds a stricter, selectable criterion for parallel analysis, and fixes the two packaging problems that returned the 0.3.x CRAN submissions.
The package continues to declare Imports: stats only.
No default changes. Existing calls return the same values as in 0.3.1;
qfactor_content() results gain two fields recording the parallel analysis
criterion that ran.
New function: content_handoff().
content_handoff(), which packages a finished workflow's item
decisions for the next stage of scale development
(#27). It carries the
item names that survived content review, the construct each belongs to where
the design defines one, a per-item evidence table with the decision rule that
was applied, the statistics behind each decision, and the provenance of the
analysis.nomologR package,
which consumes it in nomo_screen() and nomo_run(). Every field is a base
type, so neither package depends on the other.keep stay in the evidence table with
carried = FALSE rather than disappearing, and the printed output says
plainly that surviving content review does not establish how an item will
behave empirically.qfactor_content() gains parallel_criterion, choosing what parallel
analysis compares observed eigenvalues against
(#26). "mean" stays the
default, the rule Horn (1965) described and Zwick and Velicer (1986)
evaluated. "percentile" compares against an upper percentile of the
simulated eigenvalue distribution, following Glorfeld (1995), who found that
Horn's procedure still tends to retain one or two factors too many.percentile, defaulting to 95.
Both criteria read the same simulation, so they are directly comparable from
one seeded run, and the result records parallel_criterion and percentile
alongside the comparison values in parallel_eigen..Rbuildignore now excludes .git, .gitignore, and .gitattributes.
Building a release tarball from a git worktree checkout writes .git as a
file rather than a directory, which R CMD build does not drop, and CRAN's
incoming pretest reported it as a hidden file included in error
(#14).Patch release for the CRAN submission. Package code, documentation of methods, and results are unchanged from 0.3.0.
Third public release, and the first prepared for CRAN. Every addition rests on published, verifiable methodology, and one design rule runs through the release: where more than one published method exists, researchers choose through an argument whose default is the best-supported option. Methods with published evidence against them stay available for reproducing earlier work, but they are never the default, and selecting one prints the critique.
The package continues to declare Imports: stats only.
Two defaults change results for existing code. qfactor_content() now chooses
the number of factors by parallel analysis, and expert_validity() relevance
mode now bootstraps an interval for panel agreement, so set seed when printed
output must be reproducible. Other existing results are unchanged; the new
interval and agreement columns are additions.
New function: panel_agreement().
panel_agreement(), reporting one coefficient for how consistently an
expert panel rated the whole item set, with a bootstrap interval. It
complements the item-level I-CVI and modified kappa rather than replacing
them.irr::kripp.alpha().expert_validity() relevance mode reports panel agreement in its scale
summary, controlled by agreement, agreement_level, agreement_B, and
seed.qfactor_content() now chooses the number of factors
with Horn's (1965) parallel analysis by default, instead of Kaiser's
eigenvalue-greater-than-1 rule. Calls that relied on the old default can
return a different number of factors. Use retention = "kaiser" to
reproduce earlier results; it prints the finding of Zwick and Velicer (1986)
that the rule severely overestimates the number of components.n_iter and seed arguments control it, and the result
records retention, k_suggested, and parallel_eigen.k_factors still takes precedence, and is recorded as
retention = "fixed".qfactor_content() now cites Schriesheim et al. (1993, 1999), and
signal_detection() cites Anderson and Gerbing (1991).DESCRIPTION now gives DOIs for the methods it cites, and citation metadata
is stamped for 0.3.0.agreement_summary() and the plot() methods for sort, rating, expert, and
sort-power objects now have runnable examples, so every help page shows how
to use its function. agreement_summary() also points to
panel_agreement().cvi() now reports an interval for each I-CVI (I_CVI_low, I_CVI_high),
and compute_psa() does the same for each Psa (psa_low, psa_high). Both
indices previously appeared as bare proportions, usually from small panels.ci in those two functions, and
proportion_ci in expert_validity() and sort_validity(). The options are
the Wilson score interval (the default; Wilson, 1927), which Newcombe (1998)
recommends over the Wald interval; the Agresti-Coull adjusted Wald interval
(Agresti & Coull, 1998); and the Clopper-Pearson exact interval (Clopper &
Pearson, 1934), which is conservative. Printed output names the method, the
interval level, and the method's limits.ci_low and ci_high columns in expert_validity().Second public release. v0.1.0 established three item-level workflows; v0.2.0 adds the two questions those workflows could not answer — whether conclusions depend on the particular judges used, and whether the item set covers its intended domain — together with multi-round comparison, expert-panel planning, reporting helpers, and a substantial rework of how results explain themselves.
The package continues to declare Imports: stats only. Every method here is
implemented in base R, so the package installs without a compiler toolchain.
New flagship workflows: judge_validity() and domain_validity(). New
supporting functions: gtheory_content(), content_structure(),
similarity_from_sort(), compare_rounds(), expert_power(),
content_report(), contentvalid_glossary(), and an as.data.frame() method
for workflow objects.
compare_rounds(), comparing two or more fitted workflow objects from
successive pretest rounds. Reports each unit's status in every round, whether
it strengthened, weakened, or held, and which units entered or left the item
set.compare_rounds() also compares the settings of each round and marks the
comparison as not comparable when they differ. This is the audit trail the
workstream called for: a status change under a changed criterion may reflect
only the changed rule, and the output says so rather than letting a
bookkeeping change read as progress.expert_power(), reporting the exact probability that an item clears
its expert-panel criterion at a given panel size and assumed endorsement
probability, for the panel-size I-CVI guideline or the Lawshe CVR critical
count. It reports the consequences of the panel sizes asked about rather than
recommending one.expert_power() makes the I-CVI criterion's step visible instead of smoothing
it: because the guideline requires unanimity up to five experts and 0.78 from
six, a fourth or fifth expert lowers the probability of clearing while a
sixth raises it sharply. The plot is drawn as a step function for the same
reason.expert_power() accepts a response_rate below 1, averaging over the
realized panel size rather than assuming the invited panel arrives intact,
which also captures the criterion that a smaller realized panel triggers.as.data.frame() method for workflow objects, returning results or
the scale summary as a plain data frame with a workflow column so tables
from several analyses stack without losing their identity.content_report(), building a manuscript-ready table as a data frame or
as Markdown for Quarto and R Markdown. Markdown is generated directly, so no
reporting package is required and none is added as a dependency. Analysis
settings travel with Markdown output as an attribute, and columns that are
entirely missing are dropped.Review never means an item must be dropped.contentvalid_glossary(), a single source of those definitions. The
inline keys and the glossary read from the same table, so a term cannot be
defined differently in two places.options(contentvalidR.show_key = FALSE) once the terminology is familiar.
Substantive cautions are never suppressed by that option.Weak
in the same row where an HTD of 0.44 is labeled Very Strong. Output now
explains why, rather than leaving the contrast looking like an error.domain_validity(), a flagship workflow assessing whether an item set
spans its intended content domain. Its results table has one row per
blueprint cell, supporting construct-only or crossed construct-by-facet
tables of specifications.targets so
expected shares come from the blueprint instead of an assumption of equal
cells.domain. When it is omitted, the
output states plainly that empty cells could not be detected, rather than
implying full coverage.content_structure(), implementing the multidimensional scaling and
hierarchical cluster analysis of expert item-similarity data described by
Sireci and Geisinger (1992, 1995). Correspondence between recovered clusters
and blueprint cells is quantified with the chance-corrected adjusted Rand
index and reported alongside the raw cross-tabulation.similarity_from_sort(), deriving item similarities from an item-sort
task as the proportion of judges co-assigning each pair. The documentation
states why this is weaker evidence than pairwise similarity ratings collected
for the purpose.plot.contentvalid_structure(), drawing the expert content map with
items labeled by blueprint cell.judge_validity(), a flagship workflow reporting how far
content-validity conclusions depend on the particular judges who served on
the panel. Unlike the item-oriented workflows, its results table has one row
per judge.gtheory_content(), implementing the generalizability-theory treatment
of content-validity ratings in Crocker, Llabre, and Miller (1988). Reports
item, judge, and residual variance components, generalizability (relative) and
dependability (absolute) coefficients, and a decision study giving the panel
size implied by a target coefficient.Review is not a judge to delete:
disagreement may be substantive expertise, and the flag marks where a
conclusion rests on one person's ratings.GPL-3.0-only; R metadata
GPL-3), retaining the original MIT notice in inst/NOTICE. The previously
published v0.1.0 release keeps its original MIT terms; this release does not
relicense it retroactively.DESCRIPTION.bak file.contentvalidR, providing reproducible quantitative
tools for substantive and content-oriented scale pretesting.sort_validity() for item sorting, rating_validity() for construct ratings,
and expert_validity() for relevance, essentiality, and congruence expert
panels.--as-cran NOTE-as-failure gate.actions/checkout@v7 and current r-lib/actions@v2 conventions.data-raw/release-check.R developer checklist covering documentation, tests, README, standard and --as-cran checks, optional URL/spelling audits, and pkgdown construction.cran-comments.md for the first submission; it is excluded from the built package and must be populated with actual final check results before submission.data-raw/ provenance script that regenerates them.inst/CITATION for the package and a centralized inst/REFERENCES.bib bibliography covering the release-defining methods and verified DOIs.sort_validity(), rating_validity(), and expert_validity() around a common workflow-object contract: results, scale_summary, settings, design, and details, with a shared contentvalid_workflow superclass.status field (Supported, Review, Insufficient data, or Descriptive only) while preserving method-specific recommendation wording such as Retain, Strong support, and Target favored.summary() objects around common counts, reviewed-item tables, scale summaries, settings, and design metadata; retained n_retain, expert scale/flagged, and rating contrasts compatibility aliases.digits and plot show_legend controls across the primary workflow methods and exact sort-power planning object.orbiting_r specifications.NA (rather than an out-of-range sentinel) and hardened regression tests against brittle error-message wording.expert_validity() is available in clean installs and vignette builds.show_legend = FALSE to workflow plot methods for compact/custom reporting.expert_validity() with relevance, essentiality, and congruence modes plus informative print(), summary(), and plot() methods.cvr() to support item-specific panel sizes and judge-by-item 0/1 input, with exact one-sided binomial p-values and critical counts following Ayre and Scally's revisiting of Lawshe's method.ioc() input validation, duplicate detection, and missing-data reporting.actions/checkout@v4 to the current Node-24-compatible actions/checkout@v7 line.htc() and htd() for Hinkin-Tracey definitional correspondence and
distinctiveness, with explicit rating-anchor validation.anova_content() around the fully crossed within-judge design used by
the Hinkin-Tracey rating procedure. The function now uses a one-way repeated-
measures ANOVA with Greenhouse-Geisser corrected omnibus inference plus planned
target-versus-orbiting contrasts and retains a between-judge path only for
genuinely independent designs. Raw omnibus p values remain available for
transparency.rating_validity() as the recommended user-facing construct-rating
workflow, including strongest-competitor diagnostics, Retain/Review/
Insufficient-data screening, and narrative print/summary output.plot.contentvalid_rating() for dependency-free HTC/HTD item plots.sort_validity() now reports target-scale definitional correspondence and
distinctiveness alongside Howard-Melloy item-level Retain/Review decisions.compute_csv() now reports the strongest competing construct(s), making item
confusion easier to diagnose.sort_power() for exact binomial design/power planning.plot.contentvalid_sort() for dependency-free item-level Psa/Csv plots.compute_csv() when all non-target assignments fall in a single
construct; unanimous assignment to the same wrong construct now correctly
yields Csv = -1.compute_psa() and compute_csv() now use non-missing itemwise denominators,
report n_total, n, and n_missing, and validate duplicate item-rater rows
and inconsistent target mappings.cvi() to use the probability of
exactly A agreements, and added S-CVI/UA plus item-specific effective judge
counts.signal_detection() and changed
phi calculations to preserve the direction of association.csv_binom_test() as exact Howard-Melloy target-count inference and
added the critical target-assignment count to its output.sort_validity() as the first recommended user-facing workflow, with
classed results, informative print()/summary() methods, and restrained
Retain/Review/Insufficient-data recommendations.