Measurement Invariance with nomologR
Source:vignettes/measurement-invariance.Rmd
measurement-invariance.RmdWhat invariance is for
Measurement invariance asks whether a construct is measured
comparably across groups or occasions. If it is not, a difference in
observed scores can reflect how an item functions in each group rather
than a difference in the construct itself. nomologR treats
invariance as graded evidence rather than a sequence of
automatic pass/fail decisions.
Every fitted level retains:
- the generated
semTools::measEq.syntax()object and the exact lavaan syntax; - the fitted lavaan object;
- absolute fit and change in fit;
- nested-model comparison evidence where available;
- localized equality-constraint diagnostics;
- researcher-specified partial-invariance provenance.
A teaching example with a known answer
nomo_demo_network contains 800 responses collected in
two administration groups, online and paper.
Its population model builds in two features:
- the latent Agency mean is 0.25 SD higher in the
papergroup (a real construct difference); and - the intercept of item
ag3is 0.50 higher in thepapergroup (item bias:ag3is endorsed more on paper at the same level of Agency).
All loadings are equal across groups.
aggregate(
cbind(ag1, ag2, ag3, ag4) ~ group,
data = nomo_demo_network,
FUN = mean
)
#> group ag1 ag2 ag3 ag4
#> 1 online 3.98885 3.971875 3.974900 4.027750
#> 2 paper 4.23735 4.252500 4.640525 4.265525Every item mean is higher in the paper group, but
ag3 is higher by much more. Observed means alone cannot
separate the real latent difference from the item-level bias; that is
the job of the invariance sequence.
Continuous indicators
agency_model <- "Agency =~ ag1 + ag2 + ag3 + ag4"
inv <- nomo_invariance(
agency_model,
data = nomo_demo_network,
group = "group"
)
inv
#> <nomo_invariance> Measurement invariance across groups
#> Cases: 800 (online n = 400, paper n = 400) | Estimator: ML
#> Grouping variable: group | Indicators: continuous
#> Requested: configural -> metric -> scalar -> strict
#> Completed: configural -> metric -> scalar -> strict
#>
#> Fit by level
#> Level CFI RMSEA SRMR CFI change RMSEA change LRT p
#> configural 1.000 0.000 0.002 -- -- --
#> metric 1.000 0.000 0.028 .000 0.000 .146
#> scalar .954 0.121 0.065 -.046 +0.121 < .001
#> strict .954 0.102 0.066 .000 -0.019 .483
#>
#> Flagged
#> - Score diagnostics (Review): 24 univariate equality-constraint score
#> diagnostics were retained.
#>
#> CFI = comparative fit index; RMSEA = root mean square error of approximation;
#> SRMR = standardized root mean square residual; LRT = likelihood-ratio test of
#> a level against the level before it; ML = maximum likelihood.
#>
#> Fit changes and score diagnostics are evidence, not pass/fail rules, and
#> nomologR never frees a parameter because of them.
#>
#> See summary(x) for each level's chi-square test and
#> nomo_table(x, "local_strain") for all 24 score diagnostics.The conventional continuous sequence is:
configural -> metric -> scalar -> strict
fit <- nomo_table(inv, "fit")
fit[, c("level", "constraints", "cfi", "rmsea", "srmr", "delta_cfi", "delta_rmsea", "lrt_p")]
#> # A tibble: 4 × 8
#> level constraints cfi rmsea srmr delta_cfi delta_rmsea lrt_p
#> <chr> <chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
#> 1 configural none 1 0 0.00200 NA NA NA
#> 2 metric loadings 1 0 0.0275 0 0 1.46e- 1
#> 3 scalar loadings, inte… 0.954 0.121 0.0647 -0.0462 0.121 1.20e-13
#> 4 strict loadings, inte… 0.954 0.102 0.0659 0.000420 -0.0193 4.83e- 1The configural and metric models fit closely, consistent with equal loadings. Adding intercept equality at the scalar level changes CFI by -.046 and RMSEA by +0.121, and the likelihood-ratio test rejects equal intercepts (p < .001). No single CFI change, RMSEA change, SRMR change, or chi-square rule is treated as universally decisive; the evidence is read together, in context.
plot(inv, type = "change")
Localizing strain
Global change in fit says that a set of constraints is strained. Score diagnostics help locate where.
strain <- nomo_table(inv, "local_strain")
strain[, c("level", "constraint_display", "score_x2", "df", "p_value")]
#> # A tibble: 24 × 5
#> level constraint_display score_x2 df p_value
#> <chr> <chr> <dbl> <dbl> <dbl>
#> 1 metric Loading: Agency -> ag3 (online vs. paper) 4.92 1 2.66e- 2
#> 2 metric Loading: Agency -> ag2 (online vs. paper) 1.03 1 3.10e- 1
#> 3 metric Loading: Agency -> ag4 (online vs. paper) 0.690 1 4.06e- 1
#> 4 metric Loading: Agency -> ag1 (online vs. paper) 0.0273 1 8.69e- 1
#> 5 scalar Intercept: ag3 (online vs. paper) 61.1 1 5.33e-15
#> 6 scalar Intercept: ag1 (online vs. paper) 11.3 1 7.68e- 4
#> 7 scalar Intercept: ag4 (online vs. paper) 5.56 1 1.83e- 2
#> 8 scalar Intercept: ag2 (online vs. paper) 0.977 1 3.23e- 1
#> 9 scalar Loading: Agency -> ag2 (online vs. paper) 0.546 1 4.60e- 1
#> 10 scalar Loading: Agency -> ag1 (online vs. paper) 0.317 1 5.74e- 1
#> # ℹ 14 more rows
plot(inv, type = "local_strain")
The largest scalar-level diagnostic is Intercept: ag3 (online vs. paper) (score chi-square 61.12), which matches the item bias built into the data.
Two cautions matter for real research:
- Score diagnostics never authorize an automatic parameter release. With many constraints, some diagnostics will look notable by chance; look at the smaller metric-level values above for an example.
- In real data you do not know the population model. A release chosen only because its diagnostic was largest is a post hoc decision and should be reported as one.
Researcher-controlled partial invariance
release <- nomo_partial(
level = "scalar",
syntax = "ag3 ~ 1",
rationale = paste(
"The ag3 wording was expected to read differently on paper forms;",
"this expectation was documented before the analysis."
)
)
inv_partial <- nomo_invariance(
agency_model,
data = nomo_demo_network,
group = "group",
partial = release
)
nomo_table(inv_partial, "partial")
#> # A tibble: 1 × 4
#> release_id level syntax rationale
#> <chr> <chr> <chr> <chr>
#> 1 P1 scalar ag3 ~ 1 The ag3 wording was expected to read differently on…
fit_partial <- nomo_table(inv_partial, "fit")
fit_partial[, c("level", "partial_requested", "cfi", "rmsea", "delta_cfi", "lrt_p")]
#> # A tibble: 4 × 6
#> level partial_requested cfi rmsea delta_cfi lrt_p
#> <chr> <chr> <dbl> <dbl> <dbl> <dbl>
#> 1 configural "" 1 0 NA NA
#> 2 metric "" 1 0 0 0.146
#> 3 scalar "ag3 ~ 1" 1 0 0 0.582
#> 4 strict "ag3 ~ 1" 1 0 0 0.460With the ag3 intercept released, the remaining intercept
equalities are consistent with the data, and the latent mean difference
can be interpreted using the three invariant intercepts. The release is
carried forward to the strict level so that a parameter freed at one
level is not silently constrained again later.
nomologR records what was released and why. It does not
search until a model meets a preferred cutoff.
Comparing the groups: known-groups evidence
Once the intercepts are invariant, fully or partially, the groups can
be compared on the construct itself (Byrne et al., 1989). A difference
that theory predicts between groups that differ on the construct is
known-groups evidence. At each level that holds intercepts equal,
nomo_invariance() reports each group’s latent means
relative to the reference group. The reference group’s mean is fixed at
0 and its variance at 1, so each mean is a difference in the reference
group’s latent standard deviations (Hancock, 2001).
full_means <- nomo_table(inv, "latent_means")
partial_means <- nomo_table(inv_partial, "latent_means")
full_means[full_means$level == "scalar", c("group", "factor", "estimate", "ci_lower", "ci_upper")]
#> # A tibble: 1 × 5
#> group factor estimate ci_lower ci_upper
#> <chr> <chr> <dbl> <dbl> <dbl>
#> 1 paper Agency 0.444 0.286 0.603
partial_means[partial_means$level == "scalar", c("group", "factor", "estimate", "ci_lower", "ci_upper")]
#> # A tibble: 1 × 5
#> group factor estimate ci_lower ci_upper
#> <chr> <chr> <dbl> <dbl> <dbl>
#> 1 paper Agency 0.329 0.171 0.487The population difference is 0.25 SD. Holding the biased
ag3 intercept equal inflates it to 0.44, because the model
reads the item’s extra endorsement on paper as more Agency. With that
intercept released, the difference is 0.33, 95% CI [0.17, 0.49]. So
known-groups evidence is only as good as the invariance beneath it: a
group difference computed before the intercepts are examined can be item
bias.
Ordered indicators
Ordered indicators require identification-aware sequences rather than merely substituting WLSMV for ML.
For four-or-more-category ordered indicators:
configural -> thresholds -> metric -> scalar -> strict
For three-category indicators, threshold equality is part of the metric step because it is needed for identification. For binary indicators, threshold, loading, and intercept restrictions cannot be treated as independent sequential tests under the implemented Wu–Estabrook approach, so the sequence begins:
configural -> strong -> strict
where strong imposes thresholds, loadings, and
intercepts together.
The example below uses four five-category items from
nomo_demo_ordinal. The two “forms” are assigned by
alternating rows purely to demonstrate the ordered workflow; no
non-invariance was built into these groups.
ordinal_items <- nomo_demo_ordinal[, c("a1", "a2", "a3", "a4")]
ordinal_items$form <- rep(c("form_1", "form_2"), length.out = nrow(ordinal_items))
inv_ord <- nomo_invariance(
"A =~ a1 + a2 + a3 + a4",
data = ordinal_items,
group = "form",
ordered = c("a1", "a2", "a3", "a4")
)
nomo_table(inv_ord, "categories")
#> # A tibble: 4 × 2
#> item categories
#> <chr> <int>
#> 1 a1 5
#> 2 a2 5
#> 3 a3 5
#> 4 a4 5
fit_ord <- nomo_table(inv_ord, "fit")
fit_ord[, c("level", "constraints", "cfi", "rmsea", "delta_cfi")]
#> # A tibble: 5 × 5
#> level constraints cfi rmsea delta_cfi
#> <chr> <chr> <dbl> <dbl> <dbl>
#> 1 configural none 0.998 0.0522 NA
#> 2 thresholds thresholds 0.997 0.0372 -0.00119
#> 3 metric thresholds, loadings 0.997 0.0331 0.0000218
#> 4 scalar thresholds, loadings, intercepts 0.998 0.0227 0.00150
#> 5 strict thresholds, loadings, intercepts, residuals 1 0 0.00193Across occasions: longitudinal invariance
The same question arises when the same people answer the same items
more than once. A change in scores is a change in the construct only if
the items relate to it in the same way on each occasion. An item whose
meaning drifts, through practice or an intervention, otherwise shows up
as change (Widaman et al., 2010).
nomo_invariance_longitudinal() applies the same sequence
across occasions instead of groups.
nomo_demo_longitudinal has four Wellbeing items answered
on three occasions. The latent mean rises by 0.30 and then 0.50 of the
first occasion’s standard deviation, and the intercept of
w3 also rises by 0.40 after the first occasion: a drift the
analysis should find. The model is written for one occasion, in the
items’ own names, and columns says how an item and an
occasion name a column; the default "{item}_{occasion}"
matches w1_t1 to w4_t3. Each item’s residuals
are correlated across occasions, because what an item measures besides
the construct is usually stable too.
long <- nomo_invariance_longitudinal(
"Wellbeing =~ w1 + w2 + w3 + w4",
data = nomo_demo_longitudinal,
occasions = c("t1", "t2", "t3"),
levels = c("configural", "metric", "scalar")
)
summary(long)
#> <nomo_invariance_longitudinal summary> Measurement invariance across occasions
#> Cases: 500 | Estimator: ML
#> Occasions: t1, t2, t3 | Indicators: continuous
#> Levels completed: configural -> metric -> scalar
#>
#> Identification and sequence
#> Continuous indicators use the conventional configural, metric, scalar, and
#> strict sequence.
#>
#> Fit by level
#> Level Chi-square df p CFI RMSEA SRMR
#> configural 32.78 39 .748 1.000 0.000 0.020
#> metric 48.13 45 .347 .999 0.012 0.030
#> scalar 137.20 51 < .001 .963 0.058 0.045
#> Held equal: loadings from metric; intercepts from scalar.
#>
#> Changes from the preceding level
#> Level CFI change RMSEA change SRMR change Delta chi-square df p
#> metric -.001 +0.012 +0.010 15.35 6 .018
#> scalar -.036 +0.046 +0.015 89.07 6 < .001
#>
#> Latent change from t1, in its latent standard deviations
#> Level Occasion Factor Change 95% CI p
#> scalar t2 Wellbeing 0.41 [0.31, 0.51] < .001
#> scalar t3 Wellbeing 0.66 [0.54, 0.78] < .001
#> Comparable only with invariant intercepts, full or partial.
#>
#> Largest score diagnostics for equality constraints
#> Level Constraint Score chi-square df p
#> scalar Intercept: w3 (t3 vs. others) 22.93 1 < .001
#> scalar Intercept: w3 (t2 vs. others) 10.85 1 < .001
#> scalar Intercept: w4 (t3 vs. others) 7.97 1 .005
#> scalar Intercept: w1 (t3 vs. others) 5.47 1 .019
#> metric Loading: Wellbeing -> w4 (t3 vs. others) 4.23 1 .040
#> metric Loading: Wellbeing -> w3 (t3 vs. others) 4.13 1 .042
#> metric Loading: Wellbeing -> w1 (t2 vs. others) 3.52 1 .061
#> metric Loading: Wellbeing -> w4 (t2 vs. others) 3.21 1 .073
#> scalar Intercept: w2 (t2 vs. others) 2.85 1 .091
#> scalar Loading: Wellbeing -> w1 (t2 vs. others) 2.76 1 .097
#> Each diagnostic frees one occasion's parameter while the other occasions
#> stay equal: "vs. others" names the occasion freed. The first occasion (t1)
#> is the reference and is never freed on its own, so when it alone differs,
#> every other occasion shows similar strain on that parameter.
#>
#> Flagged
#> - Score diagnostics (Review): 24 univariate equality-constraint score
#> diagnostics were retained. Use these diagnostics to localize strain, not
#> to authorize automatic constraint release. Partial invariance requires an
#> explicit researcher specification and rationale.
#>
#> What these columns mean
#> CFI -- Comparative fit index.
#> RMSEA -- Root mean square error of approximation.
#> SRMR -- Standardized root mean square residual.
#> df -- Degrees of freedom.
#> CI -- Confidence interval.
#> ML -- Maximum likelihood.
#>
#> No single CFI change, RMSEA change, SRMR change, chi-square difference, or
#> score diagnostic is treated as a universal invariance rule.
#>
#> See nomo_table(x, "local_strain") for all 24 score diagnostics and
#> nomo_table(x, "decision_log") for every recorded decision.Fit holds up with the loadings equal and drops once the intercepts
are too (CFI change -.036), and the largest score diagnostics point to
the w3 intercept. Held equal, that intercept inflates the
latent change at t3 to 0.66 standard deviations. Releasing
it is a documented decision, as across groups:
long_partial <- nomo_invariance_longitudinal(
"Wellbeing =~ w1 + w2 + w3 + w4",
data = nomo_demo_longitudinal,
occasions = c("t1", "t2", "t3"),
levels = c("configural", "metric", "scalar"),
partial = nomo_partial(
level = "scalar",
syntax = "w3 ~ 1",
rationale = "The score diagnostics point to the w3 intercept."
)
)
nomo_table(long_partial, "latent_means")[, c("occasion", "estimate", "ci_lower", "ci_upper")]
#> # A tibble: 2 × 4
#> occasion estimate ci_lower ci_upper
#> <chr> <dbl> <dbl> <dbl>
#> 1 t2 0.321 0.222 0.421
#> 2 t3 0.549 0.430 0.669The release names the item as in the one-occasion model and frees its
intercept on every occasion. The change is then carried by the other
three items, and its intervals cover the population’s 0.30 and 0.50.
With real data the release still needs a reason beyond the diagnostics:
here, something about w3 that could have changed its
meaning between occasions.
Tables and figures
plot(inv, type = "fit")
nomo_table() returns ordinary tibbles
("fit", "categories", "partial",
"local_strain", "decision_log"), and plots are
ordinary ggplot objects, so researchers keep direct control over
reporting. The same evidence flows into nomo_report() when
invariance is part of a guided workflow.
Research basis
Multiple-group factor analysis originates with Jöreskog (1971), and
the configural–metric–scalar–strict hierarchy with Meredith (1993) and
Vandenberg and Lance (2000). A decrease in CFI of .01 became a widely
used rule after Cheung and Rensvold (2002); later work showed that
sensitivity of change-in-fit indices depends on sample size and model
context (Chen, 2007; Putnick & Bornstein, 2016), which is why
nomologR reports several indices without a universal
cutoff. Ordered-indicator sequences follow Wu and Estabrook (2016) as
implemented in semTools (see also Svetina et al., 2020).
Partial invariance follows Byrne et al. (1989), and latent mean
comparisons follow Hancock (2001). Invariance across occasions follows
Widaman et al. (2010), with Liu et al. (2017) for ordered items. Full
references are in ?nomo_invariance,
?nomo_invariance_longitudinal, and the research basis article.