Research basis: from historical to contemporary practice
Source:vignettes/research-basis.Rmd
research-basis.RmdPurpose
Scale development has a long methodological history. Many techniques
that graduate students meet in published articles were reasonable
responses to the computing and theory of their time, and several have
since been refined or replaced. This article maps each
nomologR workflow stage from historical
practice to contemporary practice, states
what nomologR does, and notes what is
planned.
How to read the labels:
-
Historical practice — widely used earlier and still
common in published work.
nomologRshows these methods as context when learners need to recognize them, labels them, and never silently substitutes them for contemporary evidence. “Historical” is not the same as “always wrong.” -
Contemporary practice — what the current
methodological literature recommends. These methods provide
nomologR’s primary evidence. - Planned — work accepted for a future release, linked to its issue.
Full references appear at the end of this article and in each function’s help page.
Scope: reflective measurement
Every method below assumes reflective (effect) indicators, which the
construct causes and which should therefore covary (Bollen & Lennox,
1991). Formative (causal) indicators define their construct instead.
They need not covary, and removing one changes the construct
(Diamantopoulos & Winklhofer, 2001), so internal consistency,
loadings-based item review, and the validity indices here do not apply
to them. Treating one model as the other biases structural estimates,
and Jarvis et al. (2003) and MacKenzie et al. (2005) give criteria for
choosing between them. The formative model itself remains contested
(Howell et al., 2007; Edwards, 2011; Bollen & Diamantopoulos, 2017).
nomologR does not fit formative latent variables; a
formative index can enter nomo_network() as an observed
variable.
The methods registry
The same mapping is available as data. nomo_methods()
returns one row per method the package implements, with its stage,
lineage, role, estimand, key assumptions, implementing function,
computational engine, and references.
methods <- nomo_methods()
nrow(methods)
#> [1] 102
table(methods$stage, methods$lineage)
#>
#> contemporary emerging historical
#> cfa 15 0 3
#> compare 6 0 0
#> efa 4 0 3
#> factors 7 1 4
#> invariance 8 0 0
#> network 5 0 1
#> reliability 12 0 2
#> scores 4 0 3
#> screen 10 0 1
#> validity 3 0 2
#> workflow 6 0 2The role column records how the evidence is used. A
context method is shown so that a reader can recognize
it in published work, and it takes no part in any synthesis or
decision:
nomo_methods(lineage = "historical")[, c("stage", "method", "role")]
#> # A tibble: 21 × 3
#> stage method role
#> <chr> <chr> <chr>
#> 1 screen Fixed item-total correlation reference (about .30) cont…
#> 2 factors Eigenvalue-greater-than-one rule cont…
#> 3 factors Scree test cont…
#> 4 factors Kaiser-Meyer-Olkin sampling adequacy supp…
#> 5 factors Bartlett's test of sphericity supp…
#> 6 efa Orthogonal (varimax) rotation cont…
#> 7 efa Other orthogonal rotation (quartimax, equamax, varimin, geomin… cont…
#> 8 efa Fixed loading cutoff cont…
#> 9 cfa Chi-square exact-fit test supp…
#> 10 cfa Fixed fit-index cutoffs cont…
#> # ℹ 11 more rowsThe registry lists only what the package actually computes. Methods
that are planned but not yet implemented appear in this article and in
the issue tracker, not in the registry, so that
nomo_methods() never describes a capability the package
lacks.
Given a workflow object, nomo_methods(run) returns only
the methods that workflow used. For any selection,
references = TRUE expands the result into a reference
list:
nomo_methods(stage = "reliability", references = TRUE)[, c("method", "citation")]
#> # A tibble: 30 × 2
#> method citation
#> <chr> <chr>
#> 1 Model-based coefficient omega Dunn, T. J., Baguley, T., & B…
#> 2 Model-based coefficient omega McNeish, D. (2018). Thanks co…
#> 3 Model-based coefficient omega Flora, D. B. (2020). Your coe…
#> 4 Model-based coefficient omega Bell, S. M., Chalmers, R. P.,…
#> 5 Reliability on the ordered-score scale Green, S. B., & Yang, Y. (200…
#> 6 Coefficient alpha Cronbach, L. J. (1951). Coeff…
#> 7 Coefficient alpha Sijtsma, K. (2009). On the us…
#> 8 Bootstrap confidence intervals for reliability Kelley, K., & Pornprasertmani…
#> 9 Omega hierarchical Reise, S. P. (2012). The redi…
#> 10 Omega hierarchical Reise, S. P., Bonifay, W. E.,…
#> # ℹ 20 more rowsThis is what the Methods and citations section of
nomo_report() is built from, so an archived report cites
the methods a run used rather than every method the package knows about.
Every DOI in the registry is checked against the DOI registries before
each release, including that the registered author, year, and title
match the citation.
How practice changed
The registry also records when each method entered the literature,
and, for a historical method, which contemporary methods now address the
question it answered. The tables below are drawn from
nomo_methods(), so they cannot drift from what the package
computes.
A method is dated only from the publication that introduced it, and only when the registry cites that publication. 54 of the 102 methods are not yet dated, because the registry cites a later review or critique instead of the original. Adding those originating references is ongoing work.
dated <- methods[!is.na(methods$introduced), , drop = FALSE]
dated <- dated[order(dated$introduced, dated$stage), , drop = FALSE]
knitr::kable(
dated[, c("introduced", "method", "stage", "lineage")],
col.names = c("Year", "Method", "Stage", "Lineage"),
row.names = FALSE
)| Year | Method | Stage | Lineage |
|---|---|---|---|
| 1937 | Bifactor measurement model | cfa | contemporary |
| 1950 | Bartlett’s test of sphericity | factors | historical |
| 1951 | Coefficient alpha | reliability | historical |
| 1954 | Eigenvalue-greater-than-one rule | factors | historical |
| 1955 | Nomological network of construct relations | network | historical |
| 1957 | Schmid-Leiman decomposition | reliability | historical |
| 1958 | Orthogonal (varimax) rotation | efa | historical |
| 1965 | Common-factor parallel analysis | factors | contemporary |
| 1966 | Scree test | factors | historical |
| 1969 | Maximum-likelihood confirmatory factor analysis | cfa | contemporary |
| 1969 | Chi-square exact-fit test | cfa | historical |
| 1970 | Kaiser-Meyer-Olkin sampling adequacy | factors | historical |
| 1971 | Multiple-group confirmatory factor analysis | invariance | contemporary |
| 1974 | Information criteria (AIC, BIC) | compare | contemporary |
| 1976 | Velicer original minimum average partial (MAP) | factors | contemporary |
| 1980 | Incremental fit indices (CFI, TLI) | cfa | contemporary |
| 1981 | Standardized loadings and average variance extracted | validity | historical |
| 1981 | Fornell-Larcker comparison | validity | historical |
| 1987 | Equivalence testing against a smallest effect size of interest | network | contemporary |
| 1988 | Measurement-then-structure modeling | network | contemporary |
| 1989 | Partial invariance with documented releases | invariance | contemporary |
| 1992 | RMSEA with confidence interval | cfa | contemporary |
| 1993 | Configural, metric, scalar, and strict sequence | invariance | contemporary |
| 1996 | Power of the RMSEA tests of close, not-close, and exact fit | cfa | contemporary |
| 1999 | Fixed fit-index cutoffs | cfa | historical |
| 2000 | Scaled-and-shifted difference test | compare | contemporary |
| 2000 | Velicer revised MAP (fourth matrix power) | factors | contemporary |
| 2001 | Satorra-Bentler scaled difference test | compare | contemporary |
| 2001 | Construct replicability (H) | reliability | contemporary |
| 2002 | Change-in-fit indices | compare | contemporary |
| 2002 | Change-in-fit evidence across levels | invariance | contemporary |
| 2009 | Exploratory structural equation modeling beside its CFA | cfa | contemporary |
| 2009 | Reliability on the ordered-score scale | reliability | contemporary |
| 2010 | Comprehensive CFA marker technique for method variance | cfa | contemporary |
| 2010 | Formal model-nesting check | compare | contemporary |
| 2011 | Hull method | factors | contemporary |
| 2012 | Comparison data | factors | contemporary |
| 2012 | Percentage of uncontaminated correlations | reliability | contemporary |
| 2013 | Omega hierarchical subscale | reliability | contemporary |
| 2015 | Inter-item standard deviation | screen | contemporary |
| 2015 | Heterotrait-monotrait ratio | validity | contemporary |
| 2016 | Identification-aware sequence for ordered indicators | invariance | contemporary |
| 2017 | Empirical Kaiser criterion | factors | contemporary |
| 2017 | Next Eigenvalue Sufficiency Test (NEST) | factors | emerging |
| 2018 | A priori versus post hoc prediction provenance | network | contemporary |
| 2020 | Test of the constraints unit weighting assumes | scores | contemporary |
| 2021 | HTMT2 | validity | contemporary |
| 2022 | Latent correlations with confidence intervals | validity | contemporary |
Read by era, the timeline tells a story of methods shaped by the computing of their time and then revisited.
- 1930s to 1950s. The field laid its foundations: the bifactor model (Holzinger & Swineford, 1937), coefficient alpha (Cronbach, 1951), the nomological network (Cronbach & Meehl, 1955), and the Schmid-Leiman transformation (1957). Factor analysis was computed by hand, and rules that were easy to apply, such as retaining factors with eigenvalues greater than one (Guttman, 1954; Kaiser, 1960), became habits.
- 1960s to 1980s. Structural modeling arrived. Horn (1965) proposed parallel analysis and Cattell (1966) the scree test. Jöreskog made confirmatory factor analysis practical (1969) and extended it to several groups (1971). Researchers gained information criteria (Akaike, 1974), incremental fit (Bentler & Bonett, 1980), the Fornell-Larcker comparison (1981), and the two-step logic of measurement before structure (Anderson & Gerbing, 1988).
- 1990s and 2000s. Fit and invariance were formalized: an interval for the RMSEA (Browne & Cudeck, 1992), the invariance hierarchy (Meredith, 1993), the fit-index cutoffs that became conventions (Hu & Bentler, 1999), scaled difference tests (Satorra, 2000; Satorra & Bentler, 2001), and change-in-fit criteria (Cheung & Rensvold, 2002; Chen, 2007).
- 2010s to the present. Many defaults were reconsidered: better factor-retention criteria (Hull, comparison data, the empirical Kaiser criterion, NEST), bifactor indices (Reise, 2012), heterotrait-monotrait ratios (Henseler et al., 2015; Roemer et al., 2021), identification for ordered indicators (Wu & Estabrook, 2016), the cost of sum scores (McNeish & Wolf, 2020), and latent correlations with intervals as discriminant evidence (Rönkkö & Cho, 2022).
What took over from each historical method:
old <- methods[!is.na(methods$contemporary_practice), , drop = FALSE]
now <- vapply(
strsplit(old$contemporary_practice, "; ", fixed = TRUE),
function(ids) paste(methods$method[match(ids, methods$id)], collapse = "; "),
character(1)
)
knitr::kable(
data.frame(Year = ifelse(is.na(old$introduced), "", old$introduced),
Historical = old$method, Now = now),
row.names = FALSE
)| Year | Historical | Now |
|---|---|---|
| Fixed item-total correlation reference (about .30) | Corrected item-rest correlation | |
| 1954 | Eigenvalue-greater-than-one rule | Common-factor parallel analysis; Velicer revised MAP (fourth matrix power); Empirical Kaiser criterion |
| 1966 | Scree test | Common-factor parallel analysis |
| 1958 | Orthogonal (varimax) rotation | Oblique rotation |
| Other orthogonal rotation (quartimax, equamax, varimin, geomin, Bentler) | Oblique rotation | |
| Fixed loading cutoff | Cross-loading, communality, and residual diagnostics | |
| 1969 | Chi-square exact-fit test | Incremental fit indices (CFI, TLI); RMSEA with confidence interval; Standardized root mean square residual; Localized residual correlations |
| 1999 | Fixed fit-index cutoffs | Localized residual correlations |
| Modification indices | Localized residual correlations; Recorded revision lineage | |
| 1951 | Coefficient alpha | Model-based coefficient omega |
| 1957 | Schmid-Leiman decomposition | Bifactor measurement model |
| 1981 | Fornell-Larcker comparison | HTMT2; Latent correlations with confidence intervals |
| Unit-weighted sum or mean score | Test of the constraints unit weighting assumes | |
| 1955 | Nomological network of construct relations | Measurement-then-structure modeling; A priori versus post hoc prediction provenance |
| Listwise deletion | Full-information maximum likelihood for missing data; Missing-data sensitivity comparison | |
| Pairwise deletion | Full-information maximum likelihood for missing data; Missing-data sensitivity comparison |
“Now” does not mean that the historical method is wrong. It means
that contemporary work answers the same question differently, usually
with evidence the historical method could not provide.
nomologR shows the historical method as labeled context
where a reader will meet it, and bases its own evidence on the
contemporary one.
1. Item and data audit — nomo_screen()
Historical practice. Item analysis often meant screening corrected item–total correlations against a fixed value (a reference near .30 is widely taught; see Nunnally & Bernstein, 1994) and deleting items that fell below it, with little attention to response distributions, missing data, or response quality.
Contemporary practice. Item statistics are one strand of evidence, interpreted alongside content, construct definition, and later structural evidence (Clark & Watson, 2019). Audits describe missingness, category use, and response concentration before modeling. Case-level screening for careless or insufficient-effort responding uses several complementary indices rather than a single exclusion rule (Meade & Craig, 2012; Huang et al., 2012; Curran, 2016).
What nomologR does. Reports item and case
missingness, response distributions, unused categories, response
concentration, near-zero variance (using heuristics described by Kuhn
& Johnson, 2013), corrected item–rest and inter-item correlations,
and descriptive shape statistics. Flags are review prompts; no item or
case is removed or recoded. With effort = TRUE it adds the
careless-responding indices those sources describe (long-string,
inter-item standard deviation, Mahalanobis distance, even-odd
consistency, and psychometric antonyms and synonyms). They are reported
side by side because they detect different failures, and a case is
flagged only where a source states a rule.
Items can arrive from content review: a handoff from
contentvalidR supplies the carried items and their
construct mapping. Held-back items are reported in that package’s own
words, and the log records that item membership came from content review
rather than from these data (#46).
2. Dimensionality — nomo_factors()
Historical practice. Retaining factors with eigenvalues greater than one (Guttman, 1954; Kaiser, 1960) and visually inspecting the scree plot (Cattell, 1966). The “Little Jiffy” routine — principal components, eigenvalues greater than one, and varimax rotation — became a default in software (discussed by Kaiser, 1970).
Contemporary practice. Simulation research favors parallel analysis (Horn, 1965; Crawford et al., 2010) and Velicer’s minimum average partial criterion (Velicer, 1976; Velicer et al., 2000), with newer criteria such as the empirical Kaiser criterion (Braeken & van Assen, 2017), comparison data (Ruscio & Roche, 2012), NEST (Achim, 2017), and the Hull method (Lorenzo-Seva et al., 2011). Criteria are triangulated, and the correlation model matches the measurement level of the items.
What nomologR does. Common-factor parallel analysis is the primary evidence, complemented by original and revised MAP and EKC; NEST, Hull, and comparison data are optional. The eigenvalue-greater-than-one rule is displayed only as labeled historical context and is excluded from the synthesis. KMO (Kaiser, 1974) and Bartlett’s test (Bartlett, 1950) are supporting diagnostics, not decision rules. Pearson, polychoric, tetrachoric, or mixed correlations are chosen explicitly.
3. Exploratory structure — nomo_efa()
Historical practice. Principal components analysis reported as “factor analysis,” orthogonal varimax rotation by default (Kaiser, 1958), and deleting items below loading cutoffs. Reviews documented how common these choices were (Fabrigar et al., 1999; Conway & Huffcutt, 2003).
Contemporary practice. Common-factor extraction, oblique rotation whenever factors may plausibly correlate (Browne, 2001; Fabrigar et al., 1999), attention to cross-loadings, communalities, and residual correlations, and item decisions that combine numerical evidence with content (Costello & Osborne, 2005; Watkins, 2018).
What nomologR does. MINRES common-factor extraction with oblimin rotation by default; pattern and structure matrices; communalities; cross-loading and residual diagnostics; and item guidance with explanations, flagging an item for review or concern and never for deletion. Orthogonal rotation is allowed but recorded as a choice that needs justification.
4. Confirmatory measurement model — nomo_cfa()
Historical practice. Treating the chi-square exact-fit test as the sole criterion (Jöreskog, 1969); judging incremental fit indices against .90 (Bentler & Bonett, 1980); later, treating the cutoffs of Hu and Bentler (1999) — CFI and TLI near .95, RMSEA near .06, SRMR near .08 — as universal golden rules; and freeing parameters suggested by modification indices until a model “fits.”
Contemporary practice. Fit indices are interpreted in the context of model size, estimator, sample size, and indicator type rather than as universal rules (Marsh et al., 2004). RMSEA is reported with its confidence interval (Browne & Cudeck, 1992). Local strain in residuals is inspected. Estimators match the indicators — for example WLSMV for ordered categories (Flora & Curran, 2004; Rhemtulla et al., 2012). Data-driven respecification is recognized as capitalization on chance (MacCallum et al., 1992). Improper solutions such as Heywood cases are treated as possible signs of misspecification (Kolenikov & Bollen, 2012). Model-specific (“dynamic”) fit-index cutoffs are an emerging alternative to fixed thresholds (McNeish & Wolf, 2023).
What nomologR does. Retains the unchanged
lavaan fit (Rosseel, 2012) and reports chi-square, CFI
(Bentler, 1990), TLI, RMSEA with its interval, and SRMR against teaching
references; localized residual correlations; Heywood and convergence
diagnostics; and modification indices quarantined as post hoc
diagnostics that never change the model. Declared ordered indicators
request WLSMV. nomo_split() supports calibration and
validation samples because evaluating an exploratory solution on the
same data it came from overstates confirmation (Fokkema & Greiff,
2017).
Comparing models. Historically, competing models
were compared by subtracting chi-square values and degrees of freedom,
even for robust estimators, or by applying one fixed change-in-fit
cutoff. Contemporary practice uses the difference test that matches the
estimator — scaled differences for robust ML (Satorra & Bentler,
2001, 2010) and scaled-and-shifted tests for categorical estimators
(Satorra, 2000) — verifies nesting rather than assuming it (Bentler
& Satorra, 2010), reports several changes in fit (Cheung &
Rensvold, 2002; Chen, 2007), and compares non-nested models of the same
data with information criteria (Akaike, 1974; Schwarz, 1978; Raftery,
1995; Burnham & Anderson, 2004). nomo_compare()
implements this: it requires a recorded rationale, labels post hoc
comparisons, refuses comparisons across different estimators, cases, or
data, and never selects a model automatically.
Revising a model. Historically, a model that fit
poorly was respecified in place, and the published model was reported as
though it had been the plan. nomo_revise() instead creates
a child workflow from the parent run, records what changed, the
researcher’s rationale, and whether the change was prespecified or post
hoc, compares the parent and revised models, and recommends confirming a
post hoc revision in independent data.
Cross-loadings. Historically, a confirmatory model
fixed every cross-loading at zero, and items with small cross-loadings
were deleted or their misfit tolerated. The constraint pushes the
variance those loadings carry into the factor correlations, which are
then inflated (Asparouhov & Muthén, 2009). Exploratory structural
equation modeling estimates the cross-loadings within a model that still
gives fit and standard errors, and with an a priori structure, target
rotation keeps the hypothesized pattern without fixing the
cross-loadings (Browne, 2001; Marsh et al., 2014).
nomo_esem() fits ESEM beside the CFA and compares them on
fit indices that penalize ESEM’s extra parameters and on the factor
correlations.
Method variance. Historically, common method
variance was dismissed with Harman’s single-factor test, which cannot
detect what it is used to rule out (Podsakoff et al., 2003), or removed
from observed correlations with a marker variable’s partial correlation
(Lindell & Whitney, 2001). Williams et al.’s (2010) comprehensive
CFA marker technique models the marker’s method factor, tests whether it
biases the substantive correlations, and decomposes each factor’s
reliability. With a marker that does not tap the relevant biases it can
find method variance that is absent, and it does not recover the
substantive correlations accurately (Richardson et al., 2009); other
sources of method variance may remain (Podsakoff et al., 2012).
nomo_method_variance() implements the technique and reports
its results as evidence about the marker’s method variance, not as
corrected estimates.
Sample size. Historically, sample size was set by
rules of thumb, such as a minimum N or a number of cases per indicator
or parameter. The sample a model needs varies widely with its loadings,
factors, and indicators, from 30 to 460 cases in Wolf et al.’s (2013)
simulations, so no single rule fits. Contemporary practice computes
power for a planned analysis: for the RMSEA tests of close, not-close,
and exact fit from the model’s degrees of freedom (MacCallum et al.,
1996), and for the parameters themselves by Monte Carlo simulation from
a population model, checking bias, coverage, and power together (Muthén
& Muthén, 2002). nomo_power_rmsea() and
nomo_power_simulate() implement both.
Planned. Model-specific fit cutoffs remain a research proposal (#23).
5. Reliability — nomo_reliability()
Historical practice. Reporting coefficient alpha (Cronbach, 1951) as “the” reliability of a scale regardless of whether its assumptions hold.
Contemporary practice. Model-based omega for congeneric measurement (Dunn et al., 2014; McNeish, 2018; Flora, 2020), recognition of alpha’s limits (Sijtsma, 2009), reliability defined on the score scale actually used for ordered items (Green & Yang, 2009), interval estimates (Kelley & Pornprasertmanit, 2016), and awareness that measurement misspecification biases omega (Bell et al., 2024).
What nomologR does. Model-based omega from
semTools is primary; alpha is a qualified secondary
statistic and is reported as unavailable rather than silently redefined
for ordered-score estimands; bootstrap intervals are optional.
Total and subscale scores. Historically, a scale
with subscales was often summarized by coefficient alpha for its
multidimensional total score. The bi-factor method (Holzinger &
Swineford, 1937) and the Schmid-Leiman orthogonalization of higher-order
exploratory solutions (Schmid & Leiman, 1957) long predate their
wide use. Contemporary practice fits confirmatory bifactor and
higher-order models and evaluates them with model-based indices: omega
hierarchical for the total score, omega hierarchical subscale for what
each subscale adds beyond the general factor, explained common variance,
and the percentage of uncontaminated correlations (Reise, 2012; Reise et
al., 2013; Rodriguez et al., 2016). A bifactor model will usually fit at
least as well as the alternatives even when it did not generate the data
(Reise, 2012), and a higher-order model is a constrained bifactor model
(Yung et al., 1999), so fit alone cannot choose between them.
nomo_model() writes all three structures,
nomo_hierarchical() computes the indices with their
estimands stated and no pass/fail thresholds, and
nomo_compare() compares the structures; see Total and subscale scores.
Stability over time. Historically, test-retest
reliability was a Pearson correlation between occasions, which ignores
systematic change (Weir, 2005), or an intraclass correlation whose form
went unreported. The forms answer different questions (Shrout &
Fleiss, 1979; McGraw & Wong, 1996). For test-retest data Koo and Li
(2016) recommend a two-way mixed-effects model with absolute agreement,
which for a single measurement is ICC(A,1), interpreted from its
confidence interval. The standard error of measurement and the smallest
detectable change put the reliability on the score’s scale (Weir, 2005),
and the reliable change index asks whether one person’s change exceeds
measurement error (Jacobson & Truax, 1991).
nomo_retest() reports all of these.
6. Convergent and discriminant evidence —
nomo_validity()
Historical practice. Multitrait–multimethod correlation matrices (Campbell & Fiske, 1959); later, average variance extracted above .50 and the Fornell–Larcker comparison of AVE with squared correlations (Fornell & Larcker, 1981).
Contemporary practice. Simulation work shows that the Fornell–Larcker comparison can miss discriminant-validity problems (Henseler et al., 2015; Voorhees et al., 2016). The heterotrait–monotrait ratio (Henseler et al., 2015) and its congeneric-appropriate successor HTMT2 (Roemer et al., 2021) are recommended, together with latent factor correlations and their confidence intervals (Rönkkö & Cho, 2022).
What nomologR does. Standardized loadings and AVE as convergent evidence (never as reliability); HTMT2 as primary construct-separation evidence with original HTMT for comparison; latent correlations with uncertainty; the Fornell–Larcker table only on explicit request, labeled legacy/supporting.
7. Measurement invariance — nomo_invariance(),
nomo_invariance_longitudinal(),
nomo_partial()
Historical practice. Multiple-group factor analysis (Jöreskog, 1971) and the configural–metric–scalar–strict hierarchy (Meredith, 1993; Vandenberg & Lance, 2000), evaluated with chi-square difference tests alone or a single fixed rule such as a CFI decrease of .01 (Cheung & Rensvold, 2002); ordinal items analyzed as if continuous.
Contemporary practice. Several change-in-fit indices interpreted with sample size and model context (Chen, 2007; Putnick & Bornstein, 2016); identification-aware sequences for ordered-categorical indicators (Wu & Estabrook, 2016; Svetina et al., 2020); partial invariance based on substantively justified, transparently reported releases (Byrne et al., 1989); and, once intercepts are invariant, latent mean differences between groups expressed in the reference group’s latent standard deviations, as known-groups evidence (Hancock, 2001). Across occasions, the same items are modeled at each occasion with each item’s unique factors correlated over time, so that change in a score can be read as change in the construct (Widaman et al., 2010), with identification conditions for ordered items over time (Liu et al., 2017).
What nomologR does. Category-aware sequences for continuous, binary, three-category, and four-or-more-category indicators; fit and change evidence without a universal pass/fail rule; score-test diagnostics that locate strain but never free parameters; researcher-specified releases with required rationales carried forward to more restrictive levels; and latent means compared with the reference group at every level that holds intercepts equal. The same sequence runs across occasions, with each item’s unique factors correlated over time, releases named by item, and latent change reported once intercepts are invariant.
8. Nomological network — nomo_hypotheses(),
nomo_network()
Historical practice. Construct validity articulated as a nomological network of lawful relations (Cronbach & Meehl, 1955), in practice often examined through tables of observed correlations and significance tests, with a non-significant result taken as evidence of “no relation.”
Contemporary practice. Establish the measurement
model before interpreting structural relations (Anderson & Gerbing,
1988); treat validity as an integrated argument about the meaning of
scores (Messick, 1995); test negligible predictions with equivalence
procedures and a smallest effect size of interest rather than
p > .05 (Schuirmann, 1987; Lakens et al., 2018); and
distinguish prespecified from post hoc predictions (Nosek et al.,
2018).
What nomologR does. Machine-readable directional, magnitude, and negligible predictions; researcher-specified equivalence regions (the package never invents one); latent SEM with observed outcomes; measurement context kept alongside theory concordance; a priori versus post hoc provenance; and replication of the exact same model in validation data. A change in sign between samples is called a reversal only when both confidence intervals exclude zero on opposite sides; otherwise it is reported as a direction that did not replicate, or as a sign change within sampling uncertainty, because estimates scattered around a null relation differ in sign about half the time.
A hypothesis whose endpoints are observed variables, such as sum or
factor scores, is disclosed in the decision log, because the measurement
error of a composite enters the relationship unmodeled. The disclosure
points to modeling the items as indicators, to estimating structural
relationships after the measurement model with
lavaan::sam() (Rosseel & Loh, 2024), and, for a linear
regression among factor scores, to the one scoring design shown to give
consistent coefficients: regression-method scores for the predictors and
Bartlett scores for the outcome, each block scored from a measurement
model of its own (Skrondal & Laake, 2001). None of these is applied
automatically.
Observed composites. A composite’s measurement error
attenuates the relations it enters (Spearman, 1904). Single-indicator
modeling corrects this by making the composite the one indicator of a
latent variable with its error variance fixed at (1 − reliability) × its
variance (Williams & Hazer, 1986; Hayduk, 1987; Bollen, 1989). The
correction is only as good as the reliability and the error it captures
(DeShon, 1998). In samples of 30 to 200, Savalei (2019) found path
analysis and single indicators with a reliability fixed in advance,
slightly above its true value, the most accurate and the most powerful,
with a reliability estimated from the same data in between; an omega
from the same sample is that data-estimated variant.
nomo_network() applies it only when the researcher names
the composite and its reliability, refits with the reliability .05 and
.10 lower and higher, and adds the reliability’s uncertainty to the
standard errors when it is known (Oberski & Satorra, 2013).
9. Workflow, scores, and reporting — nomo_run(),
nomo_report()
Historical practice. A sequence of undisclosed decisions — dropping items, adding residual covariances, trying estimators — reported as if it had been the plan all along; unit-weighted sum scores assumed to represent the construct; listwise deletion, pairwise deletion, or mean substitution applied without asking whether the data meet the assumption each requires.
Contemporary practice. Transparent reporting of measurement decisions and their justification (Flake et al., 2017; Flake & Fried, 2020), awareness of researcher degrees of freedom (Simmons et al., 2011; Wicherts et al., 2016), staged scale-development guidance (Clark & Watson, 1995, 2019; Hinkin, 1998; Boateng et al., 2018), missing-data methods such as full-information maximum likelihood (Enders & Bandalos, 2001; Schafer & Graham, 2002), and checking whether sum scores are consistent with the measurement model before using them (Grice, 2001; McNeish & Wolf, 2020).
What nomologR does. nomo_run() pauses
at consequential decisions and records the decision and rationale;
nomo_report() archives methods, evidence, decisions,
deviations, and session information. nomo_methods()
identifies which methods a workflow actually used, so the report cites
those methods and their literature rather than only the software that
computed them. nomo_scores() fits the parallel model a sum
score assumes and reports Grice’s criteria for the scores it returns,
and nomo_apa_table() formats evidence as manuscript tables
following APA style (American Psychological Association, 2020).
nomo_missing() refits a prespecified measurement model
or network under listwise deletion and FIML, or under listwise and
pairwise deletion for ordered indicators, and reports where estimates,
reliability, or theory evidence differ. Listwise and pairwise deletion
require data missing completely at random; FIML requires data missing at
random, and was unbiased under both in Enders and Bandalos’s (2001)
simulations while the deletion methods were biased under MAR. A
difference beyond half the reference standard error is flagged for
review, a reference adapted from the size of bias Schafer and Graham
(2002) treat as practically important, with two caveats: it estimates
bias only if the data are MAR, and within one sample, sampling
variability alone can exceed it. Because MAR cannot in general be tested
from the data at hand (Schafer & Graham, 2002), agreement between
strategies is reported as insensitivity, not as evidence that either is
unbiased. Mean substitution is explained and not offered.
Planned. Multiple-imputation integration is a candidate for a 1.x release after 1.0.0. The reasons for deferring it are recorded in #38 and #113.
References
Every DOI below was verified against Crossref or the DOI registry.
Achim, A. (2017). Testing the number of required dimensions in exploratory factor analysis. The Quantitative Methods for Psychology, 13(1), 64–74. https://doi.org/10.20982/tqmp.13.1.p064
Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19(6), 716–723. https://doi.org/10.1109/TAC.1974.1100705
American Psychological Association. (2020). Publication manual of the American Psychological Association (7th ed.). https://doi.org/10.1037/0000165-000
Anderson, J. C., & Gerbing, D. W. (1988). Structural equation modeling in practice: A review and recommended two-step approach. Psychological Bulletin, 103(3), 411–423. https://doi.org/10.1037/0033-2909.103.3.411
Asparouhov, T., & Muthén, B. (2009). Exploratory structural equation modeling. Structural Equation Modeling, 16(3), 397–438. https://doi.org/10.1080/10705510903008204
Bartlett, M. S. (1950). Tests of significance in factor analysis. British Journal of Statistical Psychology, 3(2), 77–85. https://doi.org/10.1111/j.2044-8317.1950.tb00285.x
Beauducel, A. (2011). Indeterminacy of factor score estimates in slightly misspecified confirmatory factor models. Journal of Modern Applied Statistical Methods, 10(2), 583–598. https://doi.org/10.22237/jmasm/1320120900
Bell, S. M., Chalmers, R. P., & Flora, D. B. (2024). The impact of measurement model misspecification on coefficient omega estimates of composite reliability. Educational and Psychological Measurement, 84(1), 5–39. https://doi.org/10.1177/00131644231155804
Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107(2), 238–246. https://doi.org/10.1037/0033-2909.107.2.238
Bentler, P. M., & Bonett, D. G. (1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 88(3), 588–606. https://doi.org/10.1037/0033-2909.88.3.588
Bentler, P. M., & Satorra, A. (2010). Testing model nesting and equivalence. Psychological Methods, 15(2), 111–123. https://doi.org/10.1037/a0019625
Boateng, G. O., Neilands, T. B., Frongillo, E. A., Melgar-Quiñonez, H. R., & Young, S. L. (2018). Best practices for developing and validating scales for health, social, and behavioral research: A primer. Frontiers in Public Health, 6, 149. https://doi.org/10.3389/fpubh.2018.00149
Bollen, K. A. (1989). Structural equations with latent variables. Wiley. https://doi.org/10.1002/9781118619179
Bollen, K. A., & Diamantopoulos, A. (2017). In defense of causal-formative indicators: A minority report. Psychological Methods, 22(3), 581–596. https://doi.org/10.1037/met0000056
Bollen, K., & Lennox, R. (1991). Conventional wisdom on measurement: A structural equation perspective. Psychological Bulletin, 110(2), 305–314. https://doi.org/10.1037/0033-2909.110.2.305
Bonifay, W., Lane, S. P., & Reise, S. P. (2017). Three concerns with applying a bifactor model as a structure of psychopathology. Clinical Psychological Science, 5(1), 184–186. https://doi.org/10.1177/2167702616657069
Braeken, J., & van Assen, M. A. L. M. (2017). An empirical Kaiser criterion. Psychological Methods, 22(3), 450–466. https://doi.org/10.1037/met0000074
Browne, M. W. (2001). An overview of analytic rotation in exploratory factor analysis. Multivariate Behavioral Research, 36(1), 111–150. https://doi.org/10.1207/S15327906MBR3601_05
Browne, M. W., & Cudeck, R. (1992). Alternative ways of assessing model fit. Sociological Methods & Research, 21(2), 230–258. https://doi.org/10.1177/0049124192021002005
Burnham, K. P., & Anderson, D. R. (2004). Multimodel inference: Understanding AIC and BIC in model selection. Sociological Methods & Research, 33(2), 261–304. https://doi.org/10.1177/0049124104268644
Byrne, B. M., Shavelson, R. J., & Muthén, B. (1989). Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. Psychological Bulletin, 105(3), 456–466. https://doi.org/10.1037/0033-2909.105.3.456
Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56(2), 81–105. https://doi.org/10.1037/h0046016
Cattell, R. B. (1966). The scree test for the number of factors. Multivariate Behavioral Research, 1(2), 245–276. https://doi.org/10.1207/s15327906mbr0102_10
Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 14(3), 464–504. https://doi.org/10.1080/10705510701301834
Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9(2), 233–255. https://doi.org/10.1207/S15328007SEM0902_5
Clark, L. A., & Watson, D. (1995). Constructing validity: Basic issues in objective scale development. Psychological Assessment, 7(3), 309–319. https://doi.org/10.1037/1040-3590.7.3.309
Clark, L. A., & Watson, D. (2019). Constructing validity: New developments in creating objective measuring instruments. Psychological Assessment, 31(12), 1412–1427. https://doi.org/10.1037/pas0000626
Conway, J. M., & Huffcutt, A. I. (2003). A review and evaluation of exploratory factor analysis practices in organizational research. Organizational Research Methods, 6(2), 147–168. https://doi.org/10.1177/1094428103251541
Costello, A. B., & Osborne, J. W. (2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research, and Evaluation, 10, Article 7. https://doi.org/10.7275/jyj1-4868
Crawford, A. V., Green, S. B., Levy, R., Lo, W.-J., Scott, L., Svetina, D., & Thompson, M. S. (2010). Evaluation of parallel analysis methods for determining the number of factors. Educational and Psychological Measurement, 70(6), 885–901. https://doi.org/10.1177/0013164410379332
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334. https://doi.org/10.1007/BF02310555
Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52(4), 281–302. https://doi.org/10.1037/h0040957
Curran, P. G. (2016). Methods for the detection of carelessly invalid responses in survey data. Journal of Experimental Social Psychology, 66, 4–19. https://doi.org/10.1016/j.jesp.2015.07.006
DeShon, R. P. (1998). A cautionary note on measurement error corrections in structural equation models. Psychological Methods, 3(4), 412–423. https://doi.org/10.1037/1082-989X.3.4.412
Diamantopoulos, A., & Winklhofer, H. M. (2001). Index construction with formative indicators: An alternative to scale development. Journal of Marketing Research, 38(2), 269–277. https://doi.org/10.1509/jmkr.38.2.269.18845
Dunn, T. J., Baguley, T., & Brunsden, V. (2014). From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. British Journal of Psychology, 105(3), 399–412. https://doi.org/10.1111/bjop.12046
Edwards, J. R. (2011). The fallacy of formative measurement. Organizational Research Methods, 14(2), 370–388. https://doi.org/10.1177/1094428110378369
Enders, C. K., & Bandalos, D. L. (2001). The relative performance of full information maximum likelihood estimation for missing data in structural equation models. Structural Equation Modeling, 8(3), 430–457. https://doi.org/10.1207/S15328007SEM0803_5
Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4(3), 272–299. https://doi.org/10.1037/1082-989X.4.3.272
Flake, J. K., & Fried, E. I. (2020). Measurement schmeasurement: Questionable measurement practices and how to avoid them. Advances in Methods and Practices in Psychological Science, 3(4), 456–465. https://doi.org/10.1177/2515245920952393
Flake, J. K., Pek, J., & Hehman, E. (2017). Construct validation in social and personality research: Current practice and recommendations. Social Psychological and Personality Science, 8(4), 370–378. https://doi.org/10.1177/1948550617693063
Flora, D. B. (2020). Your coefficient alpha is probably wrong, but which coefficient omega is right? A tutorial on using R to obtain better reliability estimates. Advances in Methods and Practices in Psychological Science, 3(4), 484–501. https://doi.org/10.1177/2515245920951747
Flora, D. B., & Curran, P. J. (2004). An empirical evaluation of alternative methods of estimation for confirmatory factor analysis with ordinal data. Psychological Methods, 9(4), 466–491. https://doi.org/10.1037/1082-989X.9.4.466
Fokkema, M., & Greiff, S. (2017). How performing PCA and CFA on the same data equals trouble: Overfitting in the assessment of internal structure and some editorial thoughts on it. European Journal of Psychological Assessment, 33(6), 399–402. https://doi.org/10.1027/1015-5759/a000460
Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39–50. https://doi.org/10.2307/3151312
Gorsuch, R. L. (1983). Factor analysis (2nd ed.). Lawrence Erlbaum.
Green, S. B., & Yang, Y. (2009). Reliability of summed item scores using structural equation modeling: An alternative to coefficient alpha. Psychometrika, 74(1), 155–167. https://doi.org/10.1007/s11336-008-9099-3
Grice, J. W. (2001). Computing and evaluating factor scores. Psychological Methods, 6(4), 430–450. https://doi.org/10.1037/1082-989X.6.4.430
Guttman, L. (1954). Some necessary conditions for common-factor analysis. Psychometrika, 19(2), 149–161. https://doi.org/10.1007/BF02289162
Hancock, G. R. (2001). Effect size, power, and sample size determination for structured means modeling and MIMIC approaches to between-groups hypothesis testing of means on a single latent construct. Psychometrika, 66(3), 373–388. https://doi.org/10.1007/BF02294440
Hancock, G. R., & Mueller, R. O. (2001). Rethinking construct reliability within latent variable systems. In R. Cudeck, S. du Toit, & D. Sörbom (Eds.), Structural equation modeling: Present and future (pp. 195–216). Scientific Software International.
Hayduk, L. A. (1987). Structural equation modeling with LISREL: Essentials and advances. Johns Hopkins University Press.
Henseler, J., Ringle, C. M., & Sarstedt, M. (2015). A new criterion for assessing discriminant validity in variance-based structural equation modeling. Journal of the Academy of Marketing Science, 43(1), 115–135. https://doi.org/10.1007/s11747-014-0403-8
Hinkin, T. R. (1998). A brief tutorial on the development of measures for use in survey questionnaires. Organizational Research Methods, 1(1), 104–121. https://doi.org/10.1177/109442819800100106
Holzinger, K. J., & Swineford, F. (1937). The bi-factor method. Psychometrika, 2(1), 41–54. https://doi.org/10.1007/BF02287965
Horn, J. L. (1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30(2), 179–185. https://doi.org/10.1007/BF02289447
Howell, R. D., Breivik, E., & Wilcox, J. B. (2007). Reconsidering formative measurement. Psychological Methods, 12(2), 205–218. https://doi.org/10.1037/1082-989X.12.2.205
Hu, L.-T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55. https://doi.org/10.1080/10705519909540118
Huang, J. L., Curran, P. G., Keeney, J., Poposki, E. M., & DeShon, R. P. (2012). Detecting and deterring insufficient effort responding to surveys. Journal of Business and Psychology, 27(1), 99–114. https://doi.org/10.1007/s10869-011-9231-8
Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59(1), 12–19. https://doi.org/10.1037/0022-006X.59.1.12
Jarvis, C. B., MacKenzie, S. B., & Podsakoff, P. M. (2003). A critical review of construct indicators and measurement model misspecification in marketing and consumer research. Journal of Consumer Research, 30(2), 199–218. https://doi.org/10.1086/376806
Jöreskog, K. G. (1969). A general approach to confirmatory maximum likelihood factor analysis. Psychometrika, 34(2), 183–202. https://doi.org/10.1007/BF02289343
Jöreskog, K. G. (1971). Simultaneous factor analysis in several populations. Psychometrika, 36(4), 409–426. https://doi.org/10.1007/BF02291366
Kaiser, H. F. (1958). The varimax criterion for analytic rotation in factor analysis. Psychometrika, 23(3), 187–200. https://doi.org/10.1007/BF02289233
Kaiser, H. F. (1960). The application of electronic computers to factor analysis. Educational and Psychological Measurement, 20(1), 141–151. https://doi.org/10.1177/001316446002000116
Kaiser, H. F. (1970). A second generation little jiffy. Psychometrika, 35(4), 401–415. https://doi.org/10.1007/BF02291817
Kaiser, H. F. (1974). An index of factorial simplicity. Psychometrika, 39(1), 31–36. https://doi.org/10.1007/BF02291575
Kelley, K., & Pornprasertmanit, S. (2016). Confidence intervals for population reliability coefficients: Evaluation of methods, recommendations, and software for composite measures. Psychological Methods, 21(1), 69–92. https://doi.org/10.1037/a0040086
Kolenikov, S., & Bollen, K. A. (2012). Testing negative error variances: Is a Heywood case a symptom of misspecification? Sociological Methods & Research, 41(1), 124–167. https://doi.org/10.1177/0049124112442138
Koo, T. K., & Li, M. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2), 155–163. https://doi.org/10.1016/j.jcm.2016.02.012
Kuhn, M., & Johnson, K. (2013). Applied predictive modeling. Springer. https://doi.org/10.1007/978-1-4614-6849-3
Lakens, D., Scheel, A. M., & Isager, P. M. (2018). Equivalence testing for psychological research: A tutorial. Advances in Methods and Practices in Psychological Science, 1(2), 259–269. https://doi.org/10.1177/2515245918770963
Lindell, M. K., & Whitney, D. J. (2001). Accounting for common method variance in cross-sectional research designs. Journal of Applied Psychology, 86(1), 114–121. https://doi.org/10.1037/0021-9010.86.1.114
Liu, Y., Millsap, R. E., West, S. G., Tein, J.-Y., Tanaka, R., & Grimm, K. J. (2017). Testing measurement invariance in longitudinal data with ordered-categorical measures. Psychological Methods, 22(3), 486–506. https://doi.org/10.1037/met0000075
Lorenzo-Seva, U., Timmerman, M. E., & Kiers, H. A. L. (2011). The Hull method for selecting the number of common factors. Multivariate Behavioral Research, 46(2), 340–364. https://doi.org/10.1080/00273171.2011.564527
MacCallum, R. C., Browne, M. W., & Sugawara, H. M. (1996). Power analysis and determination of sample size for covariance structure modeling. Psychological Methods, 1(2), 130–149. https://doi.org/10.1037/1082-989X.1.2.130
MacCallum, R. C., Roznowski, M., & Necowitz, L. B. (1992). Model modifications in covariance structure analysis: The problem of capitalization on chance. Psychological Bulletin, 111(3), 490–504. https://doi.org/10.1037/0033-2909.111.3.490
MacKenzie, S. B., Podsakoff, P. M., & Jarvis, C. B. (2005). The problem of measurement model misspecification in behavioral and organizational research and some recommended solutions. Journal of Applied Psychology, 90(4), 710–730. https://doi.org/10.1037/0021-9010.90.4.710
Marjanovic, Z., Holden, R., Struthers, W., Cribbie, R., & Greenglass, E. (2015). The inter-item standard deviation (ISD): An index that discriminates between conscientious and random responders. Personality and Individual Differences, 84, 79–83. https://doi.org/10.1016/j.paid.2014.08.021
Marsh, H. W., Hau, K.-T., & Wen, Z. (2004). In search of golden rules: Comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler’s (1999) findings. Structural Equation Modeling, 11(3), 320–341. https://doi.org/10.1207/s15328007sem1103_2
Marsh, H. W., Morin, A. J. S., Parker, P. D., & Kaur, G. (2014). Exploratory structural equation modeling: An integration of the best features of exploratory and confirmatory factor analysis. Annual Review of Clinical Psychology, 10, 85–110. https://doi.org/10.1146/annurev-clinpsy-032813-153700
McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1(1), 30–46. https://doi.org/10.1037/1082-989X.1.1.30
McNeish, D. (2018). Thanks coefficient alpha, we’ll take it from here. Psychological Methods, 23(3), 412–433. https://doi.org/10.1037/met0000144
McNeish, D., & Wolf, M. G. (2020). Thinking twice about sum scores. Behavior Research Methods, 52(6), 2287–2305. https://doi.org/10.3758/s13428-020-01398-0
McNeish, D., & Wolf, M. G. (2023). Dynamic fit index cutoffs for confirmatory factor analysis models. Psychological Methods, 28(1), 61–88. https://doi.org/10.1037/met0000425
Meade, A. W., & Craig, S. B. (2012). Identifying careless responses in survey data. Psychological Methods, 17(3), 437–455. https://doi.org/10.1037/a0028085
Meredith, W. (1993). Measurement invariance, factor analysis and factorial invariance. Psychometrika, 58(4), 525–543. https://doi.org/10.1007/BF02294825
Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50(9), 741–749. https://doi.org/10.1037/0003-066X.50.9.741
Murray, A. L., & Johnson, W. (2013). The limitations of model fit in comparing the bi-factor versus higher-order models of human cognitive ability structure. Intelligence, 41(5), 407–422. https://doi.org/10.1016/j.intell.2013.06.004
Muthén, L. K., & Muthén, B. O. (2002). How to use a Monte Carlo study to decide on sample size and determine power. Structural Equation Modeling, 9(4), 599–620. https://doi.org/10.1207/S15328007SEM0904_8
Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018). The preregistration revolution. Proceedings of the National Academy of Sciences, 115(11), 2600–2606. https://doi.org/10.1073/pnas.1708274114
Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). McGraw-Hill.
Oberski, D. L., & Satorra, A. (2013). Measurement error models with uncertainty about the error variance. Structural Equation Modeling, 20(3), 409–428. https://doi.org/10.1080/10705511.2013.797820
Podsakoff, P. M., MacKenzie, S. B., Lee, J.-Y., & Podsakoff, N. P. (2003). Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88(5), 879–903. https://doi.org/10.1037/0021-9010.88.5.879
Podsakoff, P. M., MacKenzie, S. B., & Podsakoff, N. P. (2012). Sources of method bias in social science research and recommendations on how to control it. Annual Review of Psychology, 63, 539–569. https://doi.org/10.1146/annurev-psych-120710-100452
Putnick, D. L., & Bornstein, M. H. (2016). Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Developmental Review, 41, 71–90. https://doi.org/10.1016/j.dr.2016.06.004
Raftery, A. E. (1995). Bayesian model selection in social research. Sociological Methodology, 25, 111–163. https://doi.org/10.2307/271063
Reise, S. P. (2012). The rediscovery of bifactor measurement models. Multivariate Behavioral Research, 47(5), 667–696. https://doi.org/10.1080/00273171.2012.715555
Reise, S. P., Bonifay, W. E., & Haviland, M. G. (2013). Scoring and modeling psychological measures in the presence of multidimensionality. Journal of Personality Assessment, 95(2), 129–140. https://doi.org/10.1080/00223891.2012.725437
Rhemtulla, M., Brosseau-Liard, P. É., & Savalei, V. (2012). When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological Methods, 17(3), 354–373. https://doi.org/10.1037/a0029315
Richardson, H. A., Simmering, M. J., & Sturman, M. C. (2009). A tale of three perspectives: Examining post hoc statistical techniques for detection and correction of common method variance. Organizational Research Methods, 12(4), 762–800. https://doi.org/10.1177/1094428109332834
Rodriguez, A., Reise, S. P., & Haviland, M. G. (2016). Evaluating bifactor models: Calculating and interpreting statistical indices. Psychological Methods, 21(2), 137–150. https://doi.org/10.1037/met0000045
Roemer, E., Schuberth, F., & Henseler, J. (2021). HTMT2—An improved criterion for assessing discriminant validity in structural equation modeling. Industrial Management & Data Systems, 121(12), 2637–2650. https://doi.org/10.1108/IMDS-02-2021-0082
Rönkkö, M., & Cho, E. (2022). An updated guideline for assessing discriminant validity. Organizational Research Methods, 25(1), 6–47. https://doi.org/10.1177/1094428120968614
Rosseel, Y. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1–36. https://doi.org/10.18637/jss.v048.i02
Rosseel, Y., & Loh, W. W. (2024). A structural after measurement approach to structural equation modeling. Psychological Methods, 29(3), 561–588. https://doi.org/10.1037/met0000503
Ruscio, J., & Roche, B. (2012). Determining the number of factors to retain in an exploratory factor analysis using comparison data of known factorial structure. Psychological Assessment, 24(2), 282–292. https://doi.org/10.1037/a0025697
Satorra, A. (2000). Scaled and adjusted restricted tests in multi-sample analysis of moment structures. In R. D. H. Heijmans, D. S. G. Pollock, & A. Satorra (Eds.), Innovations in multivariate statistical analysis (pp. 233–247). Springer. https://doi.org/10.1007/978-1-4615-4603-0_17
Satorra, A., & Bentler, P. M. (2001). A scaled difference chi-square test statistic for moment structure analysis. Psychometrika, 66(4), 507–514. https://doi.org/10.1007/BF02296192
Satorra, A., & Bentler, P. M. (2010). Ensuring positiveness of the scaled difference chi-square test statistic. Psychometrika, 75(2), 243–248. https://doi.org/10.1007/s11336-009-9135-y
Savalei, V. (2019). A comparison of several approaches for controlling measurement error in small samples. Psychological Methods, 24(3), 352–370. https://doi.org/10.1037/met0000181
Schafer, J. L., & Graham, J. W. (2002). Missing data: Our view of the state of the art. Psychological Methods, 7(2), 147–177. https://doi.org/10.1037/1082-989X.7.2.147
Schmid, J., & Leiman, J. M. (1957). The development of hierarchical factor solutions. Psychometrika, 22(1), 53–61. https://doi.org/10.1007/BF02289209
Schuirmann, D. J. (1987). A comparison of the two one-sided tests procedure and the power approach for assessing the equivalence of average bioavailability. Journal of Pharmacokinetics and Biopharmaceutics, 15(6), 657–680. https://doi.org/10.1007/BF01068419
Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics, 6(2), 461–464. https://doi.org/10.1214/aos/1176344136
Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86(2), 420–428. https://doi.org/10.1037/0033-2909.86.2.420
Sijtsma, K. (2009). On the use, the misuse, and the very limited usefulness of Cronbach’s alpha. Psychometrika, 74(1), 107–120. https://doi.org/10.1007/s11336-008-9101-0
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359–1366. https://doi.org/10.1177/0956797611417632
Skrondal, A., & Laake, P. (2001). Regression among factor scores. Psychometrika, 66(4), 563–575. https://doi.org/10.1007/BF02296196
Spearman, C. (1904). The proof and measurement of association between two things. The American Journal of Psychology, 15(1), 72–101. https://doi.org/10.2307/1412159
Svetina, D., Rutkowski, L., & Rutkowski, D. (2020). Multiple-group invariance with categorical outcomes using updated guidelines: An illustration using Mplus and the lavaan/semTools packages. Structural Equation Modeling, 27(1), 111–130. https://doi.org/10.1080/10705511.2019.1602776
Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4–70. https://doi.org/10.1177/109442810031002
Velicer, W. F. (1976). Determining the number of components from the matrix of partial correlations. Psychometrika, 41(3), 321–327. https://doi.org/10.1007/BF02293557
Velicer, W. F., Eaton, C. A., & Fava, J. L. (2000). Construct explication through factor or component analysis: A review and evaluation of alternative procedures for determining the number of factors or components. In R. D. Goffin & E. Helmes (Eds.), Problems and solutions in human assessment (pp. 41–71). Springer. https://doi.org/10.1007/978-1-4615-4397-8_3
Voorhees, C. M., Brady, M. K., Calantone, R., & Ramirez, E. (2016). Discriminant validity testing in marketing: An analysis, causes for concern, and proposed remedies. Journal of the Academy of Marketing Science, 44(1), 119–134. https://doi.org/10.1007/s11747-015-0455-4
Watkins, M. W. (2018). Exploratory factor analysis: A guide to best practice. Journal of Black Psychology, 44(3), 219–246. https://doi.org/10.1177/0095798418771807
Weir, J. P. (2005). Quantifying test-retest reliability using the intraclass correlation coefficient and the SEM. Journal of Strength and Conditioning Research, 19(1), 231–240. https://doi.org/10.1519/15184.1
Wicherts, J. M., Veldkamp, C. L. S., Augusteijn, H. E. M., Bakker, M., van Aert, R. C. M., & van Assen, M. A. L. M. (2016). Degrees of freedom in planning, running, analyzing, and reporting psychological studies: A checklist to avoid p-hacking. Frontiers in Psychology, 7, 1832. https://doi.org/10.3389/fpsyg.2016.01832
Widaman, K. F., Ferrer, E., & Conger, R. D. (2010). Factorial invariance within longitudinal structural equation models: Measuring the same construct across time. Child Development Perspectives, 4(1), 10–18. https://doi.org/10.1111/j.1750-8606.2009.00110.x
Williams, L. J., Hartman, N., & Cavazotte, F. (2010). Method variance and marker variables: A review and comprehensive CFA marker technique. Organizational Research Methods, 13(3), 477–514. https://doi.org/10.1177/1094428110366036
Williams, L. J., & Hazer, J. T. (1986). Antecedents and consequences of satisfaction and commitment in turnover models: A reanalysis using latent variable structural equation methods. Journal of Applied Psychology, 71(2), 219–231. https://doi.org/10.1037/0021-9010.71.2.219
Wolf, E. J., Harrington, K. M., Clark, S. L., & Miller, M. W. (2013). Sample size requirements for structural equation models: An evaluation of power, bias, and solution propriety. Educational and Psychological Measurement, 73(6), 913–934. https://doi.org/10.1177/0013164413495237
Wu, H., & Estabrook, R. (2016). Identification of confirmatory factor analysis models of different levels of invariance for ordered categorical outcomes. Psychometrika, 81(4), 1014–1045. https://doi.org/10.1007/s11336-016-9506-0
Yung, Y.-F., Thissen, D., & McLeod, L. D. (1999). On the relationship between the higher-order factor model and the hierarchical factor model. Psychometrika, 64(2), 113–128. https://doi.org/10.1007/BF02294531