nomo_screen() performs a conservative audit of candidate item data before
factor-retention, EFA, or CFA decisions are made. It summarizes item storage
and observed response patterns, missingness, response concentration, and
case-level completeness. It also creates a decision log that distinguishes
observations from recommendations.
Usage
nomo_screen(
data,
items = NULL,
guidance = nomo_defaults(),
effort = FALSE,
scales = NULL,
reverse = NULL,
scale_range = NULL,
pair_magnitude = 0.6
)Arguments
- data
A data frame containing candidate items.
- items
Optional character vector identifying item columns. If
NULL, all columns are audited and the decision log reminds the user to verify that identifiers, demographics, and other non-item columns were not included. May also be a handoff fromcontentvalidR'scontent_handoff(); see Items from content review.- guidance
Guidance settings from
nomo_defaults(). A list that setsauto_deleteorauto_respecifytoTRUEis refused, since nomologR never deletes an item on its own.- effort
Logical. If
TRUE, case-level indices of careless or insufficient-effort responding are added. See Careless responding.- scales
Optional named list of character vectors assigning items to scales. Needed for even-odd consistency and for the within-scale versions of long-string and inter-item standard deviation. With two or more scales, each item's corrected item-rest correlation is also computed against the rest of its own scale, and the item review uses that value. Negative inter-item correlations are then reviewed only within a scale, since items of different constructs need not correlate positively. A
contentvalidRhandoff supplies its scales here.- reverse
Optional character vector naming reverse-keyed items, each of which must be an item being screened. Used only to recode an internal copy: for the careless-responding indices that need it, and to say whether a declared item's negative item-rest correlation is the sign expected before recoding. The data is never recoded.
- scale_range
Numeric
c(min, max)of the response scale, withminbelowmax. It is needed to usereverseand is never inferred from the data. Witheffort = TRUE, naming reverse-keyed items without it is refused. Witheffort = FALSEthe audit runs, and the decision log records that the declared keying was not used. When it is given, each numeric item's responses are compared with it: an item with responses outside it gets a concern in the decision log, and witheffort = TRUEa reverse-keyed item with responses outside it is refused, since recoding it on that scale would give wrong values. It also sets the floor and ceiling of numeric-discrete items (see Details).- pair_magnitude
Minimum absolute between-person correlation for an item pair to count as a psychometric antonym or synonym. Curran (2016) suggests .60 while saying there is no firm basis for it, so it is an argument rather than a constant.
Value
An object of class nomo_screen. The fields to read are:
items: the items screened, andn_cases, the number of rows.item_summary: one row per item, with its storage, inferred type, missingness, most common response, descriptive statistics, and near-zero-variance indicators.response_distribution: counts and proportions of each response.case_summary: missingness per row.relationship_summary: each item's corrected item-rest correlation and summary of its inter-item correlations. With two or more declared scales,scale,scale_item_rest_r, andscale_item_rest_ngive each item's scale and its item-rest correlation within that scale, andscale_negative_interitem_ncounts the negative correlations the review reads: with items of the same scale, and with any screened item that is outside every declared scale. Otherwise they areNA.inter_item_correlations: one row per item pair.decision_log: the evidence and its explanations (seenomo_table()).effort,effort_pairs, andeffort_settings: the careless-responding indices per row, the pairs they used, and their settings, wheneffort = TRUE. The settings includelong_string_rule_applied, which isFALSEwhen too few items were screened for the long-string rule.handoff: the content-review handoff read fromitems, when one was supplied.
pct_missing, in item_summary and case_summary, is a proportion
between 0 and 1, as are the *_prop and proportion_* columns.
percent_unique is a percentage, 0 to 100, as the near-zero-variance rule
states it. See Conventions in returned tables in ?nomologR.
Other fields record the call, the settings used, and intermediate engine
results. They may change between releases and are not part of the stable
interface (see ?nomologR).
Printed, the object shows the counts of cases and items by data
condition and how many decision-log entries are flagged. summary()
shows each item's review in a table, with the reason for each flag in a
Flagged section.
Details
The function never removes rows or items, changes scores, reverse-keys items, or decides whether a scale is valid.
Item-type labels are descriptive, not modeling decisions. In particular,
numeric_discrete means that the observed numeric values are integer-like
with 10 or fewer distinct observed values. It does not automatically mean
that the item should be treated as ordinal in later analyses.
A constant item has only one distinct observed value. An all_missing item
has no observed values. These are hard data conditions rather than
psychometric cutoff rules.
Response concentration and near-zero-variance flags use configurable
teaching references from nomo_defaults(). They are screening heuristics,
not psychometric laws or automatic item-retention rules. Ordered and
numeric-discrete items also receive descriptive boundary concentration
summaries, floor_prop and ceiling_prop. An ordered item's boundaries are
its first and last levels. A numeric-discrete item's are the ends of
scale_range when it is given, and otherwise its lowest and highest observed
values, since a numeric item shows only the values used: a pile-up in the
middle of a scale whose lower categories went unused would read as a floor
effect, and the decision log says so. Continuous-like numeric indicators
receive descriptive skewness and excess-kurtosis summaries without a
pass/fail normality judgment.
An item in a declared scale is reviewed on its item-rest correlation within that scale. When it has none there, because it is the only item of its scale in the diagnostics or too few cases are complete on the scale's items, it is not reviewed on the pooled value, which would judge it against other constructs; the decision log says why there is no value.
Careless responding. With effort = TRUE, each case receives the indices
Meade and Craig (2012), Huang et al. (2012), and Curran (2016) describe:
long-string, inter-item standard deviation (Marjanovic et al., 2015),
Mahalanobis distance, even-odd consistency, and psychometric antonym and
synonym correlations. Curran recommends these be used in series because each
has blind spots, and the output is built to show where they disagree rather
than to combine them into one score.
They disagree for a reason. Inter-item standard deviation detects random responding and gives a respondent who answers every item identically the best possible score; long-string detects exactly that respondent. When the two disagree about a case, the decision log says so.
A case is flagged only where a source states a rule, and each flag carries the source's own qualification: long-string at half the number of items, which Curran offers as a conservative starting point and says is not the best cut score for every scale, and a positive antonym or negative synonym correlation. The other indices have no stated cut score and are reported without a flag. Huang et al. found the indices they recommended identified attentive respondents well and random responders poorly, so an unflagged case is not thereby shown to be attentive.
Long-string reads the items in the order given: the order of items, or
of the columns of data when items is NULL, which should be the order
in which they were administered. A contentvalidR handoff of schema version
1 records no administration order, so its carried items are read in the
order the handoff lists them, which need not be the order of administration.
The long-string rule is applied only when at least
guidance$long_string_min_items items are screened (20 in
nomo_defaults()). Half the length of a shorter item set is a run of a few
responses, which attentive respondents give often: with two items every case
reaches it. On a shorter set each case's longest run is still reported in
long_string, no case is flagged on it, and the decision log states the
number of items and the share of cases that reached half the length.
Each respondent's antonym, synonym, and even-odd value is a correlation whose N is the number of pairs or scales. With two, every value is exactly +1 or -1, so at least three are required; with fewer than five the log says the flags are coarse. The same holds for each respondent: one who answered fewer than three of the pairs, or both halves of fewer than three scales, has no value. Even-odd consistency is Spearman-Brown corrected, and the correction has no meaning below -1, so the value is bounded there: it lies between -1 and 1. Cases are flagged, never removed.
Items from content review. items may be the handoff that
contentvalidR's content_handoff() produces after content review. Only
items it marks as carried are screened. Every item it held back is listed in
the decision log with its status and recommendation quoted in
contentvalidR's own words, and is never analyzed or reinstated here. The log
also records the producing version, workflow, and carry rule, and that the
items came from content review rather than from these data, as did their
construct membership when the review assigned one.
A carried item that is not a column of data is refused, never dropped, and
a handoff that carries no items is refused with the review's status counts.
Where the call leaves scales, reverse, or scale_range unset, the
handoff's scales and declared keying fill them. The log records what the
call changed: scales that replace the handoff's, a reverse or
scale_range that replaces the declared keying, and a scale_range that
completes keying declared without a response scale. An item is never
treated as forward keyed because keying was undeclared, and a response scale
the handoff did not record is never inferred. A handoff with a schema version
this release does not read is refused, naming both package versions. The
interface is specified in nomologR issue #46, and contentvalidR is not
needed to read it.
References
Curran, P. G. (2016). Methods for the detection of carelessly invalid responses in survey data. Journal of Experimental Social Psychology, 66, 4-19. doi:10.1016/j.jesp.2015.07.006
Huang, J. L., Curran, P. G., Keeney, J., Poposki, E. M., & DeShon, R. P. (2012). Detecting and deterring insufficient effort responding to surveys. Journal of Business and Psychology, 27(1), 99-114. doi:10.1007/s10869-011-9231-8
Marjanovic, Z., Holden, R., Struthers, W., Cribbie, R., & Greenglass, E. (2015). The inter-item standard deviation (ISD): An index that discriminates between conscientious and random responders. Personality and Individual Differences, 84, 79-83. doi:10.1016/j.paid.2014.08.021
Meade, A. W., & Craig, S. B. (2012). Identifying careless responses in survey data. Psychological Methods, 17(3), 437-455. doi:10.1037/a0028085
Clark, L. A., & Watson, D. (2019). Constructing validity: New developments in creating objective measuring instruments. Psychological Assessment, 31(12), 1412-1427. doi:10.1037/pas0000626
Kuhn, M., & Johnson, K. (2013). Applied predictive modeling. Springer. doi:10.1007/978-1-4614-6849-3
Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). McGraw-Hill.
Examples
dat <- data.frame(
item1 = c(1, 2, 3, 4, 5),
item2 = c(1, 2, NA, 4, 5),
item3 = c(3, 3, 3, 3, 3)
)
out <- nomo_screen(dat)
out$item_summary
#> # A tibble: 3 × 23
#> item storage item_type n n_observed n_missing pct_missing n_unique mode_n
#> <chr> <chr> <chr> <int> <int> <int> <dbl> <int> <int>
#> 1 item1 numeric numeric_… 5 5 0 0 5 1
#> 2 item2 numeric numeric_… 5 4 1 0.2 4 1
#> 3 item3 numeric binary 5 5 0 0 1 5
#> # ℹ 14 more variables: mode_prop <dbl>, min <dbl>, max <dbl>, mean <dbl>,
#> # sd <dbl>, constant <lgl>, all_missing <lgl>, percent_unique <dbl>,
#> # frequency_ratio <dbl>, near_zero_variance <lgl>, floor_prop <dbl>,
#> # ceiling_prop <dbl>, skewness <dbl>, excess_kurtosis <dbl>
out$decision_log
#> # A tibble: 5 × 10
#> stage object metric value reference severity observation recommendation
#> <chr> <chr> <chr> <dbl> <chr> <chr> <chr> <chr>
#> 1 screen item_select… all_c… 3 "" info Because `i… Verify that i…
#> 2 screen item1 item_… 5 "Descrip… info `item1` ha… Do not infer …
#> 3 screen item2 missi… 0.2 "Descrip… info `item2` ha… Inspect the p…
#> 4 screen item2 item_… 4 "Descrip… info `item2` ha… Do not infer …
#> 5 screen item3 const… 1 "At leas… concern `item3` ha… Inspect codin…
#> # ℹ 2 more variables: decision <chr>, rationale <chr>
# Simulated scale-development data with known teaching features
scr <- nomo_screen(nomo_demo_continuous)
summary(scr)
#> <nomo_screen summary> Item and data audit
#> Cases: 500 | Items: 10 | Item flags: 1 review, 0 concern
#> Items with missing responses: 2 | Constant: 0 | All missing: 0
#> Items in correlation diagnostics: 10
#>
#> Item review
#> Item Type Missing Top share Item-rest r Flag
#> a1 continuous 0.0% 1.0% .56
#> a2 continuous 3.0% 1.2% .58
#> a3 continuous 0.0% 1.0% .51
#> a4 continuous 0.0% 1.2% .55
#> a5 continuous 0.0% 1.6% .59
#> b1 continuous 0.0% 1.6% .60
#> b2 continuous 0.0% 1.4% .51
#> b3 continuous 2.4% 1.0% .57
#> b4 continuous 0.0% 1.2% .48
#> b5 continuous 0.0% 1.4% .28 Review
#> Top share is the share of observed responses in the most common category.
#> Item-rest r is the correlation of an item with the sum of the other items.
#>
#> Flagged
#> - b5 (Review): `b5` has a corrected item-rest correlation of r = .28
#> (n = 473), below the teaching reference of .30.
#>
#> Flags are review aids, not decisions to keep or delete an item.
#>
#> See nomo_table(x, "decision_log") for every log entry and plot(x) for the item
#> evidence map.