Does item format impact timing?

Effect of format on timing (Blocks 1 and 2 data)

We used a multilevel model, nesting log-seconds within participant to account for dependence. Our primary predictor was format. Here, we use only Blocks 1 and 2 as data. Results are depicted in Figure @ref(fig:timingmod1). The full distribution of timing (in log-seconds) is shown in Figure @ref(fig:timingdist). Tests of pairwise comparisons are shown in Table @ref(tab:pairwiseTab).

item_block12 = filter(items_df, block %in% c("1", "2")) %>% 
  filter(!is.infinite(seconds_log)) # this was added post pre-registration

mod.format_b1 = glmmTMB(seconds_log~format + (1|block) + (1|proid), 
                  data = item_block12)

tidy(aov(mod.format_b1))

## # A tibble: 4 × 6
##   term         df   sumsq  meansq statistic    p.value
##   <chr>     <dbl>   <dbl>   <dbl>     <dbl>      <dbl>
## 1 format        3   405.  135.        453.   1.16e-291
## 2 block         1    69.3  69.3       233.   1.70e- 52
## 3 proid       974  8030.    8.24       27.7  0        
## 4 Residuals 73111 21768.    0.298      NA   NA

Predictions by condition, using only Block 1 data. Figure A shows log seconds, Figure B shows raw seconds.

Distribution of time by category, blocks 1 and 2

Pairwise comparisons of timing (log-seconds) across format

contrast	estimate	std.error	statistic	p.value
Am Adjective - Adjective Only	0.02	0.01	2.63	.009
Am someone who tends to be Adjective - Adjective Only	0.22	0.01	34.40	< .001
Am someone who tends to be Adjective - Am Adjective	0.21	0.01	31.81	< .001
Am someone who tends to be Adjective - Tend to be Adjective	0.16	0.01	24.79	< .001
Tend to be Adjective - Adjective Only	0.06	0.01	9.67	< .001
Tend to be Adjective - Am Adjective	0.05	0.01	7.05	< .001

One model for each adjective

We can also repeat this analysis separately for each trait. Results are shown in Table @ref(tab:itemtable).

mod_by_item_b1 = item_block12 %>%
  group_by(item) %>%
  nest() %>%
  mutate(mod = map(data, ~lm(seconds_log~format, data = .))) %>%
  mutate(aov = map(mod, anova)) %>% 
  ungroup()

Format effects on log-seconds by item (blocks 1 and 2)

Item	Reverse Scored?	SS	MS	df	F	raw	adj
active	N	12.22	4.07	3	11.30	< .001	< .001
adventurous	N	13.86	4.62	3	11.83	< .001	< .001
broadminded	N	5.22	1.74	3	4.42	.004	.013
calm	N	12.22	4.07	3	9.76	< .001	< .001
caring	N	6.96	2.32	3	6.59	< .001	.002
cautious	N	4.35	1.45	3	3.38	.018	.018
cold	N	5.25	1.75	3	4.77	.003	.013
creative	N	10.68	3.56	3	9.67	< .001	< .001
curious	N	9.61	3.20	3	8.04	< .001	< .001
friendly	N	20.00	6.67	3	17.37	< .001	< .001
hardworking	N	11.34	3.78	3	10.12	< .001	< .001
helpful	N	29.68	9.89	3	28.79	< .001	< .001
imaginative	N	13.44	4.48	3	11.39	< .001	< .001
intelligent	N	11.47	3.82	3	10.45	< .001	< .001
lively	N	7.42	2.47	3	5.33	.001	.007
organized	N	21.24	7.08	3	17.85	< .001	< .001
outgoing	N	18.39	6.13	3	13.54	< .001	< .001
quiet	N	7.62	2.54	3	6.94	< .001	.001
relaxed	N	9.20	3.07	3	7.13	< .001	.001
responsible	N	24.42	8.14	3	18.75	< .001	< .001
selfdisciplined	N	13.97	4.66	3	10.62	< .001	< .001
shy	N	6.13	2.04	3	6.10	< .001	.003
softhearted	N	10.64	3.55	3	8.74	< .001	< .001
sophisticated	N	5.62	1.87	3	4.43	.004	.013
sympathetic	N	7.17	2.39	3	6.44	< .001	.002
talkative	N	9.25	3.08	3	8.38	< .001	< .001
thorough	N	11.86	3.95	3	9.55	< .001	< .001
thrifty	N	6.35	2.12	3	4.65	.003	.013
uncreative	N	9.65	3.22	3	9.63	< .001	< .001
unintellectual	N	12.55	4.18	3	10.63	< .001	< .001
unsympathetic	N	7.61	2.54	3	6.86	< .001	.001
warm	N	26.59	8.86	3	21.87	< .001	< .001
careless	Y	7.64	2.55	3	7.17	< .001	.001
impulsive	Y	9.27	3.09	3	7.98	< .001	< .001
moody	Y	19.62	6.54	3	19.76	< .001	< .001
nervous	Y	10.34	3.45	3	8.73	< .001	< .001
reckless	Y	19.53	6.51	3	18.85	< .001	< .001
worrying	Y	8.92	2.97	3	8.49	< .001	< .001

Pairwise t-tests for significant ANOVAs

Here we identify the specific items with significant differences.

sig_item_b1 = summary_by_item_b1 %>%
  filter(p.value < .05) 

sig_item_b1 = sig_item_b1$item
sig_item_b1

##  [1] "outgoing"        "helpful"         "reckless"        "moody"          
##  [5] "organized"       "friendly"        "warm"            "worrying"       
##  [9] "responsible"     "lively"          "caring"          "nervous"        
## [13] "creative"        "hardworking"     "imaginative"     "softhearted"    
## [17] "calm"            "selfdisciplined" "intelligent"     "curious"        
## [21] "active"          "careless"        "broadminded"     "impulsive"      
## [25] "sympathetic"     "cautious"        "talkative"       "sophisticated"  
## [29] "adventurous"     "thorough"        "thrifty"         "quiet"          
## [33] "unsympathetic"   "relaxed"         "uncreative"      "shy"            
## [37] "cold"            "unintellectual"

Then we create models for each adjective. We use the marginaleffects package to perform pairwise comparisons, again with a Holm correction on the p-values. We also plot the means and 95% confidence intervals of each mean.

adjective_timing = function(adjective){
  
  model = item_block12 %>%
    filter(item == adjective) %>%
    lm(seconds_log~format, data = .)
  
  comp = avg_comparisons(model, 
                         variables = list(format = "pairwise")) 
  comp$p.value = p.adjust(comp$p.value, method = "holm")
  
  comp = comp %>% 
    mutate(
    across( starts_with("p"), printp )) 
  
  caption = paste("Differences in log-seconds to", 
                  adjective,
                  "by format (blocks 1 and 2)")
  
  
  plot = avg_predictions(model, variables = "format") %>% 
    mutate(across(where(is.numeric), exp)) %>% 
    ggplot(aes(x = format, y = estimate)) +
    geom_point() +
    geom_errorbar(aes(ymin = conf.low, ymax = conf.high), width = .3) +
    labs(
      x = NULL, 
      y = "seconds", 
      title = paste0("Average response time to ", str_to_sentence(adjective))) +
    theme_pubclean()
  
  return(list(
    comp = comp,
    caption = caption,
    plot = plot
  ))
}

Active

Tests of the pairwise comparisons for this item are shown in Table @ref(tab:activepairs) and means are shown in Figure @ref(fig:activeplot).

active_model = adjective_timing("active")

Differences in log-seconds to active by format (blocks 1 and 2)

					95% CI
Contrast	Mean Diff	SE	z	p	low	high
Am Adjective - Adjective Only	0.02	0.04	0.55	.583	-0.05	0.10
Am someone who tends to be Adjective - Adjective Only	0.19	0.04	5.04	< .001	0.12	0.27
Am someone who tends to be Adjective - Am Adjective	0.17	0.04	4.47	< .001	0.10	0.25
Am someone who tends to be Adjective - Tend to be Adjective	0.06	0.04	1.68	.187	-0.01	0.14
Tend to be Adjective - Adjective Only	0.13	0.04	3.38	.003	0.05	0.20
Tend to be Adjective - Am Adjective	0.11	0.04	2.82	.015	0.03	0.18

Average seconds to respond to “active” by format (blocks 1 and 2).

Adventurous

Tests of the pairwise comparisons for this item are shown in Table @ref(tab:adventurouspairs) and means are shown in Figure @ref(fig:adventurousplot).

adventurous_model = adjective_timing("adventurous")

Differences in log-seconds to adventurous by format (blocks 1 and 2)

					95% CI
Contrast	Mean Diff	SE	z	p	low	high
Am Adjective - Adjective Only	0.05	0.04	1.26	.307	-0.03	0.13
Am someone who tends to be Adjective - Adjective Only	0.23	0.04	5.66	< .001	0.15	0.31
Am someone who tends to be Adjective - Am Adjective	0.18	0.04	4.39	< .001	0.10	0.25
Am someone who tends to be Adjective - Tend to be Adjective	0.12	0.04	2.98	.011	0.04	0.20
Tend to be Adjective - Adjective Only	0.11	0.04	2.69	.021	0.03	0.19
Tend to be Adjective - Am Adjective	0.06	0.04	1.43	.307	-0.02	0.14

Average seconds to respond to “adventurous” by format (blocks 1 and 2)

Broadminded

Tests of the pairwise comparisons for this item are shown in Table @ref(tab:broadmindedpairs) and means are shown in Figure @ref(fig:broadmindedplot).

broadminded_model = adjective_timing("broadminded")

Differences in log-seconds to broadminded by format (blocks 1 and 2)

					95% CI
Contrast	Mean Diff	SE	z	p	low	high
Am Adjective - Adjective Only	0.02	0.04	0.38	> .999	-0.06	0.09
Am someone who tends to be Adjective - Adjective Only	0.13	0.04	3.31	.006	0.05	0.21
Am someone who tends to be Adjective - Am Adjective	0.12	0.04	2.92	.017	0.04	0.20
Am someone who tends to be Adjective - Tend to be Adjective	0.09	0.04	2.36	.072	0.02	0.17
Tend to be Adjective - Adjective Only	0.04	0.04	0.95	> .999	-0.04	0.12
Tend to be Adjective - Am Adjective	0.02	0.04	0.57	> .999	-0.06	0.10

Average log-seconds to “broadminded” by format (blocks 1 and 2)

Calm

Tests of the pairwise comparisons for this item are shown in Table @ref(tab:calmpairs) and means are shown in Figure @ref(fig:calmplot).

calm_model = adjective_timing("calm")

Differences in log-seconds to calm by format (blocks 1 and 2)

					95% CI
Contrast	Mean Diff	SE	z	p	low	high
Am Adjective - Adjective Only	0.06	0.04	1.48	.278	-0.02	0.14
Am someone who tends to be Adjective - Adjective Only	0.22	0.04	5.21	< .001	0.13	0.30
Am someone who tends to be Adjective - Am Adjective	0.15	0.04	3.74	.001	0.07	0.24
Am someone who tends to be Adjective - Tend to be Adjective	0.14	0.04	3.51	.002	0.06	0.23
Tend to be Adjective - Adjective Only	0.07	0.04	1.72	.258	-0.01	0.15
Tend to be Adjective - Am Adjective	0.01	0.04	0.24	.814	-0.07	0.09

Average log-seconds to “calm” by format (blocks 1 and 2)

Caring

Tests of the pairwise comparisons for this item are shown in Table @ref(tab:caringpairs) and means are shown in Figure @ref(fig:caringplot).

caring_model = adjective_timing("caring")

Differences in log-seconds to caring by format (blocks 1 and 2)

					95% CI
Contrast	Mean Diff	SE	z	p	low	high
Am Adjective - Adjective Only	0.00	0.04	-0.13	.897	-0.08	0.07
Am someone who tends to be Adjective - Adjective Only	0.14	0.04	3.79	.001	0.07	0.22
Am someone who tends to be Adjective - Am Adjective	0.15	0.04	3.91	.001	0.07	0.22
Am someone who tends to be Adjective - Tend to be Adjective	0.10	0.04	2.60	.038	0.02	0.17
Tend to be Adjective - Adjective Only	0.05	0.04	1.20	.552	-0.03	0.12
Tend to be Adjective - Am Adjective	0.05	0.04	1.33	.552	-0.02	0.12

Average log-seconds to “caring” by format (blocks 1 and 2)

Cautious

Tests of the pairwise comparisons for this item are shown in Table @ref(tab:helpfulpairs) and means are shown in Figure @ref(fig:helpfulplot).

cautious_model = adjective_timing("cautious")

Differences in log-seconds to cautious by format (blocks 1 and 2)

					95% CI
Contrast	Mean Diff	SE	z	p	low	high
Am Adjective - Adjective Only	0.02	0.04	0.57	> .999	-0.06	0.11
Am someone who tends to be Adjective - Adjective Only	0.12	0.04	2.97	.018	0.04	0.21
Am someone who tends to be Adjective - Am Adjective	0.10	0.04	2.39	.083	0.02	0.18
Am someone who tends to be Adjective - Tend to be Adjective	0.09	0.04	2.14	.130	0.01	0.17
Tend to be Adjective - Adjective Only	0.04	0.04	0.84	> .999	-0.05	0.12
Tend to be Adjective - Am Adjective	0.01	0.04	0.27	> .999	-0.07	0.09

Average log-seconds to “cautious” by format (blocks 1 and 2)

Cold

Tests of the pairwise comparisons for this item are shown in Table @ref(tab:coldpairs) and means are shown in Figure @ref(fig:coldplot).

cold_model = adjective_timing("cold")

Differences in log-seconds to cold by format (blocks 1 and 2)

					95% CI
Contrast	Mean Diff	SE	z	p	low	high
Am Adjective - Adjective Only	0.06	0.04	1.62	.314	-0.01	0.14
Am someone who tends to be Adjective - Adjective Only	0.14	0.04	3.64	.002	0.07	0.22
Am someone who tends to be Adjective - Am Adjective	0.08	0.04	2.00	.181	0.00	0.15
Am someone who tends to be Adjective - Tend to be Adjective	0.10	0.04	2.70	.035	0.03	0.18
Tend to be Adjective - Adjective Only	0.04	0.04	0.95	.687	-0.04	0.11
Tend to be Adjective - Am Adjective	-0.03	0.04	-0.68	.687	-0.10	0.05

Average log-seconds to “cold” by format (blocks 1 and 2)

Creative

Tests of the pairwise comparisons for this item are shown in Table @ref(tab:creativepairs) and means are shown in Figure @ref(fig:creativeplot).

creative_model = adjective_timing("creative")

Differences in log-seconds to creative by format (blocks 1 and 2)

					95% CI
Contrast	Mean Diff	SE	z	p	low	high
Am Adjective - Adjective Only	0.06	0.04	1.42	.309	-0.02	0.13
Am someone who tends to be Adjective - Adjective Only	0.20	0.04	5.18	< .001	0.13	0.28
Am someone who tends to be Adjective - Am Adjective	0.15	0.04	3.74	.001	0.07	0.22
Am someone who tends to be Adjective - Tend to be Adjective	0.13	0.04	3.44	.002	0.06	0.21
Tend to be Adjective - Adjective Only	0.07	0.04	1.76	.235	-0.01	0.14
Tend to be Adjective - Am Adjective	0.01	0.04	0.33	.744	-0.06	0.09

Average log-seconds to “creative” by format (blocks 1 and 2)

Curious

Tests of the pairwise comparisons for this item are shown in Table @ref(tab:curiouspairs) and means are shown in Figure @ref(fig:curiousplot).

curious_model = adjective_timing("curious")

Differences in log-seconds to curious by format (blocks 1 and 2)

					95% CI
Contrast	Mean Diff	SE	z	p	low	high
Am Adjective - Adjective Only	0.03	0.04	0.67	> .999	-0.05	0.11
Am someone who tends to be Adjective - Adjective Only	0.18	0.04	4.52	< .001	0.10	0.26
Am someone who tends to be Adjective - Am Adjective	0.16	0.04	3.85	.001	0.08	0.24
Am someone who tends to be Adjective - Tend to be Adjective	0.13	0.04	3.20	.006	0.05	0.21
Tend to be Adjective - Adjective Only	0.05	0.04	1.34	.537	-0.02	0.13
Tend to be Adjective - Am Adjective	0.03	0.04	0.67	> .999	-0.05	0.11

Average log-seconds to “curious” by format (blocks 1 and 2)

Friendly

Tests of the pairwise comparisons for this item are shown in Table @ref(tab:friendlypairs) and means are shown in Figure @ref(fig:friendlyplot).

friendly_model = adjective_timing("friendly")

Differences in log-seconds to friendly by format (blocks 1 and 2)

					95% CI
Contrast	Mean Diff	SE	z	p	low	high
Am Adjective - Adjective Only	0.02	0.04	0.61	> .999	-0.05	0.10
Am someone who tends to be Adjective - Adjective Only	0.25	0.04	6.32	< .001	0.17	0.33
Am someone who tends to be Adjective - Am Adjective	0.23	0.04	5.71	< .001	0.15	0.30
Am someone who tends to be Adjective - Tend to be Adjective	0.22	0.04	5.50	< .001	0.14	0.29
Tend to be Adjective - Adjective Only	0.03	0.04	0.84	> .999	-0.04	0.11
Tend to be Adjective - Am Adjective	0.01	0.04	0.23	> .999	-0.07	0.09

Average log-seconds to “friendly” by format (blocks 1 and 2)

Hardworking

Tests of the pairwise comparisons for this item are shown in Table @ref(tab:hardworkingpairs) and means are shown in Figure @ref(fig:hardworkingplot).

hardworking_model = adjective_timing("hardworking")

Differences in log-seconds to hardworking by format (blocks 1 and 2)

					95% CI
Contrast	Mean Diff	SE	z	p	low	high
Am Adjective - Adjective Only	-0.02	0.04	-0.57	> .999	-0.10	0.05
Am someone who tends to be Adjective - Adjective Only	0.17	0.04	4.42	< .001	0.10	0.25
Am someone who tends to be Adjective - Am Adjective	0.20	0.04	4.97	< .001	0.12	0.27
Am someone who tends to be Adjective - Tend to be Adjective	0.15	0.04	3.79	.001	0.07	0.22
Tend to be Adjective - Adjective Only	0.02	0.04	0.63	> .999	-0.05	0.10
Tend to be Adjective - Am Adjective	0.05	0.04	1.20	.695	-0.03	0.12

Average log-seconds to “hardworking” by format (blocks 1 and 2)

Helpful

Tests of the pairwise comparisons for this item are shown in Table @ref(tab:helpfulpairs) and means are shown in Figure @ref(fig:helpfulplot).

helpful_model = adjective_timing("helpful")

Differences in log-seconds to helpful by format (blocks 1 and 2)

					95% CI
Contrast	Mean Diff	SE	z	p	low	high
Am Adjective - Adjective Only	0.05	0.04	1.38	.169	-0.02	0.13
Am someone who tends to be Adjective - Adjective Only	0.33	0.04	8.65	< .001	0.25	0.40
Am someone who tends to be Adjective - Am Adjective	0.27	0.04	7.25	< .001	0.20	0.35
Am someone who tends to be Adjective - Tend to be Adjective	0.19	0.04	5.11	< .001	0.12	0.26
Tend to be Adjective - Adjective Only	0.13	0.04	3.58	.001	0.06	0.21
Tend to be Adjective - Am Adjective	0.08	0.04	2.18	.058	0.01	0.16

Average log-seconds to “helpful” by format (blocks 1 and 2)

Imaginative

Tests of the pairwise comparisons for this item are shown in Table @ref(tab:imaginativepairs) and means are shown in Figure @ref(fig:imaginativeplot).

imaginative_model = adjective_timing("imaginative")

Differences in log-seconds to imaginative by format (blocks 1 and 2)

					95% CI
Contrast	Mean Diff	SE	z	p	low	high
Am Adjective - Adjective Only	0.05	0.04	1.31	.379	-0.03	0.13
Am someone who tends to be Adjective - Adjective Only	0.22	0.04	5.59	< .001	0.15	0.30
Am someone who tends to be Adjective - Am Adjective	0.17	0.04	4.28	< .001	0.09	0.25
Am someone who tends to be Adjective - Tend to be Adjective	0.13	0.04	3.33	.003	0.06	0.21
Tend to be Adjective - Adjective Only	0.09	0.04	2.27	.069	0.01	0.17
Tend to be Adjective - Am Adjective	0.04	0.04	0.96	.379	-0.04	0.12

Average log-seconds to “imaginative” by format (blocks 1 and 2)

Intelligent

Tests of the pairwise comparisons for this item are shown in Table @ref(tab:intelligentpairs) and means are shown in Figure @ref(fig:intelligentplot).

intelligent_model = adjective_timing("intelligent")

Differences in log-seconds to intelligent by format (blocks 1 and 2)

					95% CI
Contrast	Mean Diff	SE	z	p	low	high
Am Adjective - Adjective Only	0.07	0.04	1.81	.141	-0.01	0.15
Am someone who tends to be Adjective - Adjective Only	0.21	0.04	5.48	< .001	0.14	0.29
Am someone who tends to be Adjective - Am Adjective	0.14	0.04	3.66	.001	0.07	0.22
Am someone who tends to be Adjective - Tend to be Adjective	0.10	0.04	2.72	.021	0.03	0.18
Tend to be Adjective - Adjective Only	0.11	0.04	2.79	.021	0.03	0.18
Tend to be Adjective - Am Adjective	0.04	0.04	0.96	.336	-0.04	0.11

Average log-seconds to “intelligent” by format (blocks 1 and 2)

Lively

Tests of the pairwise comparisons for this item are shown in Table @ref(tab:livelypairs) and means are shown in Figure @ref(fig:livelyplot).

lively_model = adjective_timing("lively")

Differences in log-seconds to lively by format (blocks 1 and 2)

					95% CI
Contrast	Mean Diff	SE	z	p	low	high
Am Adjective - Adjective Only	0.04	0.04	0.86	.785	-0.05	0.12
Am someone who tends to be Adjective - Adjective Only	0.17	0.04	3.81	.001	0.08	0.25
Am someone who tends to be Adjective - Am Adjective	0.13	0.04	2.95	.016	0.04	0.21
Am someone who tends to be Adjective - Tend to be Adjective	0.10	0.04	2.25	.099	0.01	0.18
Tend to be Adjective - Adjective Only	0.07	0.04	1.57	.351	-0.02	0.15
Tend to be Adjective - Am Adjective	0.03	0.04	0.71	.785	-0.05	0.12

Average log-seconds to “lively” by format (blocks 1 and 2)

Organized

Tests of the pairwise comparisons for this item are shown in Table @ref(tab:organizedpairs) and means are shown in Figure @ref(fig:organizedplot).

organized_model = adjective_timing("organized")

Differences in log-seconds to organized by format (blocks 1 and 2)

					95% CI
Contrast	Mean Diff	SE	z	p	low	high
Am Adjective - Adjective Only	0.05	0.04	1.28	.403	-0.03	0.13
Am someone who tends to be Adjective - Adjective Only	0.28	0.04	6.83	< .001	0.20	0.35
Am someone who tends to be Adjective - Am Adjective	0.22	0.04	5.53	< .001	0.14	0.30
Am someone who tends to be Adjective - Tend to be Adjective	0.19	0.04	4.85	< .001	0.12	0.27
Tend to be Adjective - Adjective Only	0.08	0.04	1.99	.140	0.00	0.16
Tend to be Adjective - Am Adjective	0.03	0.04	0.71	.480	-0.05	0.11

Average log-seconds to “organized” by format (blocks 1 and 2)

Outgoing

Tests of the pairwise comparisons for this item are shown in Table @ref(tab:outgoingpairs) and means are shown in Figure @ref(fig:outgoingplot).

outgoing_model = adjective_timing("outgoing")

Differences in log-seconds to outgoing by format (blocks 1 and 2)

					95% CI
Contrast	Mean Diff	SE	z	p	low	high
Am Adjective - Adjective Only	0.01	0.04	0.18	.861	-0.08	0.09
Am someone who tends to be Adjective - Adjective Only	0.24	0.04	5.60	< .001	0.16	0.33
Am someone who tends to be Adjective - Am Adjective	0.23	0.04	5.41	< .001	0.15	0.32
Am someone who tends to be Adjective - Tend to be Adjective	0.17	0.04	3.91	< .001	0.08	0.25
Tend to be Adjective - Adjective Only	0.07	0.04	1.71	.264	-0.01	0.16
Tend to be Adjective - Am Adjective	0.07	0.04	1.53	.264	-0.02	0.15

Average log-seconds to “outgoing” by format (blocks 1 and 2)

Quiet

Tests of the pairwise comparisons for this item are shown in Table @ref(tab:quietpairs) and means are shown in Figure @ref(fig:quietplot).

quiet_model = adjective_timing("quiet")

Differences in log-seconds to quiet by format (blocks 1 and 2)

					95% CI
Contrast	Mean Diff	SE	z	p	low	high
Am Adjective - Adjective Only	0.04	0.04	1.11	.796	-0.03	0.12
Am someone who tends to be Adjective - Adjective Only	0.16	0.04	4.24	< .001	0.09	0.24
Am someone who tends to be Adjective - Am Adjective	0.12	0.04	3.12	.007	0.04	0.20
Am someone who tends to be Adjective - Tend to be Adjective	0.13	0.04	3.44	.003	0.06	0.21
Tend to be Adjective - Adjective Only	0.03	0.04	0.81	.841	-0.04	0.11
Tend to be Adjective - Am Adjective	-0.01	0.04	-0.31	.841	-0.09	0.06

Average log-seconds to “quiet” by format (blocks 1 and 2)

Relaxed

Tests of the pairwise comparisons for this item are shown in Table @ref(tab:relaxedpairs) and means are shown in Figure @ref(fig:relaxedplot).

relaxed_model = adjective_timing("relaxed")

Differences in log-seconds to relaxed by format (blocks 1 and 2)

					95% CI
Contrast	Mean Diff	SE	z	p	low	high
Am Adjective - Adjective Only	0.09	0.04	2.08	.113	0.01	0.17
Am someone who tends to be Adjective - Adjective Only	0.19	0.04	4.58	< .001	0.11	0.27
Am someone who tends to be Adjective - Am Adjective	0.10	0.04	2.48	.052	0.02	0.19
Am someone who tends to be Adjective - Tend to be Adjective	0.12	0.04	2.87	.021	0.04	0.20
Tend to be Adjective - Adjective Only	0.07	0.04	1.71	.173	-0.01	0.15
Tend to be Adjective - Am Adjective	-0.02	0.04	-0.37	.709	-0.10	0.07

Average log-seconds to “relaxed” by format (blocks 1 and 2)

Responsible

Tests of the pairwise comparisons for this item are shown in Table @ref(tab:responsiblepairs) and means are shown in Figure @ref(fig:responsibleplot).

responsible_model = adjective_timing("responsible")

Differences in log-seconds to responsible by format (blocks 1 and 2)

					95% CI
Contrast	Mean Diff	SE	z	p	low	high
Am Adjective - Adjective Only	0.00	0.04	0.01	.992	-0.08	0.08
Am someone who tends to be Adjective - Adjective Only	0.27	0.04	6.43	< .001	0.19	0.35
Am someone who tends to be Adjective - Am Adjective	0.27	0.04	6.41	< .001	0.19	0.35
Am someone who tends to be Adjective - Tend to be Adjective	0.22	0.04	5.13	< .001	0.13	0.30
Tend to be Adjective - Adjective Only	0.06	0.04	1.32	.562	-0.03	0.14
Tend to be Adjective - Am Adjective	0.06	0.04	1.31	.562	-0.03	0.14

Average log-seconds to “responsible” by format (blocks 1 and 2)

Self-disciplined

Tests of the pairwise comparisons for this item are shown in Table @ref(tab:selfdisciplinedpairs) and means are shown in Figure @ref(fig:selfdisciplinedplot).

selfdisciplined_model = adjective_timing("selfdisciplined")

Differences in log-seconds to selfdisciplined by format (blocks 1 and 2)

					95% CI
Contrast	Mean Diff	SE	z	p	low	high
Am Adjective - Adjective Only	0.08	0.04	1.92	.110	0.00	0.17
Am someone who tends to be Adjective - Adjective Only	0.24	0.04	5.55	< .001	0.15	0.32
Am someone who tends to be Adjective - Am Adjective	0.15	0.04	3.61	.002	0.07	0.24
Am someone who tends to be Adjective - Tend to be Adjective	0.14	0.04	3.33	.004	0.06	0.22
Tend to be Adjective - Adjective Only	0.10	0.04	2.25	.074	0.01	0.18
Tend to be Adjective - Am Adjective	0.01	0.04	0.31	.756	-0.07	0.10

Average log-seconds to “selfdisciplined” by format (blocks 1 and 2)

Shy

Tests of the pairwise comparisons for this item are shown in Table @ref(tab:shypairs) and means are shown in Figure @ref(fig:shyplot).

shy_model = adjective_timing("shy")

Differences in log-seconds to shy by format (blocks 1 and 2)

					95% CI
Contrast	Mean Diff	SE	z	p	low	high
Am Adjective - Adjective Only	0.05	0.04	1.33	.370	-0.02	0.12
Am someone who tends to be Adjective - Adjective Only	0.13	0.04	3.54	.002	0.06	0.20
Am someone who tends to be Adjective - Am Adjective	0.08	0.04	2.20	.111	0.01	0.15
Am someone who tends to be Adjective - Tend to be Adjective	0.14	0.04	3.83	.001	0.07	0.21
Tend to be Adjective - Adjective Only	-0.01	0.04	-0.28	.780	-0.08	0.06
Tend to be Adjective - Am Adjective	-0.06	0.04	-1.61	.323	-0.13	0.01

Average log-seconds to “shy” by format (blocks 1 and 2)

Soft-hearted

Tests of the pairwise comparisons for this item are shown in Table @ref(tab:softpairs) and means are shown in Figure @ref(fig:softplot).

softhearted_model = adjective_timing("softhearted")

Differences in log-seconds to softhearted by format (blocks 1 and 2)

					95% CI
Contrast	Mean Diff	SE	z	p	low	high
Am Adjective - Adjective Only	-0.04	0.04	-1.01	.622	-0.12	0.04
Am someone who tends to be Adjective - Adjective Only	0.16	0.04	3.84	.001	0.08	0.24
Am someone who tends to be Adjective - Am Adjective	0.20	0.04	4.84	< .001	0.12	0.28
Am someone who tends to be Adjective - Tend to be Adjective	0.13	0.04	3.16	.006	0.05	0.21
Tend to be Adjective - Adjective Only	0.03	0.04	0.68	.622	-0.05	0.11
Tend to be Adjective - Am Adjective	0.07	0.04	1.69	.271	-0.01	0.15

Average log-seconds to “softhearted” by format (blocks 1 and 2)

Sophisticated

Tests of the pairwise comparisons for this item are shown in Table @ref(tab:sophisticatedpairs) and means are shown in Figure @ref(fig:sophisticatedplot).

sophisticated_model = adjective_timing("sophisticated")

Differences in log-seconds to sophisticated by format (blocks 1 and 2)

					95% CI
Contrast	Mean Diff	SE	z	p	low	high
Am Adjective - Adjective Only	0.06	0.04	1.53	.382	-0.02	0.15
Am someone who tends to be Adjective - Adjective Only	0.14	0.04	3.44	.004	0.06	0.22
Am someone who tends to be Adjective - Am Adjective	0.08	0.04	1.91	.226	0.00	0.16
Am someone who tends to be Adjective - Tend to be Adjective	0.03	0.04	0.80	.537	-0.05	0.11
Tend to be Adjective - Adjective Only	0.11	0.04	2.64	.042	0.03	0.19
Tend to be Adjective - Am Adjective	0.05	0.04	1.11	.537	-0.04	0.13

Average log-seconds to “sophisticated” by format (blocks 1 and 2)

Sympathetic

Tests of the pairwise comparisons for this item are shown in Table @ref(tab:symppairs) and means are shown in Figure @ref(fig:sympplot).

sympathetic_model = adjective_timing("sympathetic")

Differences in log-seconds to sympathetic by format (blocks 1 and 2)

					95% CI
Contrast	Mean Diff	SE	z	p	low	high
Am Adjective - Adjective Only	0.01	0.04	0.19	.851	-0.07	0.08
Am someone who tends to be Adjective - Adjective Only	0.15	0.04	3.80	.001	0.07	0.23
Am someone who tends to be Adjective - Am Adjective	0.14	0.04	3.62	.001	0.06	0.22
Am someone who tends to be Adjective - Tend to be Adjective	0.07	0.04	1.70	.177	-0.01	0.14
Tend to be Adjective - Adjective Only	0.08	0.04	2.12	.137	0.01	0.16
Tend to be Adjective - Am Adjective	0.08	0.04	1.93	.160	0.00	0.15

Average log-seconds to “sympathetic” by format (blocks 1 and 2)

Talkative

Tests of the pairwise comparisons for this item are shown in Table @ref(tab:talkativepairs) and means are shown in Figure @ref(fig:talkativeplot).

talkative_model = adjective_timing("talkative")

Differences in log-seconds to talkative by format (blocks 1 and 2)

					95% CI
Contrast	Mean Diff	SE	z	p	low	high
Am Adjective - Adjective Only	0.01	0.04	0.33	.740	-0.06	0.09
Am someone who tends to be Adjective - Adjective Only	0.17	0.04	4.50	< .001	0.10	0.25
Am someone who tends to be Adjective - Am Adjective	0.16	0.04	4.16	< .001	0.09	0.24
Am someone who tends to be Adjective - Tend to be Adjective	0.10	0.04	2.70	.028	0.03	0.18
Tend to be Adjective - Adjective Only	0.07	0.04	1.80	.214	-0.01	0.15
Tend to be Adjective - Am Adjective	0.06	0.04	1.47	.283	-0.02	0.13

Average log-seconds to “talkative” by format (blocks 1 and 2)

Thorough

Tests of the pairwise comparisons for this item are shown in Table @ref(tab:thoroughpairs) and means are shown in Figure @ref(fig:thoroughplot).

thorough_model = adjective_timing("thorough")

Differences in log-seconds to thorough by format (blocks 1 and 2)

					95% CI
Contrast	Mean Diff	SE	z	p	low	high
Am Adjective - Adjective Only	0.04	0.04	0.94	.693	-0.04	0.12
Am someone who tends to be Adjective - Adjective Only	0.21	0.04	5.03	< .001	0.13	0.29
Am someone who tends to be Adjective - Am Adjective	0.17	0.04	4.07	< .001	0.09	0.25
Am someone who tends to be Adjective - Tend to be Adjective	0.14	0.04	3.31	.004	0.06	0.22
Tend to be Adjective - Adjective Only	0.07	0.04	1.72	.256	-0.01	0.15
Tend to be Adjective - Am Adjective	0.03	0.04	0.77	.693	-0.05	0.11

Average log-seconds to “thorough” by format (blocks 1 and 2)

Thrifty

Tests of the pairwise comparisons for this item are shown in Table @ref(tab:thriftypairs) and means are shown in Figure @ref(fig:thriftyplot).

thrifty_model = adjective_timing("thrifty")

Differences in log-seconds to thrifty by format (blocks 1 and 2)

					95% CI
Contrast	Mean Diff	SE	z	p	low	high
Am Adjective - Adjective Only	0.09	0.04	2.07	.156	0.00	0.18
Am someone who tends to be Adjective - Adjective Only	0.16	0.04	3.66	.001	0.07	0.24
Am someone who tends to be Adjective - Am Adjective	0.07	0.04	1.57	.350	-0.02	0.15
Am someone who tends to be Adjective - Tend to be Adjective	0.10	0.04	2.30	.107	0.01	0.18
Tend to be Adjective - Adjective Only	0.06	0.04	1.38	.350	-0.02	0.14
Tend to be Adjective - Am Adjective	-0.03	0.04	-0.70	.482	-0.12	0.05

Average log-seconds to “thrifty” by format (blocks 1 and 2)

Uncreative

Tests of the pairwise comparisons for this item are shown in Table @ref(tab:uncreativepairs) and means are shown in Figure @ref(fig:uncreativeplot).

uncreative_model = adjective_timing("uncreative")

Differences in log-seconds to uncreative by format (blocks 1 and 2)

					95% CI
Contrast	Mean Diff	SE	z	p	low	high
Am Adjective - Adjective Only	0.08	0.04	2.22	.053	0.01	0.16
Am someone who tends to be Adjective - Adjective Only	0.20	0.04	5.33	< .001	0.12	0.27
Am someone who tends to be Adjective - Am Adjective	0.11	0.04	3.09	.010	0.04	0.19
Am someone who tends to be Adjective - Tend to be Adjective	0.09	0.04	2.40	.050	0.02	0.16
Tend to be Adjective - Adjective Only	0.11	0.04	2.95	.013	0.04	0.18
Tend to be Adjective - Am Adjective	0.03	0.04	0.71	.477	-0.05	0.10

Average log-seconds to “uncreative” by format (blocks 1 and 2)

Unintellectual

Tests of the pairwise comparisons for this item are shown in Table @ref(tab:unintellectualpairs) and means are shown in Figure @ref(fig:unintellectualplot).

unintellectual_model = adjective_timing("unintellectual")

Differences in log-seconds to unintellectual by format (blocks 1 and 2)

					95% CI
Contrast	Mean Diff	SE	z	p	low	high
Am Adjective - Adjective Only	0.14	0.04	3.58	.002	0.07	0.22
Am someone who tends to be Adjective - Adjective Only	0.22	0.04	5.56	< .001	0.14	0.30
Am someone who tends to be Adjective - Am Adjective	0.08	0.04	1.96	.099	0.00	0.16
Am someone who tends to be Adjective - Tend to be Adjective	0.09	0.04	2.19	.085	0.01	0.17
Tend to be Adjective - Adjective Only	0.14	0.04	3.37	.003	0.06	0.21
Tend to be Adjective - Am Adjective	-0.01	0.04	-0.22	.823	-0.09	0.07

Average log-seconds to “unintellectual” by format (blocks 1 and 2)

Unsympathetic

Tests of the pairwise comparisons for this item are shown in Table @ref(tab:unsympatheticpairs) and means are shown in Figure @ref(fig:unsympatheticplot).

unsympathetic_model = adjective_timing("unsympathetic")

Differences in log-seconds to unsympathetic by format (blocks 1 and 2)

					95% CI
Contrast	Mean Diff	SE	z	p	low	high
Am Adjective - Adjective Only	0.04	0.04	1.02	.619	-0.04	0.12
Am someone who tends to be Adjective - Adjective Only	0.17	0.04	4.30	< .001	0.09	0.24
Am someone who tends to be Adjective - Am Adjective	0.13	0.04	3.29	.005	0.05	0.20
Am someone who tends to be Adjective - Tend to be Adjective	0.12	0.04	2.99	.011	0.04	0.19
Tend to be Adjective - Adjective Only	0.05	0.04	1.33	.549	-0.02	0.13
Tend to be Adjective - Am Adjective	0.01	0.04	0.31	.755	-0.06	0.09

Average log-seconds to “unsympathetic” by format (blocks 1 and 2)

Warm

Tests of the pairwise comparisons for this item are shown in Table @ref(tab:warmpairs) and means are shown in Figure @ref(fig:warmplot).

warm_model = adjective_timing("warm")

Differences in log-seconds to warm by format (blocks 1 and 2)

					95% CI
Contrast	Mean Diff	SE	z	p	low	high
Am Adjective - Adjective Only	0.15	0.04	3.55	.001	0.07	0.23
Am someone who tends to be Adjective - Adjective Only	0.33	0.04	8.06	< .001	0.25	0.41
Am someone who tends to be Adjective - Am Adjective	0.18	0.04	4.47	< .001	0.10	0.26
Am someone who tends to be Adjective - Tend to be Adjective	0.15	0.04	3.70	.001	0.07	0.23
Tend to be Adjective - Adjective Only	0.18	0.04	4.37	< .001	0.10	0.26
Tend to be Adjective - Am Adjective	0.03	0.04	0.80	.426	-0.05	0.11

Average log-seconds to “warm” by format (blocks 1 and 2)

Careless

Tests of the pairwise comparisons for this item are shown in Table @ref(tab:carelesspairs) and means are shown in Figure @ref(fig:carelessplot).

careless_model = adjective_timing("careless")

Differences in log-seconds to careless by format (blocks 1 and 2)

					95% CI
Contrast	Mean Diff	SE	z	p	low	high
Am Adjective - Adjective Only	0.05	0.04	1.28	.604	-0.03	0.12
Am someone who tends to be Adjective - Adjective Only	0.17	0.04	4.39	< .001	0.09	0.24
Am someone who tends to be Adjective - Am Adjective	0.12	0.04	3.11	.007	0.04	0.19
Am someone who tends to be Adjective - Tend to be Adjective	0.13	0.04	3.34	.004	0.05	0.20
Tend to be Adjective - Adjective Only	0.04	0.04	1.06	.604	-0.03	0.12
Tend to be Adjective - Am Adjective	-0.01	0.04	-0.22	.827	-0.08	0.07

Average log-seconds to “careless” by format (blocks 1 and 2)

Impulsive

Tests of the pairwise comparisons for this item are shown in Table @ref(tab:impulsivepairs) and means are shown in Figure @ref(fig:impulsiveplot).

impulsive_model = adjective_timing("impulsive")

Differences in log-seconds to impulsive by format (blocks 1 and 2)

					95% CI
Contrast	Mean Diff	SE	z	p	low	high
Am Adjective - Adjective Only	0.01	0.04	0.13	.900	-0.07	0.08
Am someone who tends to be Adjective - Adjective Only	0.17	0.04	4.28	< .001	0.09	0.25
Am someone who tends to be Adjective - Am Adjective	0.17	0.04	4.15	< .001	0.09	0.24
Am someone who tends to be Adjective - Tend to be Adjective	0.13	0.04	3.19	.006	0.05	0.20
Tend to be Adjective - Adjective Only	0.04	0.04	1.10	.812	-0.03	0.12
Tend to be Adjective - Am Adjective	0.04	0.04	0.97	.812	-0.04	0.12

Average log-seconds to “impulsive” by format (blocks 1 and 2)

Moody

Tests of the pairwise comparisons for this item are shown in Table @ref(tab:moodypairs) and means are shown in Figure @ref(fig:moodyplot).

moody_model = adjective_timing("moody")

Differences in log-seconds to moody by format (blocks 1 and 2)

					95% CI
Contrast	Mean Diff	SE	z	p	low	high
Am Adjective - Adjective Only	0.02	0.04	0.62	.618	-0.05	0.10
Am someone who tends to be Adjective - Adjective Only	0.25	0.04	6.89	< .001	0.18	0.33
Am someone who tends to be Adjective - Am Adjective	0.23	0.04	6.25	< .001	0.16	0.30
Am someone who tends to be Adjective - Tend to be Adjective	0.19	0.04	5.26	< .001	0.12	0.26
Tend to be Adjective - Adjective Only	0.06	0.04	1.64	.303	-0.01	0.13
Tend to be Adjective - Am Adjective	0.04	0.04	1.02	.618	-0.03	0.11

Average log-seconds to “moody” by format (blocks 1 and 2)

Nervous

Tests of the pairwise comparisons for this item are shown in Table @ref(tab:nervouspairs) and means are shown in Figure @ref(fig:nervousplot).

nervous_model = adjective_timing("nervous")

Differences in log-seconds to nervous by format (blocks 1 and 2)

					95% CI
Contrast	Mean Diff	SE	z	p	low	high
Am Adjective - Adjective Only	-0.01	0.04	-0.32	> .999	-0.09	0.07
Am someone who tends to be Adjective - Adjective Only	0.16	0.04	4.00	< .001	0.08	0.24
Am someone who tends to be Adjective - Am Adjective	0.17	0.04	4.31	< .001	0.09	0.25
Am someone who tends to be Adjective - Tend to be Adjective	0.17	0.04	4.20	< .001	0.09	0.25
Tend to be Adjective - Adjective Only	-0.01	0.04	-0.19	> .999	-0.09	0.07
Tend to be Adjective - Am Adjective	0.01	0.04	0.13	> .999	-0.07	0.08

Average log-seconds to “nervous” by format (blocks 1 and 2)

Reckless

Tests of the pairwise comparisons for this item are shown in Table @ref(tab:recklesspairs) and means are shown in Figure @ref(fig:recklessplot).

reckless_model = adjective_timing("reckless")

Differences in log-seconds to reckless by format (blocks 1 and 2)

					95% CI
Contrast	Mean Diff	SE	z	p	low	high
Am Adjective - Adjective Only	-0.01	0.04	-0.22	> .999	-0.08	0.07
Am someone who tends to be Adjective - Adjective Only	0.23	0.04	6.08	< .001	0.16	0.30
Am someone who tends to be Adjective - Am Adjective	0.24	0.04	6.30	< .001	0.16	0.31
Am someone who tends to be Adjective - Tend to be Adjective	0.23	0.04	6.02	< .001	0.15	0.30
Tend to be Adjective - Adjective Only	0.00	0.04	0.07	> .999	-0.07	0.08
Tend to be Adjective - Am Adjective	0.01	0.04	0.30	> .999	-0.06	0.08

Average log-seconds to “reckless” by format (blocks 1 and 2)

Worrying

Tests of the pairwise comparisons for this item are shown in Table @ref(tab:worryingpairs) and means are shown in Figure @ref(fig:worryingplot).

worrying_model = adjective_timing("worrying")

Differences in log-seconds to worrying by format (blocks 1 and 2)

					95% CI
Contrast	Mean Diff	SE	z	p	low	high
Am Adjective - Adjective Only	0.04	0.04	1.09	.604	-0.03	0.12
Am someone who tends to be Adjective - Adjective Only	0.18	0.04	4.75	< .001	0.11	0.25
Am someone who tends to be Adjective - Am Adjective	0.14	0.04	3.65	.001	0.06	0.21
Am someone who tends to be Adjective - Tend to be Adjective	0.13	0.04	3.47	.002	0.06	0.21
Tend to be Adjective - Adjective Only	0.05	0.04	1.28	.604	-0.03	0.12
Tend to be Adjective - Am Adjective	0.01	0.04	0.18	.854	-0.07	0.08

Average log-seconds to “worrying” by format (blocks 1 and 2)

Inclusion of “I” (Blocks 1 and 3)

We used a multilevel model, nesting response within participant to account for dependence. Our primary predictors are format and also the presence of the word “I”. Here, we use data from blocks 1 and 3. Results are depicted in Figure @ref(fig:secondsi).

items_13 = items_df %>%
  filter(block %in% c("1","3")) %>%
  filter(condition != "A") %>%
  filter(time2 == "yes") %>%
  filter(!is.infinite(seconds_log))

mod.format_b3_1 = glmmTMB(seconds_log~format + i + (1|proid),
                  data = items_13)
tidy(aov(mod.format_b3_1)) %>%
  mutate(p.value = papaja::printp(p.value))

## # A tibble: 4 × 6
##   term         df    sumsq meansq statistic p.value 
##   <chr>     <dbl>    <dbl>  <dbl>     <dbl> <chr>   
## 1 format        2    42.9  21.5       60.7  "< .001"
## 2 i             1     2.28  2.28       6.46 ".011"  
## 3 proid       660  5542.    8.40      23.8  "< .001"
## 4 Residuals 49612 17536.    0.353     NA    ""

mod.format_b3_2 = glmmTMB(seconds_log~format*i + (1|proid),
                  data = items_13)
tidy(aov(mod.format_b3_2)) %>%
  mutate(p.value = papaja::printp(p.value))

## # A tibble: 5 × 6
##   term         df    sumsq meansq statistic p.value 
##   <chr>     <dbl>    <dbl>  <dbl>     <dbl> <chr>   
## 1 format        2    42.9  21.5       60.7  "< .001"
## 2 i             1     2.28  2.28       6.46 ".011"  
## 3 proid       660  5542.    8.40      23.8  "< .001"
## 4 format:i      2     5.25  2.63       7.43 ".001"  
## 5 Residuals 49610 17531.    0.353     NA    ""

Predicted log-seconds on personality items by condition and I, using Block 1 and Block 3 data.

One model for each adjective

Additive effects of I (controlling for format) are summarized in Table @ref(tab:itemi). Tests of the interaction of I with format (for each item) are summarized in Table @ref(tab:iinteraction).

mod_by_item_i1 = items_13 %>%
  group_by(item) %>%
  nest() %>%
  mutate(mod = map(data, ~glmmTMB(seconds_log~format+i + (1|proid), data = .))) %>%
  mutate(aov = map(mod, aov)) %>%
  ungroup()

summary_by_item_i1 = mod_by_item_i1 %>%
  mutate(tidy = map(aov, broom::tidy)) %>%
  select(item, tidy) %>%
  unnest(cols = c(tidy)) %>%
  filter(term == "i") %>%
  mutate(reverse = case_when(
    item %in% reverse ~ "Y",
    TRUE ~ "N"
  )) %>%
  mutate(p.adj = p.adjust(p.value, method = "holm"))

Additive effect of I on timing for each item

item	reverse	sumsq	meansq	df	statistic	p.value	p.adj
active	N	0.73	0.73	1	2.12	.146	> .999
adventurous	N	0.18	0.18	1	0.51	.477	> .999
broadminded	N	0.17	0.17	1	0.51	.475	> .999
calm	N	0.09	0.09	1	0.27	.602	> .999
caring	N	0.01	0.01	1	0.04	.845	> .999
cautious	N	0.06	0.06	1	0.14	.708	> .999
cold	N	0.38	0.38	1	1.21	.271	> .999
creative	N	0.06	0.06	1	0.17	.683	> .999
curious	N	0.42	0.42	1	1.22	.270	> .999
friendly	N	0.01	0.01	1	0.02	.885	> .999
hardworking	N	0.10	0.10	1	0.35	.556	> .999
helpful	N	1.37	1.37	1	5.90	.015	.540
imaginative	N	0.01	0.01	1	0.04	.851	> .999
intelligent	N	0.30	0.30	1	0.85	.358	> .999
lively	N	0.02	0.02	1	0.06	.809	> .999
organized	N	0.83	0.83	1	2.74	.098	> .999
outgoing	N	3.23	3.23	1	11.62	.001	.026
quiet	N	0.14	0.14	1	0.52	.470	> .999
relaxed	N	0.56	0.56	1	1.82	.178	> .999
responsible	N	0.53	0.53	1	1.45	.229	> .999
selfdisciplined	N	1.46	1.46	1	4.54	.034	> .999
shy	N	0.07	0.07	1	0.22	.642	> .999
softhearted	N	0.02	0.02	1	0.05	.827	> .999
sophisticated	N	0.68	0.68	1	2.00	.158	> .999
sympathetic	N	0.16	0.16	1	0.56	.453	> .999
talkative	N	0.02	0.02	1	0.07	.797	> .999
thorough	N	0.76	0.76	1	2.37	.124	> .999
thrifty	N	0.25	0.25	1	0.79	.376	> .999
uncreative	N	0.07	0.07	1	0.20	.653	> .999
unintellectual	N	0.33	0.33	1	0.98	.322	> .999
unsympathetic	N	0.26	0.26	1	0.97	.326	> .999
warm	N	0.00	0.00	1	0.01	.931	> .999
careless	Y	0.13	0.13	1	0.49	.485	> .999
impulsive	Y	0.30	0.30	1	0.77	.380	> .999
moody	Y	1.66	1.66	1	6.70	.010	.365
nervous	Y	0.63	0.63	1	2.02	.156	> .999
reckless	Y	1.79	1.79	1	6.46	.011	.406
worrying	Y	0.00	0.00	1	0.01	.926	> .999

mod_by_item_i2 = items_13 %>%
  group_by(item) %>%
  nest() %>%
  mutate(mod = map(data, ~glmmTMB(seconds_log~format*i + (1|proid), data = .))) %>%
  mutate(aov = map(mod, aov)) %>%
  ungroup()

Interaction of I with format on timing for each item

item	reverse	sumsq	meansq	df	statistic	p.value	p.adj
active	N	1.42	0.71	2	2.08	.126	> .999
adventurous	N	1.02	0.51	2	1.46	.234	> .999
broadminded	N	0.31	0.15	2	0.46	.631	> .999
calm	N	1.03	0.51	2	1.64	.194	> .999
caring	N	1.03	0.51	2	1.94	.144	> .999
cautious	N	1.83	0.91	2	2.13	.119	> .999
cold	N	0.04	0.02	2	0.07	.937	> .999
creative	N	0.13	0.07	2	0.18	.834	> .999
curious	N	1.29	0.64	2	1.85	.157	> .999
friendly	N	1.01	0.51	2	1.55	.214	> .999
hardworking	N	0.14	0.07	2	0.25	.779	> .999
helpful	N	1.18	0.59	2	2.54	.080	> .999
imaginative	N	0.45	0.23	2	0.69	.501	> .999
intelligent	N	1.69	0.85	2	2.42	.090	> .999
lively	N	1.92	0.96	2	2.56	.078	> .999
organized	N	0.77	0.39	2	1.28	.280	> .999
outgoing	N	0.01	0.00	2	0.01	.989	> .999
quiet	N	0.02	0.01	2	0.04	.956	> .999
relaxed	N	0.16	0.08	2	0.27	.766	> .999
responsible	N	0.29	0.15	2	0.40	.673	> .999
selfdisciplined	N	0.52	0.26	2	0.81	.443	> .999
shy	N	1.65	0.82	2	2.44	.088	> .999
softhearted	N	0.42	0.21	2	0.55	.579	> .999
sophisticated	N	0.15	0.07	2	0.22	.803	> .999
sympathetic	N	0.43	0.22	2	0.75	.474	> .999
talkative	N	0.01	0.00	2	0.02	.985	> .999
thorough	N	0.27	0.13	2	0.42	.659	> .999
thrifty	N	0.06	0.03	2	0.10	.905	> .999
uncreative	N	0.50	0.25	2	0.77	.463	> .999
unintellectual	N	1.26	0.63	2	1.87	.155	> .999
unsympathetic	N	1.17	0.58	2	2.22	.110	> .999
warm	N	0.21	0.11	2	0.31	.734	> .999
careless	Y	2.21	1.11	2	4.05	.018	.676
impulsive	Y	0.00	0.00	2	0.00	.997	> .999
moody	Y	0.05	0.03	2	0.11	.898	> .999
nervous	Y	1.88	0.94	2	3.05	.048	> .999
reckless	Y	0.97	0.49	2	1.76	.173	> .999
worrying	Y	0.64	0.32	2	1.13	.325	> .999

Here we identify the specific items with significant differences.

sig_item_b3 = summary_by_item_i2 %>%
  filter(p.value < .05)

sig_item_b3 = sig_item_b3$item
sig_item_b3

## [1] "nervous"  "careless"

adjective_timing_i = function(adjective){

  model = items_13 %>%
    filter(item == adjective) %>%
    filter(condition != "A") %>%
    glmmTMB(seconds_log~format*i + (1|proid), data = .)

  plot = avg_predictions(model, variables = c("format", "i")) %>%
    mutate(across(where(is.numeric), exp)) %>%
    ggplot(aes(x = format, y = estimate, group = i)) +
    geom_point(aes(color = i),
               position = position_dodge(.3),
               size = 3) +
    geom_errorbar(
      aes(ymin = conf.low, ymax = conf.high),
               position = position_dodge(.3),
      width = .3) +
    labs(
      x = NULL,
      y = "seconds",
      title = paste0("Average response time to ", str_to_sentence(adjective))) +
    theme_pubclean()

  return(plot)
}

Nervous

adjective_timing_i("nervous")

Average seconds to “nervous” by format and inclusion of i (blocks 1 and 3)

Careless

adjective_timing_i("careless")

Average seconds to “careless” by format and inclusion of i (blocks 1 and 3)