Testing Sampling Bias in Estimates of Adolescent Social Competence and Behavioral Control

Background

Although cohort studies generally aim at selecting a sample that is representative for the whole population, vulnerable groups in our society are less often part of these cohort studies (Jang and Vorderstrasse, 2019). This can result in a sampling bias of participants with a higher socioeconomic status (SES; Bornstein et al., 2013). An important question that follows is whether findings from such samples reflect the psychosocial development of the whole population or of a subsample of our society (LeWinn et al., 2017). We investigated whether estimates of social competence and behavioral control in adolescents from 6 Dutch developmental cohorts differ between the unweighted samples and their weighted samples that are more socioeconomically representative of the general Dutch population.

Method

Participants of 6 large cohort studies from The Netherlands (Generation R, L-CID, NTR, RADAR, TRAILS, YOUth) were investigated. We contrasted the unweighted versus the weighted sample of 6 large cohort studies from the Netherlands. The unweighted sample consisted of adolescents with complete observations on SES variables, and at least one observed score on social competence or behavioral control. The weighted sample was created using a raking procedure, and is representative of the Dutch population in terms of socioeconomic status. The unweighted sample and the weighted sample consisted of the exact same participants. National census data on parental education and income was retrieved from the open data portal of Statistics Netherlands.

Results

Mean estimates of social competence and behavioral control were mostly similar between the unweighted sample and the weighted counterpart. Due to a lack of initial differences in adolescent social competence and behavioral control between SES strata, the raking procedure yielded no change in mean scores or correlations in L-CID, RADAR, and YOUth. Despite adolescents from the lower SES categories scoring lower on social competence or behavioral control than adolescents form the higher SES categories, our raking procedure also yielded no changes in normative estimates in Generation R, NTR, and TRAILS. Contrary to our expectations, these findings suggest that normative adolescent social competence and behavioral control is not overestimated as a result of predominantly high SES adolescents in Dutch developmental research cohorts.

Conclusion

By testing our research question in 6 different developmental cohorts; with various measures of SES, social competence and behavioral control; on various test statistics; across a broad age range of late childhood and adolescence; and verified through several sensitivity analyses, our findings can be considered robust. However, it can be questioned how representative the low SES participants in the cohorts are for the low SES population, given their small numbers. Also, the number of weighing variables was lower than planned, as a result of cohort differences in the measurement of occupational status. We recommend researchers to assess the socioeconomic validity of samples: 1) by counting the number of SES indicators that are measured, 2) by checking for undersampling in any combination of SES indicators (e.g., lower educated parents and low income), and 3) by contrasting the SES characteristics of excluded participants to those of included participants.

Contact: m.fakkel@uu.nl

Link to paper > Contact > Website >
Read all news