Ritchey ME, Armstrong MA, Hunter S, Shi JM, Getahun DT, Xie F, Dierig C. Evaluation of observational study data with multilevel propensity scores: the planning process. Poster presented at the 35th Annual ICPE Conference; August 28, 2019. Philadelphia, PA. [abstract] Pharmacoepidemiol Drug Saf. 2019 Aug 20; 28(S2):1216.


BACKGROUND: Pharmacoepidemiology database studies often utilize propensity scores (PS) to adjust for confounding between two exposure groups while reducing dimensionality by including fewer independent variables within outcome models. A multi-database study was originally planned with a binary exposure, but then modified to 4 exposure levels at FDA request.

OBJECTIVES: To codify a process for shifting the design from a binary to 4-level exposure, highlight considerations for discussion.

METHODS: Three distinct parts of PS model development were discussed in considering the change from binary to 4-level exposure: (1) determine the model to develop the PS, (2) assess balance in the PS model, and (3) decide on whether and how to do trimming. The FDA requested specific documentation of how covariate balance would be assessed and achieved and how the 4-level PS would be incorporated into outcome models.

RESULTS: After deciding that PS was still appropriate for the study, PS model development was discussed. (1) Exposure levels could be treated as ordinal or nominal; nominal and multinomial logistic regression models were chosen. (2) Balance could be assessed using standardized bias assessment (e.g., absolute standardized differences [ASD]) either as summary measures or via pairwise comparisons between all pairs of exposure levels (n = 6 comparisons) or as comparison versus a single referent (n = 3 comparisons). Summary measures could be created via averages, min/max of pairwise comparisons, or as a composite score accounting for distance between all 4 levels simultaneously. Pairwise comparisons versus the referent category of ASD for all variables included in the PS model was chosen. If any ASD was above 0.2, further refinement of the PS model would be used. (3) Overlap could be considered as the range of PS covered by any pair of exposure levels, the coverage between the referent and each of the other 3 levels, or the coverage only for the range of PS occurring in all 4 levels of the exposure. “Fully overlapping” regions would be ideal for interpretation but would narrow the range of PS. A broader PS range of any overlapping may be more robust analytically and was deemed appropriate for the primary analysis, with the caveat that this would be revisited after examination of the data.

CONCLUSIONS: This process allowed the study team to discuss and decide upon changes to PS model development to address regulatory research questions.

Share on: