Live Webinar with Dr. Shao-Hsien Liu

Thank you all so much for attending our OAI.edu live webinar series hosted by the OAI core knowledgebase. I want to thank today's speaker Dr. Shao-Hsien Liu who is associate professor in the department of population and quantitative health sciences at UMass Chan Medical School. The title of today's talk is causal inference using osteoarthritis initiative data opportunities challenges and practical considerations. Following the presentation, we should have a few minutes for questions and answers. If you have any questions, please feel free to type them into the chat and we'll try to get to them. And now I'll let Shao-Hsien take over. Thank you Shao-Hsien. Good morning everyone. Thank you Jeff. Um and thank you everyone for joining today's um today's uh webinar title is causal inference using osteoarthritis and initiative data opportunities challenges and practical considerations. I wanted to um frame today's around the practical uh possibility. OAI is an observational cohort but is repeat clinical medication imaging behavior and uh the patient reported data allow us to ask some questions in a way that resembles the trial we wish we could run. So the goal is not to pretend that OAI is randomized. The goal is to use the causal design principles to make the causal questions um index day or time zero comparison groups or any confounding issues and limitations clear so that we can decide whether the OAI can be used to support the questions.

Um sorry imaging that the uh clinical question is whether sustained prescription NSA use improves symptoms or slows structural progression in knee oa ideally we would answer that with a multi-year randomized trial. While OAI cannot recreate the randomization process and a causal analysis using OAI does not replace a randomized trial design. It can complement trial evidence by examining how interventions are used in the real world, generating evidence that supports a future trial and exploring long-term or otherwise in feasible trial questions. As you may know, OAI contains many of the data elements we would need. Uh for example, the repeat treatment information, repeated uh outcome measurements such as WOMAC uh measurements. uh they also provide a serial radioraphase and enrich information on disease severity and other treatments so that this causal inference opportunity is to organize those data around the trial question we wish we had um sorry this is a little bit okay by the end of this webinar I hope you will be able to do uh the four things uh we show on the slide first formulate an actionable causal questions by defining the population comparisons groups. Uh anchor your time zero uh index day um determine your outcomes and the exact causal effect of interest which we call the estimate. Second, take the ideal target trial and map it on to what OAI provides. For example, uh the annual visits, the variables, the measurement windows and the follow-up structure. Third, recognize the major longitudinal causal challenges including the confounding uh issues, the treatment confounding feedback, sensoring issue, missing data and then um the identifiable assumption to understand where methods such as marginal structural models may be appropriate.

Before we introduce uh the causal terminology um I think this slide will help to distinguish some uh fundamental um different goals of your clinical questions. For example, the predictions asks who is likely to experience an outcome. Association asks whether an exposure and the outcome tend to occur together. A causal question asks what would it happen if the exposure or treatment strategy were changed for the whole population. Um all three uh questions are valuable and the OAI can support all three uh designs but they answer but these three questions answer different uh but they answer answers uh differently. Um and they require different design and interpretations. uh using our NSET uh working example in the NSET uh the first one who uses Nset is the descriptive or predictive which is related to the first elements are NSA user different from non- users is more likely a a social uh examination the causal question is what would happen to symptoms or structural progression under sustained NSA use versus non-users Here

are the six terms that I would like to introduce first uh since they will appear repeatedly. Uh for example, the target trial is the randomized trial we wish we could conduct. Uh the estimate is the exact cause effect we want to we want from that trial for the estimation. Time zero is the point when it is confirmed. The treatment strategy is also assigned and the followup begins. Treatment can start immediately after assignment. The key for here is that the eligibility assignment and the followup are aligned. A confounder effects both uh affects both treatment choice and outcome. The positivity means both treatment strategy are realistically represented among uh the comparable participants within the subgroups. Finally, IPW stands for the inverse probability waiting and the MSS stands for the marginal structural models. I will return to those uh terms later. For now, the key idea is that the waiting process can create a pseudo population in which major confounder relationship are reduced allowing a marginal comparison across treatment histories. Now back at the OAI some of you may know that OAI is a large longitudinal clinical cohort with multiodel data. It includes repeated patient reported symptoms and function, clinical measures, performance tests, radiographics and MRI images, medication and behavioral information and other uh B specimens at some selected visits. For causal inference, the key advantage is not simply the number of variables. It is the repeat measurements. We can reconstruct those treatment or behavior histories. uh understand the time varying compounding histories in relations to our ultimate outcomes that we would like to estimate. Even when OAI maps well onto the target trial, the analysis remains observational. Uh here's the caution. For example, the treatment is still not randomized. Excuse me. People use NSAIDs, they change their activities or they lose weight for reasons related to their symptoms uh and disease severity and other treatments. Visit cadence and measurement windows can can also be misaligned. Followup and imaging availability can create selection bias and the positivity assumption can or we may not be made uh be able to meet when one treatment strategy is very uncommon in the important subgroup. So the point of causal methods is not to make these issues disappear. It is to make the design and assumption clear and to address the parts on the bias structure that the data can actually support.

Once we have the causal question, I recommend first writing down the trial protocol before we think about the statistical model. For example, who will be eligible? What treatment strategy we are trying to compare here? Uh when is the potential time zero index day? How long is the potential uh followup could be available or we envision to be a able to follow up the participants? What is the potential outcome and what exact effect we are trying to estimate for our running example? We would like to implement a trial um among adults with radiographic neoa who are not using prescription inset as baseline. We will compare initiating the NSAID use versus uh and then sustaining prescription NSAID use versus the non-users and then we measure their symptoms and structural progression over time. So the treatment assignment uh starts with the index day the time zero together with eligibility and the start of followup excuse me treatment initiation can occur immediately after assignment. Keeping those elements align helps prevent immortal time bias and selection bias.

So this slide uh I would like to try to show the actual emulation step for every component of the target trial. We ask whether the closest defensible OAI analog is. For example, electricity is quite straightforward and using our previous example as well. The treatment strategy are harder because OAI records what happened in the observational nature rather than assign assigning the treatment. Um time zero needs to align the eligibility treatment and the beginning of outcome followup. followup has to reflect the actual oi visit structure and then outcomes has to be um measured in the right window. So the biggest difference is the treatment assignment. Uh for example, we cannot reproduce the randomization process. But using the statistical techniques, we we can use the measure career information and causal assumption to make the treated and untreated uh group comparable and over time. So the uh the target trial emulation is not a statistical methods. It is a design framework. It tells us exactly what we are trying to imitate and whether oai is an approximation rather than the ideal trial.

So to decide whether to pursue uh the causal question or project, I think we can use this simple framework to ask how well OAI can align or or approximate the target trial we have in mind. The first green uh light questions has the repeat treatments uh or exposure measurements. The confounders measure before treatment decisions outcomes on also on a useful schedule and enough participant follow throughout your available data. However, if the key exposure confounders or outcomes and treatment strategy are well uh represent in the OAI um but some places are available but the timing of measurement is imperfect. This is more potentially a yellow uh place. And if cr for example the critical treatment information key confounders or adequate overlap are missing. This may be a red light area and a sign that we should rethink the question we have in mind and whether we should pursue further regarding using the design from OAI data. I think this process help us decide not only how to do the analysis but whether the question is identifiable enough uh to be uh worth pursuing. Once the target trial is defined, the next practical step is to draw the uh potential um timeline using the OAI data structure. Um at every visit ask what information is known before the treatment or exposure classification. What changes over time and when the outcome is assessed. Here on the figure the L repres represents the clinical and confounder history. A is the exposure or treatment strategy and Y is our final outcome. So typically we determine the eligibility using baseline or past information. Confounders should also be uh preceding the treatment decision. Uh they intend to be adjusted and the outcome followup should begin after time zero.

Uh using the general timeline as we show uh previous in the previous slide we this slide shows our inset uh example. At baseline we define eligibility using only information available. Then uh the radiographic knee oa uh a kale gray greater than two and no reported prescription use a baseline and then baseline clinical history provide the pre treatment cores uh uh for our treatment uh history and and information at baseline and then at the 12 24 and 36 months visits OAI updated the clinical history and capture prescription ay used during the preceding 30 days. Those three reports form the cumulative treatment history. Four outcomes in terms of uh the WMAC uh outcome measurements and joint space waste changes over time were then assessed through the 4year visit. This makes the ordering ver visible. Uh for example, the eligibility comes first. Each exposure report precedes the outcome used for the interval and the outcome follows the treatment history. It also reveals a limitation. Uh in our study, the annual medication inventories do not provide the exact start or stop dates. So we assume the medication use is continued between annual visits.

In addition, the cohort construction is also part of the causal design, not just the data management step. Uh the common st mistake is to let the future information determine eligibility. for example, require future visits, selecting a knee because of the later it later progressed or choosing the index knee based on the data uh total knee replacement surgery uh status. Um that effective looks into the future may not be the best uh appropriate approach. The the target trial design give us a clean rules. The eligibility again is determined at the time zero using information available. Then we also need to define the causal unit such as a person knee or knee within the person and be clear about whether the treatment strategy concerns new users um or prevent users or sustain uh treatment histories. Here on this slide is an example of the causal diagram that forces us to state which variable cost treatment choices and which variable cost the outcome. As such the deck is the transparent statement of assumption behind the adjustment set. Um in the OAI the longitudinal challenge becomes more interesting when the confronter it is is itself affected by the primary prior treatment. Again here on the figure the A represent the exposure or treatment strategy and L represents the time varying clinical history. At time zero the participant received exposure A Z. The exposure may change clinical uh throughout the um follow-up time and the exposure may change clinical history L1. L1 then predicts both the next exposure A1 and the outcome. In the traditional uh regression approach, if we adjust for L1 in the in the traditional model, we may block part of the earlier treatment effect. But if we do not adjust for L1 the later treatment remains confounded. So that treatment uh confounded feedback is the classic uh motivation for causal methods approach. Here is uh we use example from marginal structural models. Now let's make the time variant confounding issue concrete using our running example. Again, A on the figure represents the prescription essay use and L represent the evolving clinical history including the pain or function. And then pain at one visit can influence whether someone use NSA at the next visit, but prior NSA use history may also have changed that pain. The updated pain then predicts later treatment use and eventually affect our ultimate outcome. So update p up updated pain is both a consequence of earlier treatment use and a confounder for the later treatment. This exactly this is exactly why the ordinary or traditional um regression adjustment becomes difficult um because conditioning on the pain can block part of the earlier treatment effects and however ignoring the pain leaves the later use compounded. So again this feedback structure is the reason why we use the marginal structural models approach for our analytic approach.

On the other hand uh sensoring and missing data could also uh affect our estimates. A missing item at one visit is different from complete loss to followup. uh the total knee replacement surgery can make subsequent knee structure outcomes undefined or change their clinical meaning. Death is a competing event for example and for many longitudinal outcomes as well. Imaging availability may reflect study design or ancillary reading projects rather than random missingness. So those critical issues uh is we have some um adjustment or we can conduct sensitivity analysis to pull all these together into uh but rather I recommend not just pull those into a generic missing data issue or bucket. we need to understand our data and decide what each event means for the target trial and then choose the appropriate strategy that corresponds to those um the final estimates we have in mind. So overall not every causal question needs a methods like we mentioned earlier like the marginal structural models. Uh in this slide for example the bas line treatment with the baseline confounding may be handled with conventional regression or the propensity score approaches. However time varying exposure without treatment confounded feedback may also be addressed with longitudinal regression or other causal methods. In our example, marginal structural models becomes es especially useful in this case when treatment changes over time and the time varying confounders are affect by the prior prior treatment. More complex dynamic strategy may motivate the other methods and approach as well. So uh the methods should follow the causal structure and the assumption including uh the uh the the major assumption within the uh causal methods approach including the consistency exchangeability positivity assumption also uh in com in um in combined with the adequate measurement regardless of which method we choose.

Next, I will use a a few couple slides to illustrate our published uh NSET study as an example to show how this causal design principles can be applied using the OAI data. Um we try to estimate whether the recent and long-term prescription use res leave the symptoms and delay structural progression among people with radiographically confirmly osteoarthritis. A key design feature was the new user approach of the 2500 participants with radiographic OA. Uh 2,000 participant were not reporting prescription NSAID use at baseline. They gave us a pre-treatment starting point rather than mixing long-term prevalent users with non-users. And then prescription say use come from the medication inventory covering the 30 days before the uh annual visit. We consider the cumulative use reported across one two or three annual assessment outcomes were changes in pain stiffness physical function and radiographic joint space width. OAI also provides us the advantage that will be challenging in many trials. The real world treatment patterns in the henius h histoenius cohort and repeated outcomes over four years. So this important design point is that we construct the cohort around treatment initiation and repeated treatment history rather than treating baseline status as a fixed exposure. Our study had exactly the longitudal feedback problem we just described. This is the simple simplified deck showing on the figure. Uh again symptoms and other clinical characteristics predicted the later NSA use but some of those same variables could have been affected by the earlier NSAT treatment. We therefore model the probability of the observed unset treatment each year using baseline and tying clinical history. We also model the sensoring uh because of uh the sensoring reasons such as total knee replacement surgery uh death loss to follow up and missing data could make the remaining observed sample selective. So the advantage of the marginal structural models framework is that we use the updated confounder history to construct weights rather than the simp rather than simply putting every updated variable into the final outcome regression model.

Conceptually the ways creates a pseudo population in which measure confounder association are balanced allowing a marginal comparison or treatment history. This still depends on the usual causal assumption and sufficiently specified treatment and sensory models. Overall, the analysis was conducted in four steps. First, we estimate the probability of each participants uh observed in treatment use at each annual visit given their major compounding history. Second, we estimated the probability of remaining on sensor. It's called the sensoring weight. Third, we combine those uh treatment weights and sensoring weights across visits. And then we also uh uh truncate the final weights at the 99th percentile to reduce the influence of extreme extreme weights. Finally, the fourth steps is we fit the weighted linear models for all outcomes such as the warm subscales and join space width.

Um overall we observed that our point estimate suggest that reporting prescription NSAID use at all three annual visit was associated with clinically meaningful uh magnitudes of improvement in stiffness and functional and less less joint space narrowing but not the pain. However, the confidence interval include the no uh recent NCA use also showed a no clinically or sa statistically significant effect. The correct interpretation from our observations that the estimates suggested the potential meaningful long-term signals but with substantial certainty. The uncertainty is part of the lesson we learned. The causal methods can reduce some bias but they do not create precision or may be uh able to create the precision when the uh persistent treatment use is rare. In terms of the strength and limitation of the the our NSAID example, we see that the OAI provides a new user starting point, repeat medication information. Uh um and they also provide repeated symptoms and structure outcomes, reach clinical history and enough follow-up uh uh uh period to construct the longitudinal treatment and censoring ways. But OAI did not record exact inset start and stop days or uh the the dose uh so uh dosage uh information are not available. The medication inventory cover only the 30 days before the annual visit. So our analysis assumed the use was continuous uh between the annual visit. So the persistent prescription essay use was uh in in the we know is uncommon. So there's also a limitation and then the residual compounding could still be a possibility.

So stepping back from the NSAID example, this is the workflow I would recommend for any causal uh OAI project. First you write your intervention and estimate in one sentence. You specify the target trial and then draw your causal map. Building the visit by visit data map. Write the bias plan uh potential for company issue, missing data, sensoring and potential for positivity issues. Choose the simplest methods compatible with the structure. Check the um if you are doing some waiting technique, check the uh using the diagnostic uh tools as well. then use sensitivity analysis to pull up the specific assumption. My practical recommendation to write this design protocol before writing the analytic code. This keeps the questions in charge of the analysis rather than letting the available variables of every methods drive the study. One thing I also wanted to know is that the sensitivity analysis should be tied to the causal argument rather than added as a generic checklist. For example, uh change the exposure definition when treatment measurement is uncertain. Examine richer or alternative confounder history and you should also inspect the extreme ways or alternative truncation rules. um those are will be helpful to determine um to give you your understanding of the US dimension. Finally, I will end today's webinar with four messages. First, start with the decision, not the model. Second, use the target trial thinking process as the blueprint for mapping that uh decision to the data. Third, when treatment and confounders evolve together, methods such as marginal structural model may be needed because the problem is the causal structure, not simply the inf insufficient cover adjustment. And fourth, no causal methods can fix treatment timing or dosage uh in need. uh there was never measures a key u major commanders or treatment strategy with essentially uh no overlap is a potential issue as well. So overall the reason I'm excited about the causal inference within the over data structure is that it expands the kind of question we can ask instead of stopping at what is associated with progression we can often move toward a more actionable question for example what would happen under an alternative strategy.

Thank you uh today uh very much for your time and attention. I hope this main takeaway is that the causal inference with OAI is less about choosing the s self sophisticated methods and more about aligning the research question to the data uh for example the temporal structure the analytic sample and the assumption and methods. If you have causal question and would like to uh would like feedback on your study design or analytic approach, please please reach out to the OAI core knowledgebase. Our email address and website are also shown here. And thank you again. Now I'm happy to take questions. Thanks Shao-Hsien. Uh we do we are about at time but if anybody has any questions we can certainly uh reach out to Shao-Hsien via the OAI knowledgebase. Um, Arlene Ash added into the comments, very nice talk. If P or one minus P is close to zero, I suggest using overlap weights. Um, and this just uh that's just about all the time we have for today. I want to thank you again for that excellent presentation and thank everybody for joining us today. As we wrap up, I want to invite you all for uh invite you all to our upcoming webinar on October 19th with Dr. Pamela Semanik. She'll be talking about accelerometry data from the OAI. This is going to be another great session. So, we hope you all can make it. The link to register is in the chat along with the links to our website and the webinar survey. And I hope you'll please complete that survey. And that concludes today's webinar. Thank you everyone and have a great rest of your day. Thank you.