*This post is part of our Q&A series.*

A question from graduate students in our Spring 2021 offering of the new course “Targeted Learning in Practice” at UC Berkeley:

## Question:

Hi Mark,

We were discussing practical implementations of stochastic treatment regimes and came up with the following questions we would like to hear your thoughts about.

Question 1 (Practical positivity):Is there a recommended procedure for deciding truncation threshold with respect to shifts in the framework of stochastic treatment regimes? Further, what might be an acceptable proportion of thresholded observations (or the distribution of`$g_{\delta}/g$`

) such that we are comfortable estimating a shift magnitude? In this sense, one defines a reasonable grid to estimate shift values or a marginal structural model (MSM).

Question 2 (Other form of stochastic treatment):We are interested in what stochastic treatment regime is of most interest to your group to derive estimation for and program into`tmle3shift`

, following the work you’ve done for an additive shift. Practically, are there situations where it might be optimal to have a different function implemented in`tmle3shift`

, such as a multiplicative scaling or nonlinear functions of`$A$`

? Can you data-adaptively select a regime that maximizes the causal parameter?

Question 3 (Combining stochastic and optimal individualized treatment):Somewhat playing off of the end of our last question, if you are interested in estimating optimal continuous treatment values for different individuals or subgroups of interest from the population, can you leverage the estimation of the impact of different shift values (or functions) on your subgroups of interest?Best,

A.M. and S.F.

## Answer:

Hi A.M. and S.F.,

Thank you for your excellent questions.

**Question 1:**
There are two ways to think about this. If one is aiming for
`$\mathbb{E}Y_{g^{\star}}$`

for a particular `$g^{\star}$`

, then truncation of
`$g^{\star}/g$`

by some constant `$M$`

tells us that the TMLE for this target
parameter is indexed by `$M$`

, and that it represents a tuning parameter for
this `$M$`

-specific TMLE, `$psi_{n,M}$`

. In that case, one is concerned with
selecting `$M$`

to optimize the MSE of `$\psi_{n, M}$`

with respect to
`$\psi_0$`

under the constraint that the bias should be smaller than the
standard error by a factor of approximately `$1 / \max(10, \log(n))$`

, which
ensures that statistical inference (i.e., confidence interval construction) is
not affected by the bias. For that type of problem, we have been using a variant
of Lepski’s method – see, for example, chapter 25 of the 2018 book *Targeted
Learning in Data Science*.

This comes down to looking for a plateau of `$\psi_{n,M}$`

in `$M$`

, where the
plateau corresponds with the smallest `$M$`

at which `$d/dM \psi_{n,M}$`

is of
the same order (e.g., 2 times) as `$d/dM \sigma_{n,M}$`

, where `$\sigma_{n,M}$`

is the standard error of the TMLE `$\psi_{n,M}$`

. That is, the plateau can be
thought of as appearing before that value of `$M$`

at which the change in the
point estimate that occurs as a result of increasing `$M$`

outweighs the
increase in its standard error. It also corresponds with looking for the first
local optimum of the lower or upper bound of a confidence interval of
`$\psi_{n,M}$`

. This is a powerful method for tuning `$M$`

. One can estimate the
variance of `$\psi_{n,M}$`

with the efficient influence curve (EIC) of TMLE
using this to truncate `$M$`

, where we note that the EIC will have larger and
larger variance as `$M$`

increases.

On the other hand, one might also simply view the selection of the shift
`$\delta$`

, and thereby `$M$`

, as a way to define a stochastic intervention and
accept that we care about the mean outcome under that selected stochastic
intervention. In that case, this becomes an example of a data-adaptive target
parameter, but it was only based on the likelihood of `$g$`

(making it
outcome-blind), so that no CV-TMLE is needed to correct for overfitting.

When making this selection, one still has to keep in mind a certain `$M$`

to
bound `$g^{\star}_{\delta}/g < M$`

. Then, your question concerns what a good way
would be to select `$M$`

. I don’t think we have a very satisfying answer yet on
that question, and I think we should think more about that. The question comes
down to what kind of criterion makes sense.

As one increases `$M$`

, one will obtain wider confidence intervals, so one
somehow needs to decide what kind of variance is still acceptable, i.e., is
there enough support in the data we would want about our target parameter? Maybe
a good strategy is to report the results for all `$M$`

, so that the reader can
honestly evaluate what the impact of `$M$`

really is. We could then think of
providing a simultaneous confidence band to correct for having seen all of these
results. Since the TMLEs `$\psi_{n,M}$`

are very correlated, the adjustment will
not be too costly, and, of course, we can make the width proportional to the
standard error so that the confidence intervals for large `$M$`

are not
dominating the other ones.

**Question 2:**
Yes, we definitely need more interesting shifts or more general choices of
`$d(A,W)$`

, including percentage increases relative to `$A$`

. We really need
a function that can take as input any conditional distribution ```
$g^{\star}(A
\mid W)$
```

, i.e., any stochastic intervention, with the program then computing
TMLEs of `$\mathbb{E} Y_{g^{\star}}$`

. This then includes shift interventions,
but the user can think of their own choices too. The TMLE applies to any choice
`$g^{\star}$`

, so this is not hard. The user could also specify a family
`$g^{\star}_{\delta}$`

and indicate tuning parameters of this intevention that
should be optimized for the sake of `$\mathbb{E} Y_{g^{\star}_{\delta}}$`

or
that should be pushed to the maximal allowed level of the ratio of the
post-intervention and observed treatment densities, `$g^{\star}/g$`

.

**Question 3:**
Yes, we could optimize `$\mathbb{E} Y_{g^{\star}_{\delta}}$`

as a function of
the shift parameter or optimize the upper bound of the confidence interval for
this parameter so that it also takes into account the increased standard error.
I prefer the latter. This applies to any user-supplied parametric family or
nonparametric family of candidate stochastic interventions. Optimal rules for
continuous-valued treatments is an important topic itself and gets into working
with non-pathwise differentiable target parameters. We have some work on it with
Alex Luedtke, also Michael
Kosorok and others, but more to do for sure.

Best Wishes,

Mark

**P.S.**, remember to write in to our blog at ```
vanderlaan (DOT) blog [AT]
berkeley (DOT) edu
```

. Interesting questions will be answered on our blog!