Parameter Estimation with Unknown Transition Point
In today’s information driven world, there is a huge focus on learning parameters from data. Typically, problems involving learning parameters assume that the target parameter does not change in the data set, and while this is true in certain settings, it is not always the case. There are applications where the parameter changes at some unknown time in the data set and only the most recent parameter is of interest to the predictor. In this work, we first show that for an infinite data stream of Bernoulli iid data, when the parameter changes at some unknown transition point, it is possible for what we call a “time-invariant” estimator to achieve asymptotically vanishing error for estimating the most recent parameter. We provide an algorithm which accomplishes this and analyze the algorithm’s mean-squared error loss as a function of the transition point. We also quantify what is the inherent loss of not knowing the transition point, providing a lower bound on all time-invariant estimators.
Authors: Jennifer Tang