Learning under concept drift and non-stationary noise: Introduction of the concept of persistence

Abstract

Learning from noisy data is a challenging task especially when the system under consideration has a non-stationary nature. The source of the noise is often assumed to be stationary, however the severity or characteristics of noise may also be time-varying, which causes multiple sources of drift in the collected data. This study introduces a novel adaptive learning rate approach to improve learning when the observations from a non-stationary system is altered by an also non-stationary source of noise. As an example to this approach, we propose Persistence Aware Robust Learner (PeARL), an online learning method that utilizes a novel concept called persistence, which is a local noisiness estimation metric to measure the correspondence of a signal to discrete white noise. Making use of this metric, PeARL is able to adaptively adjust the learning rate for each observation during learning to reduce the effect of noise. With this level of control on the learning rate, noisy instances have less disruptive effect on the maintained estimate. We experimentally evaluate PeARL on (a) systematically generated synthetic data and (b) real-world data, including accelerometer readings collected from people (HASC2010corpus) and current measurements from electric motors collected within the scope of EU-funded research project iRel40. The experiments reveal a favorable region of noise rate, in which the proposed method achieves up to 40% reduction in mean absolute error (MAE).