Learning under concept drift and non-stationary noise: Introduction of the concept of persistence - TÜMER, MUSTAFA BORAHAN (My University, 2023)
Learning from noisy data is a challenging task especially when the system under consideration has a non-stationary nature. The source of the noise is often assumed to be stationary, however the severity or characteristics of noise may also be time-varying, which causes multiple sources of drift in the collected data. This study introduces a novel adaptive learning rate approach to improve learning when the observations from a non-stationary system is altered by an also non-stationary source of noise. As an example to this approach, we propose Persistence Aware Robust Learner (PeARL), an online learning method that utilizes a novel concept called persistence, which is a local noisiness estimation metric to measure the correspondence of a signal to discrete white noise. Making use of this metric, PeARL is able to adaptively adjust the learning rate for each observation during learning to reduce the effect of noise. With this level of control on the learning rate, noisy instances have less disruptive effect on the maintained estimate. We experimentally evaluate PeARL on (a) systematically generated synthetic data and (b) real-world data, including accelerometer readings collected from people (HASC2010corpus) and current measurements from electric motors collected within the scope of EU-funded research project iRel40. The experiments reveal a favorable region of noise rate, in which the proposed method achieves up to 40% reduction in mean absolute error (MAE).
Keywords: Bilgi Sistemleri, Haberleşme ve Kontrol Mühendisliği; Kontrol ve Sistem Mühendisliği; Sinyal İşleme; Bilgisayar Bilimleri; Algoritmalar; Mühendislik ve Teknoloji; Information Systems, Communication and Control Engineering; Control and System Engineering; Signal Processing; Computer Sciences; algorithms; Engineering and Technology; Mühendislik, Bilişim ve Teknoloji (ENG); Bilgisayar Bilimi; Mühendislik; OTOMASYON & KONTROL SİSTEMLERİ; BİLGİSAYAR BİLİMİ, YAPAY ZEKA; MÜHENDİSLİK, ELEKTRİK VE ELEKTRONİK; Engineering, Computing & Technology (ENG); COMPUTER SCIENCE; ENGINEERING; AUTOMATION & CONTROL SYSTEMS; COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE; ENGINEERING, ELECTRICAL & ELECTRONIC; Fizik Bilimleri; Yapay Zeka; Elektrik ve Elektronik Mühendisliği; Control and Systems Engineering; Physical Sciences; Artificial Intelligence; Electrical and Electronic Engineering; Concept drift; Dynamic learning rate; Non-stationary noise; Parameter estimation; Stochastic learning