Article Details

Title · Authors · Abstract · Keywords · DOI · References

ICAF: Unsupervised Continual Learning with Interpolation Consistency and Adaptive Filtering

YE Gencheng1,2, WANG Xiang1,2

(1.Beijing Key Lab of Traffic Data Analysis and Mining (Beijing Jiaotong University), Beijing 100044, China; 2.School of Computer Science and Technology, Beijing Jiaotong University, Beijing 100044, China)

Section:Artificial Intelligence · 2025, Vol. 9, No. 11 · Pages:49-53,58 · 2096-4706(2025)11-0049-06 · CLC number:TP18;TP391.41 · Document code:A · DOI:10.19850/j.cnki.2096-4706.2025.11.010

Abstract

Unsupervised Continual Learning addresses the challenge of sequentially learning tasks without supervision information. However, this setting often leads to catastrophic forgetting. To overcome this issue, memory banks are used to store samples from past tasks. However, storing and replaying appropriate samples is a complex task, and the simple and commonly used random selection is not effective, especially for long-sequence tasks with large-scale samples. To solve this issue, an Unsupervised Continual Learning method with Interpolation Consistency and Adaptive Memory Filtering (ICAF) is proposed. It can effectively select high-quality samples by using the features learned from randomly augmented samples, adjust the threshold during training, and achieve interpolation consistency through a specially designed linear interpolation and data combination strategy, thereby preventing insufficient learning caused by relying solely on interpolated samples from past tasks. This method has achieved the best results on multiple experimental datasets.

Keywords

Continual Learning; Unsupervised Learning; Representation Learning; Contrastive Learning; Self-supervised Learning

References

  1. [1] DELANGE M,ALJUNDI R,MASANA M,et al. A Continual Learning Survey: Defying Forgetting in Classification Tasks [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2022,44(7):3366-3385.
  2. [2] MICHIELI U,ZANUTTIGH P. Incremental Learning Techniques for Semantic Segmentation [C]//2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).Seoul:IEEE,2019:3205-3212.
  3. [3] HADSELL R,RAO D,RUSU A A,et al. Embracing Change:Continual Learning in Deep Neural Networks [J].Trends in Cognitive Sciences,2020,24(12):1028-1040.
  4. [4] BIESIALSKA M,BIESIALSKA K,COSTA-JUSSA M R. Continual Lifelong Learning in Natural Language Processing: A Survey [C]//Proceedings of the 28th International Conference on Computational Linguistics.Barcelona:ACL,2020:6523-6541.
  5. [5] ALJUNDI R,LIN M,GOUJAUD B,et al. Gradient Based Sample Selection for Online Continual Learning [C]//Advances in Neural Information Processing Systems.Red Hook:Curran Associates,Inc.,2019,32[2025-01-03].https://proceedings.neurips.cc/paper_files/paper/2019/file/e562cd9c0768d5464b64cf61da7fc6bb-Paper.pdf.
  6. [6] BUZZEGA P,BOSCHINI M,PORRELLI A,et al. Dark Experience for General Continual Learning: A Strong, Simple Baseline [C]//Advances in Neural Information Processing Systems.Red Hook:Curran Associates,Inc.,2020,33:15920-15930.
  7. [7] CHAUDHRY A,RANZATO M A,ROHRBACH M,et al. Efficient Lifelong Learning with A-GEM [C]//International Conference on Learning Representations.New Orleans:Cambridge University Press,2019:1-20.
  8. [8] RUSU A A,RABINOWITZ N C,DESJARDINS G,et al. Progressive Neural Networks [J/OL].arXiv:1606.04671 [cs.LG].(2022-10-22)https://arxiv.org/abs/1606.04671.
  9. [9] YU X,GUO Y,GAO S,et al. SCALE: Online Self-Supervised Lifelong Learning Without Prior Knowledge [J/OL].arXiv:2208.11266v5 [cs.LG].(2022-08-24).https://arxiv.org/pdf/2208.11266.
  10. [10] ZENKE F,POOLE B,GANGULI S. Continual Learning through Synaptic Intelligence [C]//International Conference on Machine Learning.Sydney:PMLR,2017:3987-3995.
  11. [11] HAO J,JI K,LIU M. Bilevel Coreset Selection in Continual Learning: A New Formulation and Algorithm [EB/OL].(2023-11-03).https://openreview.net/forum?id=2dtU9ZbgSN.
  12. [12] MADAAN D,YOON J,LI Y,et al. Representational Continuity for Unsupervised Continual Learning [J/OL].arXiv:2110.06976 [cs.LG].(2021-10-13).https://arxiv.org/abs/2110.06976.
  13. [13] SOHN K,BERTHELOT D,CARLINI N,et al. FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence [J/OL].arXiv:2001.07685 [cs.LG].[2024-12-23].https://doi.org/10.48550/arXiv.2001.07685.
  14. [14] NIU C,SHAN H,WANG G. SPICE: Semantic Pseudo-Labeling for Image Clustering [J].IEEE Transactions on Image Processing,2022,31:7264-7278.
  15. [15] CHEN X,HE K. Exploring Simple Siamese Representation Learning [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Nashville:IEEE,2021:15750-15758.
  16. [16] ZHANG H,CISSE M,DAUPHIN Y N,et al. Mixup: Beyond Empirical Risk Minimization [J/OL].arXiv:1710.09412 [cs.LG].(2018-04-27).https://arxiv.org/abs/1710.09412.
  17. [17] WANG L,ZHANG X,SU H,et al. A Comprehensive Survey of Continual Learning: Theory, Method and Application [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2024,46(8):5362-5383.
  18. [18] ARAZO E,ORTEGO D,ALBERT P,et al. Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learning [C]//2020 International Joint Conference on Neural Networks (IJCNN).Glasgow:IEEE,2020:1-8.
  19. [19] KRIZHEVSKY A,HINTON G. Learning Multiple Layers of Features from Tiny Images [EB/OL].[2024-09-16].https://api.semanticscholar.org/CorpusID:18268744.
  20. [20] DENG J,DONG W,SOCHER R,et al. ImageNet: A Large-Scale Hierarchical Image Database [C]//2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami:IEEE,2009:248-255.
  21. [21] HE K,ZHANG X,REN S,et al. Deep Residual Learning for Image Recognition [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas:IEEE,2016:770-778.
  22. [22] WU Z,XIONG Y,YU S X,et al. Unsupervised Feature Learning via Non-Parametric Instance Discrimination [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City:IEEE,2018:3733-3742.
  23. [23] BORSOS Z,MUTNY M,KRAUSE A. Coresets via Bilevel Optimization for Continual Learning and Streaming [J/OL].arXiv:2006.03875 [cs.LG].(2020-10-22).https://arxiv.org/abs/2006.03875.
  24. [24] TIWARI R,KILLAMSETTY K,IYER R,et al. GCR: Gradient Coreset Based Replay Buffer Selection for Continual Learning [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.New Orleans:IEEE,2022:99-108.
  25. [25] ZHOU X,PI R,ZHANG W,et al. Probabilistic Bilevel Coreset Selection [C]//International Conference on Machine Learning.PMLR,2022:27287-27302.
  26. [26] YOON J,MADAAN D,YANG E,et al. Online Coreset Selection for Rehearsal-Based Continual Learning [J/OL].arXiv:2106.01085 [cs.LG].(2022-05-18).https://arxiv.org/abs/2106.01085.

Funding

Basic Research Funds for Central Universities (20119jbz110); National Natural Science Foundation of China (62176020); National Key Research and Development Program Project (2020AAA0106800); Beijing Natural Science Foundation Project L211016; Chinese Academy of Sciences (OEIP-O-202004)

About the authors

Ye Gencheng (2000-), male, Han ethnicity, from Handan, Hebei Province, is a graduate student studying clustering, continuous learning, and unsupervised learning.