Related work

The foundational work on continual learning, 1991 to 2026: methods, theory, benchmarks, surveys, and the neuroscience of memory. Updated weekly; some carry our notes.

162 papers of 4,574 · showing 101–150Sort Recent · Most cited
  1. 2024
    Rethinking Momentum Knowledge Distillation in Online Continual LearningNicolas Michel, Maorong Wang, Ling Xiao, Toshihiko YamasakiICML
    PDF ↗
  2. 2023
    Do You Remember? Overcoming Catastrophic Forgetting for Fake Audio DetectionXiaohui Zhang, Jiangyan Yi, Jianhua Tao … Chuyuan ZhangICML
    PDF ↗
  3. 2024
    Lookbehind-SAM: k steps back, 1 step forwardGonçalo Mordido, Pranshu Malviya, Aristide Baratin, Sarath ChandarICML
    PDF ↗
  4. 2023
    Learning Expressive Priors for Generalization and Uncertainty Estimation in Neural NetworksDominik Schnaus, Jong‐Seok Lee, Daniel Cremers, Rudolph TriebelICML
    PDF ↗
  5. 2023
    Parameter-Level Soft-Masking for Continual LearningTatsuya Konishi, Mori Kurokawa, Chihiro Ono … Bing LiuICML
    PDF ↗
  6. 2023
    Learnability and Algorithm for Continual LearningGyuhak Kim, Changnan Xiao, Tatsuya Konishi, Bing LiuICML
    PDF ↗
  7. 2023
    Continual Learners are Incremental Model GeneralizersJaehong Yoon, Sung Ju Hwang, Yue CaoICML
    PDF ↗
  8. 2024PDF ↗
  9. 2023
    Continual Learning in Linear Classification on Separable DataItay Evron, Edward Moroshko, Gon Buzaglo … Daniel SoudryICML
    PDF ↗
  10. 2023
    Memory-Based Dual Gaussian Processes for Sequential LearningPaul E. Chang, Prakhar Verma, St. John … Mohammad Emtiyaz KhanICML
    PDF ↗
  11. 2023
    Continual Task Allocation in Meta-Policy Network via Sparse PromptingYijun Yang, Tianyi Zhou, Jing Jiang … Yuhui ShiICML
    PDF ↗
  12. 2023
    Lifelong Language Pretraining with Distribution-Specialized ExpertsWuyang Chen, Yanqi Zhou, Nan Du … C. C. Iuras ̧cuICML
    PDF ↗
  13. 2023PDF ↗
  14. 2023
    BiRT: Bio-inspired Replay in Vision Transformers for Continual LearningKishaan Jeeveswaran, Prashant Bhat, Bahram Zonooz, Elahe AraniICML
    PDF ↗
  15. 2023
    DualHSIC: HSIC-Bottleneck and Alignment for Continual LearningZifeng Wang, Zheng Zhan, Yifan Gong … Jennifer DyICML
    PDF ↗
  16. 2023
    The Ideal Continual Learner: An Agent That Never ForgetsLiangzu Peng, Paris V. Giampouras, René VidalICML
    PDF ↗
  17. 2023
    Does Continual Learning Equally Forget All Parameters?Haiyan Zhao, Tianyi Zhou, Guodong Long … Chengqi ZhangICML
    PDF ↗
  18. 2023
    Prototype-Sample Relation Distillation: Towards Replay-Free Continual LearningNader Asadi, MohammadReza Davar, Sudhir P. Mudur … Eugene BelilovskyICML
    PDF ↗
  19. 2023
    Understanding plasticity in neural networksClare Lyle, Zeyu Zheng, Evgenii Nikishin … Will DabneyICML
    PDF ↗
  20. 2023
    Task-Specific Skill Localization in Fine-tuned Language ModelsAbhishek Panigrahi, Nikunj Saunshi, Haoyu Zhao, Sanjeev AroraICML
    PDF ↗
  21. 2023
    Theory on Forgetting and Generalization of Continual LearningSen Lin, Peizhong Ju, Yingbin Liang, Ness B. ShroffICML
    PDF ↗
  22. 2023
    Exploring the Benefits of Training Expert Language Models over Instruction TuningJoel Jang, Seungone Kim, Seonghyeon Ye … Minjoon SeoICML
    PDF ↗
  23. 2023
    Efficient Parametric Approximations of Neural Network Function Space DistanceNikita Dhawan, Sicong Huang, Juhan Bae, Roger GrosseICML
    PDF ↗
  24. 2023
    Neuro Symbolic Continual Learning: Knowledge, Reasoning Shortcuts and Concept RehearsalEmanuele Marconato, Gianpaolo Bontempo, Elisa Ficarra … Stefano TesoICML
    PDF ↗
  25. 2023PDF ↗
  26. 2022
    Probabilistic Bilevel Coreset SelectionXiaofang Zhou, Renjie Pi, Weizhong Zhang … Tong ZhangICML
    PDF ↗
  27. 2023
    Discrete Key-Value BottleneckFrederik Träuble, Anirudh Goyal, Nasim Rahaman … Bernhard SchölkopfICML
    PDF ↗
  28. 2022PDF ↗
  29. 2022PDF ↗
  30. 2022PDF ↗
  31. 2022
    Continual Learning with Guarantees via Weight Interval ConstraintsMaciej Wołczyk, Karol J. Piczak, Bartosz Wójcik … Przemysław SpurekICML
    PDF ↗
  32. 2022
    StreamingQA: A Benchmark for Adaptation to New Knowledge over Time in Question Answering ModelsAdam Liška, Tomáš Kočiský, Elena Gribovskaya … Angeliki LazaridouICML
    PDF ↗
  33. 2022
    Efficient Test-Time Model Adaptation without ForgettingShuaicheng Niu, Jiaxiang Wu, Yifan Zhang … Mingkui TanICML
    PDF ↗
  34. 2022PDF ↗
  35. 2022
    Controlling Conditional Language Models without Catastrophic ForgettingTomasz Korbak, Hady Elsahar, Germán Kruszewski, Marc DymetmanICML
    PDF ↗
  36. 2022
    Wide Neural Networks Forget Less CatastrophicallySeyed Iman Mirzadeh, Arslan Chaudhry, Yin, Dong … Mehrdad FarajtabarICML · Google (United States)
    PDF ↗
  37. 2021
    Kernel Continual LearningMahammad Mahdi Derakhshani, Xiantong Zhen, Ling Shao, Cees G. M. SnoekICML · University of Amsterdam · Inception Institute of Artificial Intelligence
    PDF ↗
  38. 2021
    Continual Learning in the Teacher-Student Setup: Impact of Task SimilaritySebastian Lee, Sebastian Goldt, Andrew SaxeICML · Microsoft Research (United Kingdom) · Scuola Internazionale Superiore di Studi Avanzati · +1
    PDF ↗
  39. 2021
    Continuous Coordination As a Realistic Scenario for Lifelong LearningHadi Nekoei, Akilesh Badrinaaraayanan, Aaron Courville, Sarath ChandarICML · Centre Universitaire de Mila · Université de Montréal · +1
    PDF ↗
  40. 2021
    GP-Tree: A Gaussian Process Classifier for Few-Shot Incremental LearningIdan Achituve, Aviv Navon, Yochai Yemini … Ethan FetayaICML · Bar-Ilan University
    PDF ↗
  41. 2021
    Online Limited Memory Neural-Linear Bandits with Likelihood MatchingOfir Nabati, Tom Zahavy, Shie MannorICML · Technion – Israel Institute of Technology · Google DeepMind (United Kingdom)
    PDF ↗
  42. 2020
    Neural Topic Modeling with Continual Lifelong LearningPankaj Gupta, Yatin Chaudhary, Thomas A. Runkler, Schütze, HinrichICML · Siemens (Germany) · Technical University of Munich · +2
    PDF ↗
  43. 2020
    Optimal Continual Learning has Perfect Memory and is NP-hardJeremias Knoblauch, Hisham Husain, Tom DietheICML · University of Warwick · The Alan Turing Institute · +2
    PDF ↗
  44. 2021
    Variational Auto-Regressive Gaussian Processes for Continual LearningSanyam Kapoor, Theofanis Karaletsos, Thang D. BuiICML · Supélec · University of Applied Sciences and Arts of Southern Switzerland · +3
    PDF ↗
  45. 2021
    Addressing Catastrophic Forgetting in Few-Shot ProblemsPauching Yap, Hippolyt Ritter, David BarberICML · University College London
    PDF ↗
  46. 2021
    Federated Continual Learning with Weighted Inter-client TransferJaehong Yoon, Wonyong Jeong, Giwoong Lee … Sung Ju HwangICML · Korea Advanced Institute of Science and Technology · Korea Institute of Science and Technology
    PDF ↗
  47. 2021
    Overcoming Catastrophic Forgetting by Bayesian Generative RegularizationPatrick H. Chen, Wei Wei, Cho‐Jui Hsieh, Bo DaiICML · University of California, Los Angeles
    PDF ↗
  48. 2019
    Single-Net Continual Learning with Progressive Segmented TrainingXiaocong Du, Gouranga Charan, Frank Liu, Yu CaoICML · Arizona State University · Oak Ridge National Laboratory
    PDF ↗
  49. 2019
    Frosting Weights for Better Continual TrainingXiaofeng Zhu, Feng Liu, Goce Trajcevski, Dingding WangICML · Northwestern University · Florida Atlantic University · +1
    PDF ↗
  50. 2019
    Hierarchically Structured Meta-learningHuaxiu Yao, Ying Wei, Junzhou Huang, Zhenhui LiICML · Pennsylvania State University
    PDF ↗
About this index

We keep this list because we read the field and wanted one place to see it. It covers work on continual learning itself, in the core areas of machine learning, and leaves out papers that apply it inside another field, such as medical imaging or fault diagnosis. It lists only papers we have a reason to trust: published at a venue like NeurIPS, ICML, ICLR, CVPR or TPAMI, or led by someone who has published there, or cited a few hundred times, and only papers with a PDF we can point you at, so every title opens the paper itself. Preprints that later get accepted, and authors who later publish at those venues, are picked up by the weekly run. It is seeded from the community lists kept by ContinualAI and by Xialei Liu, then filled out from OpenAlex, and every week a script looks for new papers on OpenAlex and arXiv. A model reads each candidate and decides whether it belongs; a person reviews the additions before they go live. Authors and affiliations come from OpenAlex, so a recent preprint can lack its institutions for a week or two.

Missing something, or filed under the wrong venue? Write to hello@unify.ai with the arXiv id or DOI.