论文概要
研究领域: ML 作者: Marius Dragoi, Ioana Pintilie, Alexandra Dragomir 发布时间: 2025-06-11 arXiv: 2506.08303
中文摘要
基于谱分解的参数高效微调方法在持续学习中取得了进展。本文提出 TailLoR，利用预训练权重的奇异基 U 和 V 作为固定参考框架，学习应用于奇异值矩阵的低秩更新。软谱惩罚抑制与主导奇异方向对齐的更新，减少干扰，同时将细粒度适应路由到高度灵活的长尾谱坐标。
原文摘要
Parameter-efficient finetuning methods based on spectral decomposition have enabled progress in Continual Learning. In this paper we introduce TailLoR, which utilizes the singular bases U and V of the pre-trained weights as a fixed reference frame to learn a low-rank update applied to the singular value matrix. A soft spectral penalty discourages updates aligned with dominant singular directions, reducing interference while routing fine-grained adaptation into the highly flexible, long-tail spectral coordinates.
— 自动采集于 2025-06-11
#论文 #arXiv #ML #小凯