因果格拉斯曼序列建模

因果格拉斯曼序列建模架构


友情链接: 借一步  背多分   ACEJoy


 

:root {
–primary-color: #1E3A8A; / Deep Blue /
–secondary-color: #3B82F6; / Bright Blue /
–accent-color: #60A5FA; / Light Blue /
–bg-color: #F0F4F8;
–card-bg: #FFFFFF;
–text-main: #1F2937;
–text-secondary: #4B5563;
–gradient-main: linear-gradient(135deg, #1E3A8A 0%, #2563EB 100%);
}

* {
margin: 0;
padding: 0;
box-sizing: border-box;
}

body {
font-family: ‘Noto Sans SC’, ‘Roboto’, sans-serif;
background-color: #E2E8F0;
display: flex;
justify-content: center;
color: var(–text-main);
line-height: 1.5;
}

.poster-container {
width: 720px;
min-height: 960px;
background-color: var(–bg-color);
position: relative;
overflow: hidden;
box-shadow: 0 10px 25px rgba(0,0,0,0.1);
}

/ Geometric Background Decoration /
.bg-shape {
position: absolute;
border-radius: 50%;
opacity: 0.1;
z-index: 0;
}
.shape-1 {
width: 400px;
height: 400px;
background: var(–secondary-color);
top: -100px;
right: -100px;
}
.shape-2 {
width: 300px;
height: 300px;
background: var(–primary-color);
bottom: -50px;
left: -50px;
}

/ Header /
header {
background: var(–gradient-main);
color: white;
padding: 40px 30px;
position: relative;
z-index: 1;
clip-path: polygon(0 0, 100% 0, 100% 85%, 0 100%);
}

h1 {
font-size: 36px;
font-weight: 900;
margin-bottom: 10px;
letter-spacing: -0.5px;
}

h2 {
font-size: 20px;
font-weight: 400;
opacity: 0.9;
margin-bottom: 15px;
}

.citation {
font-size: 12px;
font-family: ‘Roboto’, sans-serif;
background: rgba(255,255,255,0.2);
padding: 4px 8px;
border-radius: 4px;
display: inline-block;
}

/ Main Content /
.content {
padding: 20px 30px 40px;
position: relative;
z-index: 1;
display: flex;
flex-direction: column;
gap: 20px;
}

.card {
background: var(–card-bg);
border-radius: 12px;
padding: 20px;
box-shadow: 0 4px 6px rgba(0,0,0,0.05);
border-left: 4px solid var(–secondary-color);
}

.card-title {
font-size: 18px;
font-weight: 700;
color: var(–primary-color);
margin-bottom: 12px;
display: flex;
align-items: center;
gap: 8px;
}

.card-title i {
font-size: 20px;
}

p {
font-size: 14px;
color: var(–text-secondary);
text-align: justify;
margin-bottom: 10px;
}

/ Problem vs Solution Comparison /
.comparison-grid {
display: grid;
grid-template-columns: 1fr 1fr;
gap: 15px;
margin-top: 10px;
}

.comp-item {
background: #F8FAFC;
padding: 12px;
border-radius: 8px;
border: 1px solid #E2E8F0;
}

.comp-header {
font-weight: 700;
font-size: 14px;
margin-bottom: 5px;
display: flex;
justify-content: space-between;
}

.comp-desc {
font-size: 12px;
}

/ Architecture Flow /
.flow-container {
display: flex;
align-items: center;
justify-content: space-between;
background: #EFF6FF;
padding: 15px;
border-radius: 10px;
margin-top: 5px;
}

.flow-step {
display: flex;
flex-direction: column;
align-items: center;
text-align: center;
flex: 1;
position: relative;
}

.step-icon {
width: 36px;
height: 36px;
background: var(–primary-color);
color: white;
border-radius: 50%;
display: flex;
align-items: center;
justify-content: center;
font-size: 18px;
margin-bottom: 5px;
z-index: 2;
}

.step-text {
font-size: 12px;
font-weight: 600;
color: var(–primary-color);
}

.step-sub {
font-size: 10px;
color: var(–text-secondary);
}

.arrow {
color: var(–secondary-color);
font-size: 20px;
flex: 0 0 20px;
text-align: center;
}

/ Mathematical Highlight /
.math-box {
background: #1E3A8A;
color: white;
padding: 10px 15px;
border-radius: 6px;
font-family: ‘Roboto’, sans-serif;
font-style: italic;
text-align: center;
margin: 10px 0;
font-size: 13px;
}

/ Results Grid /
.results-grid {
display: grid;
grid-template-columns: 1fr 1fr;
gap: 15px;
margin-top: 10px;
}

.result-item {
text-align: center;
background: #F0FDF4;
border: 1px solid #BBF7D0;
padding: 10px;
border-radius: 8px;
}

.result-value {
font-size: 20px;
font-weight: 700;
color: #166534;
margin-bottom: 2px;
}

.result-label {
font-size: 11px;
color: #15803D;
text-transform: uppercase;
letter-spacing: 0.5px;
}

/ Features List /
.feature-list {
list-style: none;
}

.feature-list li {
display: flex;
align-items: flex-start;
margin-bottom: 8px;
font-size: 13px;
}

.feature-list li i {
color: var(–secondary-color);
font-size: 16px;
margin-right: 8px;
margin-top: 2px;
}

/ Footer /
footer {
background: white;
padding: 20px;
text-align: center;
font-size: 10px;
color: #9CA3AF;
border-top: 1px solid #E5E7EB;
}

/ Decorative Elements /
.math-symbol {
position: absolute;
font-family: ‘Times New Roman’, serif;
font-style: italic;
color: rgba(30, 58, 138, 0.05);
font-weight: bold;
pointer-events: none;
}

Gr(2,r)

因果格拉斯曼序列建模

Causal Grassmann Sequence Modeling

挑战自注意力机制:通过几何流形构建更具可解释性且高效的深度学习框架。

Source: Attention Is Not What You Need (arXiv:2512.19428)

psychology_alt
核心变革:从张量到流形

传统Transformer的”不可解释性”源于其复杂的张量提升过程(Tensor Lifting)。新架构将词元状态视为低维流形上的几何对象。

传统 Attention
grid_on

• 二次方复杂度 O(L²)

• 成对交互过于密集

• 难以追踪数学规律

Grassmann Flow
waves

• 线性复杂度 O(L)

• 局部几何流形映射

• 显式几何不变性

architecture
架构原理:格拉斯曼混合层

通过普吕克坐标(Plücker coordinates)捕捉局部几何特征,信息在低秩子空间中流动。

Input H ∈ ℝL×d → Low-dim Z ∈ ℝL×r → Gr(2, r) Manifold

compress

线性降维

Linear Reduction

arrow_forward

link

多尺度配对

Multi-scale Pair

arrow_forward

share

普吕克编码

Plücker Embed

arrow_forward

merge_type

门控融合

Gated Fusion

verified
核心优势

  • speed

    线性计算复杂度

    避开注意力机制的 O(L²) 成本,与序列长度呈线性比例,适合长序列建模。

  • insights

    显式几何不变性

    模型在有限维流形(Grassmannian)上操作,便于数学分析和解释,不再”不可追踪”。

  • memory

    高效的信息流

    通过低秩子空间的受控变形传播信息,而非简单的权重加权。

bar_chart
实验表现:媲美甚至超越 Transformer

在语言建模和自然语言推理任务上,该架构表现出极强的竞争力。

10-15%

Wikitext-2 困惑度差距
(更接近基线)

85.5%

SNLI 准确率
(略优于基线)

* 特定分类任务中表现略胜一筹

Attention Is Not What You Need: Grassmann Flows as an Attention-Free Alternative for Sequence Modeling

Designed based on arXiv:2512.19428

留下评论

人生梦想 - 关注前沿的计算机技术 acejoy.com 🐾 步子哥の博客 🐾 背多分论坛 🐾 借一步网 沪ICP备2024052574号-1