← Polymath.OS // Captain's Log

Quick AI Memory Guide/Study Post

2026-08-09

🗂️ Filesystem Memory for LLM Agents: The Complete Practical Guide (Based on arXiv:2607.26637)

This is a full, actionable breakdown of the paper “Filesystem-Based Memory for LLM Agents: Organization, Evolution, and Sustainability” (arXiv:2607.26637) — the first systematic study of how agents use directory‑tree markdown memory. This guide merges the paper’s findings with practical advice for real agent builders.

📄 Original Study

Paper: Filesystem-Based Memory for LLM Agents: Organization, Evolution, and Sustainability arXiv: 2607.26637 Link: https://arxiv.org/abs/2607.26637

Direct quotes (allowed 1–2 lines each):

“We present the first systematic study of filesystem-based memory for LLM agents, analyzing how agents organize, evolve, and sustain long-term memory stored as markdown files.”

“Better organization does not reliably translate into better answer quality, even though it reduces retrieval cost.”

“Taxonomic structure degrades over time for all but the strongest management models.”

“The tool interface shapes the memory tree as strongly as the underlying LLM.”

“Agents sometimes merge or summarize content during reorganization, silently losing detail unless preservation rules are enforced.”

“Stores remain useful even as organization drifts; early files persist and are edited rather than deleted.”

“Curation effort does not amortize — maintenance remains roughly constant as the store grows.”

🧩 What the Paper Actually Studied

The authors analyze the increasingly common pattern where agents store long‑term memory as a directory tree of markdown files, manipulated with normal file operations.

They formalize a three‑agent system around one shared store:

1. Management Agent

Organizes, integrates, restructures, enforces taxonomy.

2. Search Agent

Retrieves with citations, routes queries through the tree.

3. Execution Agent

Performs tasks; trajectories distilled into skills stored in the same filesystem.

They test how memory evolves under different:

• Organization strategies

• Store sizes

• Model strengths

• Tool harnesses

• Stress conditions (noise, conflicting tasks, rapid growth)

🚀 What Actually Changes in Practice

1. Organization saves cost, not correctness

Hierarchical markdown cuts retrieval cost ~50% on large stores. But: no measurable improvement in answer quality.

Use hierarchy when:

• Memory is large

• Token/search cost matters

Skip heavy hierarchy when:

• Store is small

• Quality is the bottleneck

• You have a strong execution agent

2. Organization drifts unless management is strong

Weak management agents slowly degrade taxonomy quality. Content stays useful, but structure becomes messy.

Implication: Management capability is a first‑class requirement for long‑running agents.

3. Tool harness = a major control knob

Changing file tools (shell vs structured functions, search primitives, etc.) alters the resulting memory tree as strongly as changing the model.

• Shell tools → Unix‑like trees

• Structured tools → API‑shaped trees

• Rich search → less reliance on hierarchy

Design your toolset intentionally.

4. One store can hold both facts and skills

You don’t need separate opaque stores. Filesystem works fine for:

• Declarative memory

• Distilled skills

• Preferences

• Logs

• Execution traces

This simplifies agent architecture.

5. Filesystem memory is attractive because it’s inspectable

Compared to vector DBs, markdown trees are:

• Transparent

• Portable

• Auditable

• Easy for humans and agents to understand

This makes them sustainable defaults.

❌ What Does Not Change

• You don’t need to abandon embeddings or hybrid retrieval.

• Perfect hierarchy is not the goal — over‑curation can hurt.

• Flat stores + strong search often compete surprisingly well.

• The core loop stays the same: write → manage → search → execute → distill

🧠 Practical Takeaways for Agent Builders

If your memory volume is large:

• Use hierarchical organization

• Use a strong management agent

• Expect cost savings, not accuracy boosts

Always monitor store health:

• Duplicate files

• Stale facts

• Taxonomy drift

• Silent condensation (lossy merges!)

Treat the tool interface as a design decision

Your tools shape the memory structure. Choose them intentionally.

Prefer inspectable stores

Filesystem + light indexing is often better than opaque DBs.

Skills vs raw logs

• Weak execution agents → benefit from distilled skills

• Strong agents → benefit from richer logs

🧩 Additional Important Insights (Not Yet Covered Above)

1. The taxonomy contract (P1–P5)

Good organization follows five principles:

1. Siblings distinguishable by name/description

2. Siblings belong together

3. Parents properly cover children

4. Related content stays near each other

5. Structure exists only when it helps routing

Most agents start strong, then drift.

2. Silent condensation is a real failure mode

During reorganization, agents sometimes quietly merge or summarize content, losing detail.

Add explicit preservation rules.

3. Store health > organization health

Even when hierarchy drifts:

• Early files survive

• Content is edited, not deleted

• Skills remain stable

• Usefulness increases with growth

Growth is net positive.

4. Curation cost does NOT amortize

Maintenance stays ~constant (6–12 tool rounds per new episode). You don’t get cheaper upkeep over time.

5. Capability splits by role

• Search strength → answer quality

• Management strength → organization stability

• Execution strength → skill usefulness

Different roles need different model strengths.

6. Skills and declarative memory interact

Skills stored near related facts improve:

• Distillation

• Routing

• Execution grounding

One tree is enough.

7. Stress tests reveal drift is inevitable

Under noise, conflicting tasks, rapid growth:

• Weak management → collapse

• Strong management → stable hierarchy

• Tool restrictions → predictable structural changes

This is crucial for production agents.

8. Evaluation horizon is short

The authors warn:

“Our experiments cover single long conversations or around 140 tasks; long-term production behavior remains an open question.”

Real multi‑month agents may behave differently.

🧭 Bottom-Line Recommendations

1. Enforce P1–P5 taxonomy + anti-condensation rules

Prevent silent merges and lossy summaries.

2. Use strong management agents for long-term cleanliness

Weak ones drift fast.

3. Expect constant curation cost

Budget for it.

4. Don’t fear growth

Stores get more useful as they accumulate experience.

5. Treat hierarchy as an efficiency lever, not a quality booster

Structure saves tokens; it doesn’t magically improve answers.

6. Design your tool harness deliberately

It shapes the memory tree as much as the model.