Learning

I Gave Claude Code a Super Memory with Supabase

How I built a semantic memory system that connects what I've learned, what I'm working on, and where everything is heading. Supabase pgvector, Voyage AI, and 4 interfaces to 1 brain.

March 16, 202611 min297 views

At some point, when you work with AI tools every day, you stop wanting them to just answer questions. You want them to learn with you. To evolve with your projects. To remember the decisions you made last Tuesday and the person who helped you figure it out.

That's where I've been for a while now. All of my projects have context. I've managed that context carefully - CLAUDE.md files, case logs, project documentation, voice guides. I wrote about this a few weeks ago on LinkedIn in a piece called "Own Your Context," because I genuinely believe the context you build around your tools is what makes them yours.

But I wanted to take it further. I wanted my tools to understand not just what I'm doing right now, but what I've done before and where I'm heading.

What I was really after

This wasn't about saving tokens or optimizing for speed (though both happened). What I really wanted was:

  • The best possible responses for what I actually need. Answers shaped by everything I've learned over months
  • Better control over all my projects. Knowing exactly where I left off, what decisions were made, and why
  • A memory that doesn't live only on my machine. A persistent backup I can reference from anywhere
  • Something that works for any agent. Not just Claude Code, but any LLM or tool I use in the future

I use Claude Code for almost everything. But I also need it for things that require remembering people, tracking the last moment I left something, managing progress across a dozen projects. I use MCPs heavily, especially Linear, where I leave detailed progress notes and pick things up depending on where I left off. And I work with parallel agents a lot, which means context needs to be accessible from multiple places at once.

The memory system couldn't be a Claude Code-only thing. It had to be broader.

The context I already had

As part of Velora, my system for tracking efficiency gains, I dedicate time every week to improving my Claude Code workflow. We've explored what my usage patterns look like. We've built custom skills, automation scripts, and MCP integrations.

At this point, almost everything runs through Claude Code and MCPs (integrations that let the AI talk directly to external services from the terminal):

  • Supabase MCP. Database, schema changes, security policies
  • Linear MCP. Project tracking, detailed progress notes
  • Gmail + Calendar MCPs. Email triage, scheduling
  • Slack MCP. Community communication
  • Resend MCP. Email campaigns
  • Custom scripts. Morning setup, check-ins throughout the day
  • GitHub, ClickUp. Project-specific workflows

Even the articles I write for my blog come from what I do daily with Claude Code. Everything is connected.

But what I didn't have was a systematic process where knowledge saved itself. At the beginning I used MEMORY.md and CLAUDE.md. That worked for a while. But as months passed, I had hundreds of pieces of knowledge with no way to search by meaning, no timestamps, no automatic organization.

Keyword search vs semantic search: scattered documents vs connected knowledge

Why I built it myself

I could have used an existing solution. There are memory layers, plugins, services. But I didn't want something pre-built. I wanted to understand it from scratch.

I found a series by Christian Loschiavo called "Building LLM Memory from Scratch" that walks through different approaches: sliding-window buffers, vector memory, knowledge graph memory, and hierarchical self-managed memory. That series helped me understand *why* I needed a vector database, where the limits are, and how far I could take this.

I wanted to understand the decisions. Why vectors over keywords? Why cosine similarity? How far can pgvector scale? I know I can take this much further, but I deliberately stopped here to use it for a week before advancing to the next layer.

Building on what Claude Code already has

Claude Code already has a memory system. MEMORY.md stores notes that are always loaded - my preferences, my tone, key patterns. It's like a sticky note the AI always sees.

I didn't replace that. I built on top of it.

The built-in memory tells Claude who I am. The Supabase layer tells it what we've done together - 540+ memories of cases solved, decisions made, patterns discovered. Without it, every session starts with a coworker who read a one-page briefing. With it, that coworker can search through eight months of shared work.

Why Supabase

I already run everything on Supabase - blog, portfolio, project data. When I needed a place for searchable memories, I didn't want another service.

  • All my projects already live there
  • My future agents could feed from it when they start a session
  • Adding memory integrates naturally with everything else
  • I wanted a backup that's not just on my machine

Supabase supports pgvector - a Postgres extension that adds vector storage and similarity search to your existing database. No separate service needed. My memories live next to my blog posts. One platform.

Anthropic's own research on context engineering confirms why this matters: "context degradation is the primary failure mode" of tools like Claude Code. When your AI stops being useful, it's usually the context, not the model.

How it works

Here's the full architecture:

System architecture

Cloud + local redundancy

Claude Code

Terminal AI assistant

MCP Server

Node.js bridge

Supabase

pgvector + memories

Local Files

Markdown backup

Voyage AI

Embeddings

Primary
Backup

And here's what happens when I search:

How semantic search works

~500ms per search

Query

Natural language

Voyage AI

Generates embedding

Vector

1,024 dimensions

Supabase

Cosine similarity

Example query

"user can't log in with Google, gets 403"

Results

Google Auth 403: check redirect URI + OAuth consent screen0.91
Case: OAuth redirect mismatch on custom domain0.87
Pattern: white screen after login (token in URL hash)0.82

The core idea: an embedding model converts text into 1,024 numbers representing meaning. Two sentences about the same thing get similar numbers even with completely different words. That similarity is measured with cosine similarity (0 = completely different, 1 = identical meaning).

I use Voyage AI's voyage-3.5-lite (Anthropic recommends it, 200M free tokens/month). Vectors go into Supabase via pgvector. Search takes ~500ms in any language.

I designed this with general endpoints and Edge Functions so that any agent or LLM can use it, not just Claude Code.

Past + Present + Plan

This is the part that makes everything click.

The memory system doesn't work alone. It works because everything is connected:

  • Memory (Supabase) = the past. Every decision, pattern, solution, and lesson learned
  • Linear / ClickUp (MCPs) = the present. What's in progress, what's blocked, what just shipped
  • The long-term plan = where we're going. The roadmap, the vision, the next steps
Past, present, and plan: a journal for what you learned, a compass for where you are, a telescope for where you are heading

Most tools give you one of these. A project manager shows the present. A knowledge base shows the past. But rarely do you get all three connected so that your AI understands not just what you're doing, but why and where it fits in the bigger picture.

A real example: there's a task in Linear called CAR-80, "Sync feedback memories to MEMORY.md." Without memory, that's just a task title. With it, I can search and find the full context - we created it because preferences in Supabase don't automatically influence Claude's behavior. The problem, the reasoning, and the solution are all searchable. And because it's in Linear, I know its priority and how it connects to everything else.

The plan layer means the AI can understand trajectory. Not just "what did Carol do last week" but "where is all of this heading." That's the difference between a tool that helps you do things and a tool that helps you build toward something.

4 interfaces, 1 brain

I didn't build this for just one tool. The same memory is accessible from anywhere:

  • Claude Code. search_memory MCP native tool (fastest, most integrated)
  • Marcus (my OpenClaw agent). memory search "query" CLI command
  • Automation. Cron jobs and shell scripts calling the memory directly
  • Any agent. HTTP Edge Functions on Supabase (universal, no setup needed)
Four interfaces, one brain: CLI, MCP, HTTP, and automation all connecting to the same memory

The knowledge doesn't live in any one tool. It lives in Supabase, and everything connects to it.

What it looks like in practice

Cross-language search

I think in Spanish, my docs are in English, and the search just works. "User can't log in with Google" finds the same results as "usuario no puede iniciar sesion con Google." And because all projects share the same memory, a pattern from Festie can surface when I'm working on something completely different.

Picking up where I left off

Switching from carolmonroe.com back to Festie after two weeks:

search_memory: "Festie Spotify integration status"

[0.92] Festie - Spotify stage integration
       Decision: use Spotify Web API for stage playlists.
       Blocker: rate limits on playlist creation.
       Status: API key approved, waiting on OAuth flow.

[0.85] Spotify API rate limiting workaround
       Use exponential backoff with 429 status checks.

Two results. Full context. 500ms. No re-reading anything.

Deduplication

Without it, memory bloats. Before every insert, the system checks similarity: above 0.95 gets blocked as duplicate, below passes through. Tested with real data - exact same text scores 1.000 (blocked), rephrasing scores ~0.91 (allowed).

Redundancy

Every save writes to Supabase (semantic search) and local markdown files (always on my machine). Neither depends on the other. A morning sync script reconciles them.

Under the hood

  • 540+ memories across cases, patterns, references, projects, and preferences
  • Sub-agents. Start fresh by design - I built workarounds: context injection files, custom agent definitions with MCP access, and Edge Function endpoints
  • Performance. Sequential scan under 10ms at this scale. At ~2,000 memories, I'll add an HNSW index
  • Technical path. I'm becoming more technical by design. I'd already built AI assistants and worked with embeddings, but I wanted to understand the *decisions*, not just use the tools

What comes next

  • Knowledge graph. The next layer from Loschiavo's series. Mapping relationships between memories, not just storing them
  • Marcus integration. My personal AI agent (self-hosted, Anthropic's Opus). Same brain, different interface. The CLI is ready
  • HNSW indexing. At ~2,000 memories
  • Automatic preference sync. Preferences shaping behavior automatically
  • Memory health. Tracking usage, staleness, pruning

One step at a time. Use it for a week. See what works. Then advance.

The bigger picture

For months, my knowledge lived in .md files and CLAUDE.md instructions. It worked. But it was fragile - tied to one tool, hard to search, impossible to share across agents.

This is the shift: going from saving artifacts in a text file to building something solid underneath. A real database, with semantic search, accessible from any tool I use today or whatever comes next.

That's what this project is really about. Not the vector database or the embeddings - those are implementation details. It's about making sure that everything I learn, every pattern I find, every decision I make has a place to live that outlasts any single tool.

Whatever comes next, the foundation is there.

Past, present, plan.

Enjoyed this?

Carol Ships: building, shipping, figuring it out.

Have another workaround to share?

Start the thread below!

Comments

No comments yet. Be the first to share your thoughts!