Tom Liu

Building Your Second Brain using Claude Code and Obsidian - Part 1

2026-05-06T00:00:00+00:00

The bottleneck in my learning was never finding good content - it was the grunt work after reading it - summarizing, connecting ideas across articles, filing things where I’d actually find them again. I read technical blogs constantly, but I just… didn’t do any of that. For years, knowledge entered, sparked something, and faded - scattered across browser tabs and half-started notes I’d never reopen.

Then I came across Andrej Karpathy’s tweet and something clicked. What if the LLM didn’t just answer questions about my documents - what if it transformed them into structured knowledge? A persistent wiki where concepts are linked, connections are surfaced, and rationale is made explicit. From there, I could drill down into the relationships between ideas, and bring my own lens to deep-dive into the topics that matter to me.

The idea is awesome; however, Andrej didn’t share ‘step-by-step’ guidance. Others have reproduced his method in different forms — using the llm-wiki gist as a system prompt, creating reusable skills, even building apps. I wasn’t happy with the results, so I decided to build my own approach using Claude Code and Obsidian.

Here’s a sneak peek before we dive in. Say you’re reading Anthropic’s engineering blog - Building effective agents, what if you could transform it into a wiki and navigate the key parts using Obsidian:

Andrej’s llm-wiki

The purpose of llm-wiki is to build personal wiki knowledge bases for topics you care about. It has three flows: ingest (transform documents into structured wiki pages), query (ask questions with citations), and maintenance (keep the wiki healthy over time). This post covers ingest, Obsidian and Obsidian Web Clipper setup. The rest will be covered in Part 2.

Here’s a reference architecture:

How It Works

The whole process standardizes the wiki generation, from flat documents to structured knowledge. I built a plugin so you can do this repeatably.

Here’s what the pipeline looks like conceptually:

To build the ingest process, the handy way is to use an AI coding agent (Claude Code, Kiro, Codex, etc.) with Andrej’s recipe.

Luckily, Andrej shared his recipe - llm-wiki gist. The simplest approach: paste the gist into Claude Code and point it at your source document. This works, but doesn’t guarantee consistent quality and ties you to a single session. To make it repeatable, I built a plugin - llm-wiki plugin - that encodes the pipeline as reusable skills I can refine over time.

Try It

The llm-wiki plugin is ready to use. Here’s the process step by step:

#Initial working folder setup
#Porting web page(s) to Obsidian
#Transforming flat documents to wiki
#Navigate wiki with Obsidian

Prerequisites: Obsidian installed, Obsidian Web Clipper extension installed on browser (e.g. Chrome), Claude Code installed (with subscription or using API), terminal familiarity.

Initial working folder setup

Open a terminal and create the working folder:

mkdir -p test/raw/assets  

This gives you the folder structure:

test/
  ├── raw/                 # Source documents (articles, assets)
  │   └── assets/          # Images referenced by source docs

Next, download a blog (e.g. Building effective agents) into the raw folder, and save related images into ‘raw/assets’. The next section shows how to do this with Obsidian Web Clipper.

For reference, here’s what the folder looks like after ingestion is complete:

test/
  ├── .claude/              # Claude Code configuration
  │   ├── settings.json
  │   └── skills/active-learning/
  ├── .llm-wiki/            # Wiki system config
  │   └── lenses/
  ├── .obsidian/            # Obsidian vault settings
  ├── wiki/                 # Structured knowledge base
  │   ├── index.md
  │   ├── overview.md
  │   ├── log.md
  │   ├── concepts/        # Core concepts (augmented LLM, agentic systems, etc.)
  │   ├── techniques/      # Implementation patterns (routing, parallelization, etc.)
  │   ├── synthesis/       # Cross-cutting analyses
  │   └── entities/
  ├── raw/                 # Source documents (articles, assets)
  │   ├── Building Effective AI Agents.md
  │   └── assets/          # Images referenced by source docs

Porting web page(s) to Obsidian

Follow the steps to configure Obsidian & Web Clipper extension:

On Obsidian, create an Obsidian vault with ‘Open folder as vault’ and select the ‘test’ folder
On Obsidian, configure a hotkey to download attachments. When you clip a note, this lets you download its images as local attachments for the wiki.
- Select ‘Setting’ (On macOS, it’s under top-left ‘Obsidian’ menu)
- Select ‘Hotkeys’ menu, and input ‘attachment’ in the filter at the panel
- Click the ‘+’ button at the item ‘Download attachments for the current file’, then press ‘Cmd’ + ‘Shift’ + ‘D’ (or your preference.)
On Obsidian Web Clipper extension setting, configure vault settings for note import.
- Under ‘General’, add the newly created vault ‘test’ (when creating vault on folder, by default the name will be the folder name.) into ‘Vaults’ section, and select ‘Add to Obsidian’ in ‘Save behavior’
- Configure the vault:
  - Input ‘raw’ for the ‘Note location’ - match to the raw folder under your vault
  - Select ‘test’ from ‘Vault’
Open your target web page, e.g. Building effective agents on Chrome, then click ‘Obsidian Web Clipper ‘, or press ‘Cmd’ + ‘Shift’ + ‘O’
Press the ‘Add to Obsidian’ button, Obsidian fires up automatically and you can view the document. At the moment, the images in the note are backed by URLs. To download the images, use the hotkey for ‘Download Attachments for the current file’ - pressing ‘Cmd’ + ‘Shift’ + ‘D’ (per your setup before). Once you confirm and click the ‘Download’ button, all the images will be saved under ‘assets’ folder

Transforming flat documents to wiki

Now that we have the ingredients - “Building Effective AI Agent” note and images under the ‘raw’ folder, we are ready to kick off the pipeline. Open a terminal in the ‘test’ folder and run claude, and install my llm-wiki plugin to ingest documents.

To set up the environment, run these commands on Claude Code, or refer to the screen dump.

# register the plugin 
/plugin marketplace add tom5610/llm-wiki

# install the plugin 
/plugin install llm-wiki@llm-wiki-plugins

# reload the plugin so that the skills could be loaded
/reload-plugins

# [optional] use 'max' effort
/effort max

# about model, highly recommend 'Opus' / 'Sonnet'
# /model

Reference Claude Code terminal

Once the llm-wiki skill(s) are ready, use the ingest skill to transform the document to wiki.

Run this in Claude Code:

/ingest "raw/Building Effective AI Agents.md"

What happens during ingest:

The ingest skill standardizes the transformation pipeline. Claude Code will think and plan between steps.
When starting from scratch, Claude Code will instruct to create the related folders under ‘wiki’, and ask for permission before doing so. For file editing, press ‘shift + tab’ to allow all.
There are 5 steps in the process:
- Read & Comprehend
- Discuss with User (build wiki by lens?)
- Plan the Transform
- Write and Integrate
- File & Report

Reference process details:

Navigate wiki with Obsidian

Once wiki creation is done, open the ‘test’ vault in Obsidian to navigate your knowledge base.

What does a generated wiki page look like? Here’s a sample concept page generated from the “Building Effective Agents” article. Notice how the LLM extracts a focused concept, adds cross-references via wiki-links, and cites the source:

And here’s the full graph view - each node is a concept page, each edge a wiki-link:

What’s Next

You now have a working pipeline: clip an article, run /ingest, and browse structured knowledge in Obsidian. One article in, a connected wiki out - in about 4-8 minutes.

A few things to try from here:

Ingest 2-3 more articles on a related topic and watch the cross-references multiply
Open Obsidian’s graph view and look for unexpected connections between concepts
Ask questions against your wiki using the query skill (covered in Part 2)

Part 2 will cover:

Query flow - asking questions against your wiki with citations
Feeding insights back - promoting good answers into new wiki pages
Maintenance - keeping your wiki healthy as it grows (orphan detection, missing links, stale pages)
Extract insights - extract knowledge / insights for your downstream tasks

The pipeline does the heavy lifting, but the learning still happens when you decide which connections matter, which concepts to dive deeper on, and what lens to bring. The tool builds the scaffolding - you build the understanding.

Reference Resources

llm-wiki plugin - my curated llm-wiki skills
llm-wiki gist - Andrej Karpathy’s llm-wiki recipe
Building effective agents

Building Clip Buddy with Claude Code

2026-04-06T00:00:00+00:00

This blog shares my experience building Clip Buddy, backed by Lenny and Boris’s awesome podcast — Head of Claude Code: What happens after coding is solved - Boris Cherny.

Clip Buddy takes any video URL, extracts the audio, transcribes it, and produces structured analysis — key arguments, speaker identification, and an interactive chat for deep-diving into specific topics.

My Problem

One day, my account manager threw me a customer CEO’s podcast link, talking about their growth journey, business strategies, and technical challenges. I knew I had to listen. This was critical context for understanding their trajectory and potential needs. But between travel schedules and competing priorities, I knew I wouldn’t give it the deep attention it deserved.

I started thinking why not use AI to back me up, help me navigate the listening / watching journey, and coach me to deep-dive into clips collaboratively. That would be ideal, but I couldn’t find any tools doing exactly that. Hence, I wanted to build one, maybe with Claude Code?

The Engineer’s Paradox: 17+ Years of “Proper” Engineering

But I believe in craftsmanship. I have 17 years of backend development experience. I built systems the “right way” — choosing language(s) carefully, selecting tech stacks thoughtfully, designing POCs methodically. I knew these choices would paralyze me — and they did.

I had the idea. I had the outcome I wanted. But the process felt like climbing a mountain just to start.

I knew Claude Code may be useful, but I was so reluctant to give it a shot because of my concerns:

Would it rob me of the joy of building?
Would I end up shipping something I didn’t actually understand?
Would I just become a prompt engineer rather than a real engineer?

Then I heard Boris talk about the printing press analogy (FYI — reference clip timestamp).

The Turning Point: The Printing Press Analogy

Boris described how the printing press transformed literacy:

Before:
Literacy was ~1% globally (mostly scribes who manually copied texts)

After:
Literacy expanded to ~70% in about 50 years

But here’s the part that struck me: the printing press didn’t make scribes obsolete. It freed them from manual copying so they could focus on craft — on thinking about what they were copying rather than just copying it.

A scribe might have worried: “Will the printing press steal my job? Will I stop being a craftsman?”

The answer was no. Scribes who adopted the printing press became more valuable, not less. They could focus on editing, translating, designing beautiful layouts — the meaningful work. As Boris put it, “in the 50 years after the printing press was built, there was more printed material created than in the 1,000 years before.” (reference clip timestamp) The tool didn’t diminish output — it multiplied it.

Something shifted in my mind.

Maybe Claude Code wasn’t about speed. Maybe it was about democratization freeing me to stop worrying about boilerplate & tech stack selection, and start focusing on thinking — on the architecture, the user experience, the learning.

If scribes could use the printing press to free time from manual copying and focus on craft, then maybe I could use Claude Code to free time from tedious implementation and focus on building something I understand deeply.

I decided to try.

The Workflow: Three Steps That Changed Everything

In the podcast, Boris emphasized always using the most capable model, not boxing the model in, and leveraging Plan Mode. In addition, he envisioned a near-term future where traditional role boundaries dissolve — ‘software engineer’ gives way to ‘builder’.

Could a tool really shift that identity? Here’s the workflow that changed my mind.

Step 1: Start with Outcome, Not Specs (yet)

I started with the following prompt on the app development, with Claude Code coaching me to drive a clear outcome:

Hi Claude, please brainstorm with me without writing any code. Strictly follow a structured process to inspire me to clarify business value and requirements of an application. Remember one question at a time. Once we finish, please summarize our discussion and requirements into ‘./doc/uclip-intro.md’. Let’s start.

Notice what I didn’t do:

Didn’t specify features (“build audio download, transcripts, and chat”)
Didn’t prescribe a tech component / stack (“use Python with FastAPI”)
Didn’t write step-by-step instructions (“first do X, then do Y”)

I used Plan Mode to steer collaborative iteration before execution, then auto-accepted edits to produce the product intro in one shot.

Claude Code asked clarifying questions:

Who are the users? (Me, busy professionals)
What’s the core objective? (Extract insights efficiently; deep-dive on specific topics)
What interactions do users need? (Summary, argument flow, structural insights, real-time Q&A)
What technical constraints exist? (I don’t need to scale to millions; I need something I can iterate on fast)
What technical components are being used? (python, streamlit with yt-dlp package, etc.)

As we discussed, Claude Code designed the product intro, which I used as the basis for a more detailed design spec, and tried Plan Mode to produce a draft.

Key insight: By not boxing Claude Code in with prescriptive steps, it could reason about the full picture — user needs, architecture, scalability — and propose something more coherent than I would have in isolation.

Step 2: Iterate on Outcomes, Not Code

While Claude Code may generate surprisingly strong results, the real unlock is iterating on outcomes, not code.

The initial implementation of Chat function (the feature allowing users to chat with AI on the clip content) was not useful — the interaction was not conversational style and each turn could take 20–30s, e.g. when I asked for “brainstorm a problem, one question at a time”, Chat function threw multiple questions at once, rather than one at a time. Likely the system prompt and prompt template in Chat function weren’t good enough. Instead of jumping to solutions, I collaborated with Claude Code to understand the problem first.

Then, I used this outcome-focused feedback:

In the Chat function, I tried to guide the chat service to collaborate with me one question at a time, but it responds with a bunch of questions. Could you please review the chat prompt and analyze what I may improve? Please don’t implement any solutions, but collaborate with me, help me understand, until I confirm to implement it.

Notice what I did here:

Described the outcome I wanted (one question at a time)
Described the current behavior (bunch of questions)
Asked Claude Code to analyze, not execute (“don’t implement”)
Invited collaboration (help me understand)

Claude Code helped me think through the UX philosophy. I learned why one question at a time matters for conversational flow. When I was happy with the collaborative suggestion, I let Claude Code act on it.

In addition, it turned out the initial implementation passed the entire transcript as context. What a naive approach! Maybe I could ask it to brainstorm with me on better options, but I paused and reflected how I actually converse with people — listening first, understanding intent, then responding. Could Chat function do the same? I tried this:

Would it be possible to review the user’s question, clarify the intention, then decide what context we need to provide to answer the question / coach the user? THINK HARD on this. Don’t implement anything until my confirmation.

Claude Code redesigned the entire chat flow:

Before:
User asks question → Retrieve entire transcript as context → LLM generates response (slow, generic)

After:
User asks question → Intention discovery → Context construction (retrieve only relevant parts for the intention) → LLM invocation → Answer (fast, targeted, collaborative)

This redesign taught me more about systems thinking than I expected. This kind of insight felt more natural when arrived at collaboratively than when designed in isolation.

Meanwhile, the redesign improved latency. Rather than putting entire transcript as context, it became frugal with context usage and achieved faster response each turn.

Overall, the chat changed everything about the user experience, from a Q&A function to a thoughtful interaction: Question → Intention Discovery → Context Construction → LLM Invocation → Answer.

Key insight: My building process is to collaborate with Claude Code and iterate on outcomes, rather than coding.

Step 3: When It Breaks: Collaborate, Don’t Command

Over the app building process, not every session was smooth — Claude Code sometimes went in circles on edge cases, and I had to learn when to reset context vs. push forward.

Hence, I managed context carefully. On one hand, once I finished a task, I always used ‘/clear’ to clear up the session so that I could start a brand-new conversation on the next. I wanted to keep it simple and avoid unnecessary context pollution. On the other hand, working on context is interesting. Given Claude models are so good at using tools, very often Claude Code could use bash commands to find information / context to complete tasks autonomously. However, there were some edge cases.

There was a bug when doing speaker identification in ‘Deep Dive’ analysis, and it prompted “Expecting value: line 1 column 1 (char 0)”. I took it easy and just asked Claude Code to “please investigate the error” with the error message. Claude Code worked like a charm — it formed a structured process to resolve the bug, and confirmed the fix worked. Everything sounded perfect, but the fix didn’t work at all. Next, I tried to be patient — “your fix doesn’t work, please investigate again”. Then, Claude Code worked harder but didn’t deliver any proper fixes…

Maybe not every session was as smooth as steps 1 & 2 suggest. I got frustrated, and kept asking Claude Code to “FIX It…”. Guess what?! The bug still existed.

So I did what most frustrated engineers do — I kept slamming the same button. Then, I heard this voice in my mind — “Ask not what the model can do for you”, from Boris (reference clip timestamp). Maybe I should “ask what I can do for the model”?! First, I googled the error message, but I couldn’t get much useful information. Then, I decided to collaborate with Claude Code to know more -

your fix doesn’t work, could you please analyze the issue and explain what may be missed. If you need more information for the investigation, please let me know.

This one did change the game a bit. Claude Code shifted to focus on analyzing the problem (but not implementing the fix). It nailed down the culprit as an empty string being parsed by JSON. Then, where was the empty string from and what was that for? While reading the code, I started realizing that the function made an LLM call to identify speakers — the system prompt had a dedicated instruction for multi-speaker transcript — “You are analyzing a multi-speaker transcript to identify speakers …”. However, when I tried to analyze a single-speaker clip, the LLM call returned an empty string instead of proper JSON, and resulted in the JSON parsing error.

I tuned the prompt template to support single-speaker clip explicitly, and gave this instruction to improve further -

The speaker identification fails on single-speaker videos — the model seems confused when there’s only one speaker label. I think the prompt saying ‘multi-speaker transcript’ is misleading it. Can you fix the prompt to handle both single and multi-speaker cases?

Claude Code did better prompt engineering — being specific and direct on return value — “Always return the JSON array even if there is only one speaker.” Surely, the fix worked perfectly.

Key insight: When Claude Code fails, the fix is usually better context, not more pressure.

Not the End, But a New Builder Path

Remember that CEO podcast I couldn’t properly digest? It was a 53-minute podcast with 2 hosts and the CEO. It took me about 10 minutes to run through Clip Buddy; in addition, I conducted an in-depth chat session to explore the business strategy and planning, then shared the analysis with my account manager. He was thrilled.

Did Claude Code rob me of the joy of building? No — the joy shifted from typing to thinking. Did I ship something I didn’t understand? I understand it more deeply than most code I’ve written, because I had to articulate my intent clearly enough for a collaborator to act on it. Am I just a prompt engineer? No — I’m still the architect. Claude Code is the printing press.

That’s not the end, but it paved a new path for me to build, to learn from, and to collaborate with Claude Code.