The New Vertical by Scott Steinhardt

Exporting Pocket to Obsidian

May 21, 2025

Pocket is shutting down, which means I'm on the hunt for an alternative Read It Later application.

Yet Obsidian has a really good Web Clipper tool, and I'm 100% into Obsidian for other things, so why not use it for Read it Later?

Unfortunately, the Web Clipper is not a bulk import tool for old Pocket links, nor do any good tools for Obsidian exist. So I vibe coded one with Claude.

Using Defuddle

Defuddle was built by Obsidian's CEO Steph Ango as the backbone of the Web Clipper. After he pointed out to me on LinkedIn that you can use it in a CLI, I decided to have Claude help me. Here's the steps to recreating it (which were created by Claude).

1. Set Up Your Computer

2. Create Your Project Folder

Make a new folder on your Desktop:

cd Desktop
mkdir link-converter
cd link-converter

3. Create the Required Files

Create a file called package.json with this exact content:

{
  "name": "link-converter",
  "version": "1.0.0",
  "type": "module",
  "dependencies": {
    "defuddle": "latest",
    "jsdom": "latest"
  }
}

Create a file called urls.txt and paste your links (one per line):

https://example.com/article1
https://example.com/article2
https://example.com/article3

Create a file called convert.js with this code:

import { JSDOM } from 'jsdom';
import { Defuddle } from 'defuddle/node';
import fs from 'fs/promises';

// Read your URLs from urls.txt
const urlsText = await fs.readFile('urls.txt', 'utf8');
const urls = urlsText.split('\n').filter(line => line.trim() && !line.startsWith('#'));

// Create output folder
await fs.mkdir('markdown-files', { recursive: true });

// Track results
const successful = [];
const failed = [];

// Process each URL
console.log(`Checking ${urls.length} links...\n`);
console.log('=' .repeat(60));

for (let i = 0; i < urls.length; i++) {
  const url = urls[i].trim();
  
  try {
    console.log(`\n[${i + 1}/${urls.length}] Testing: ${url}`);
    
    // Get the webpage
    const dom = await JSDOM.fromURL(url, {
      userAgent: 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36'
    });
    
    // Extract content with Defuddle
    const result = await Defuddle(dom, {
      markdown: true,
      url: url
    });
    
    // Create filename from title
    let filename = result.title || `Untitled ${i + 1}`;
    filename = filename
      .replace(/[<>:"/\\|?*]/g, '') // Remove characters not allowed in filenames
      .replace(/\s+/g, ' ')         // Normalize multiple spaces to single space
      .trim()                       // Remove leading/trailing spaces
      .substring(0, 100);           // Limit length
    
    // Check if file already exists and add number if needed
    let finalFilename = `${filename}.md`;
    let counter = 1;
    while (await fs.access(`markdown-files/${finalFilename}`).then(() => true).catch(() => false)) {
      finalFilename = `${filename} ${counter}.md`;
      counter++;
    }
    
    // Create markdown file content
    const content = `# ${result.title || 'Untitled'}\n\nSource: ${url}\n\n---\n\n${result.content}`;
    
    // Save file
    await fs.writeFile(`markdown-files/${finalFilename}`, content, 'utf8');
    
    console.log(`✓ SUCCESS: Saved as "${finalFilename}"`);
    console.log(`  Title: ${result.title || 'No title found'}`);
    console.log(`  Words: ${result.wordCount || 0}`);
    
    successful.push({
      url: url,
      title: result.title,
      filename: finalFilename,
      wordCount: result.wordCount
    });
    
    // Wait 1 second before next request
    await new Promise(resolve => setTimeout(resolve, 1000));
    
  } catch (error) {
    console.log(`✗ FAILED: Could not crawl this URL`);
    console.log(`  Error: ${error.message}`);
    
    // Determine error type
    let errorType = 'Unknown error';
    if (error.message.includes('getaddrinfo ENOTFOUND')) {
      errorType = 'Domain not found';
    } else if (error.message.includes('404')) {
      errorType = 'Page not found (404)';
    } else if (error.message.includes('403')) {
      errorType = 'Access forbidden (403)';
    } else if (error.message.includes('401')) {
      errorType = 'Authentication required (401)';
    } else if (error.message.includes('timeout')) {
      errorType = 'Connection timeout';
    } else if (error.message.includes('ECONNREFUSED')) {
      errorType = 'Connection refused';
    } else if (error.message.includes('certificate')) {
      errorType = 'SSL certificate error';
    }
    
    failed.push({
      url: url,
      error: error.message,
      errorType: errorType
    });
  }
}

// Create detailed report
console.log('\n' + '=' .repeat(60));
console.log('FINAL REPORT');
console.log('=' .repeat(60));
console.log(`\nTotal URLs: ${urls.length}`);
console.log(`✓ Successful: ${successful.length} (${Math.round(successful.length/urls.length*100)}%)`);
console.log(`✗ Failed: ${failed.length} (${Math.round(failed.length/urls.length*100)}%)`);

// Save success report
const successReport = successful.map(s => 
  `${s.url}\n  → Saved as: ${s.filename}\n  → Title: ${s.title || 'No title'}\n  → Words: ${s.wordCount}\n`
).join('\n');
await fs.writeFile('successful-urls.txt', `SUCCESSFULLY CRAWLED URLS (${successful.length})\n${'='.repeat(40)}\n\n${successReport}`, 'utf8');

// Save failure report
const failureReport = failed.map(f => 
  `${f.url}\n  → Error Type: ${f.errorType}\n  → Details: ${f.error}\n`
).join('\n');
await fs.writeFile('failed-urls.txt', `FAILED TO CRAWL URLS (${failed.length})\n${'='.repeat(40)}\n\n${failureReport}`, 'utf8');

// Save JSON report for programmatic use
const fullReport = {
  summary: {
    total: urls.length,
    successful: successful.length,
    failed: failed.length,
    successRate: Math.round(successful.length/urls.length*100) + '%',
    timestamp: new Date().toISOString()
  },
  successful: successful,
  failed: failed
};
await fs.writeFile('crawl-report.json', JSON.stringify(fullReport, null, 2), 'utf8');

// Display failed URLs summary
if (failed.length > 0) {
  console.log('\n❌ FAILED URLS:');
  console.log('-'.repeat(60));
  
  // Group by error type
  const errorGroups = {};
  failed.forEach(f => {
    if (!errorGroups[f.errorType]) errorGroups[f.errorType] = [];
    errorGroups[f.errorType].push(f.url);
  });
  
  for (const [errorType, urls] of Object.entries(errorGroups)) {
    console.log(`\n${errorType} (${urls.length}):`);
    urls.forEach(url => console.log(`  - ${url}`));
  }
}

console.log('\n' + '=' .repeat(60));
console.log('📁 FILES CREATED:');
console.log('  - successful-urls.txt (list of crawled URLs)');
console.log('  - failed-urls.txt (list of failed URLs with errors)');
console.log('  - crawl-report.json (detailed JSON report)');
console.log('  - markdown-files/ (folder with successful conversions)');
console.log('=' .repeat(60));

4. Install Everything

In your terminal, make sure you're in the link-converter folder, then type:

npm install

Wait for it to finish (might take 1-2 minutes)

5. Run the Converter

Type this command:

```
node convert.js
```

Watch it work - you'll see:

```
Converting 3 links to markdown files...

Processing 1/3: https://example.com/article1
✓ Saved: 1-article-title.md

Processing 2/3: https://example.com/article2
✓ Saved: 2-another-article.md
```

6. Find Your Files

This creates 4 groups of files:

This way, if you have any articles that didn't get parsed, you'll be notified about them.

7. Drag your markdown files to Obsidian

That's all you need to move from Pocket to Obsidian!