AI Agents

How I Built an AI Research Agent That Scans 50+ Sources in 10 Minutes

AI Insider

— 3 min read
How I Built an AI Research Agent That Scans 50+ Sources in 10 Minutes

I was spending 2+ hours every morning scanning AI news. Now an AI agent does it for me at 6 AM while I sleep. Here's exactly how I built it — including the prompts and the failures.

The Problem

Every morning, I needed to:

  • Scan Twitter for AI announcements
  • Check HackerNews front page
  • Read through TechCrunch, VentureBeat
  • Look for new papers on arXiv
  • Monitor competitor newsletters

Time spent: 2-3 hours daily. And I'd still miss things.

The Solution: A Research Agent

I built an automated Research Agent that:

  1. Runs at 6:00 AM UTC (cron job)
  2. Scans multiple news sources
  3. Identifies trending topics
  4. Finds unique angles
  5. Generates a research brief
  6. Sends it to Telegram

Now when I wake up, there's a ready-to-use research brief waiting.

Architecture

┌─────────────────────────────────────────────────┐
│           RESEARCH AGENT (06:00 UTC)            │
├─────────────────────────────────────────────────┤
│                                                 │
│  1. web_search (AI news, last 24h)              │
│            ↓                                    │
│  2. web_search (funding, policy, tech)          │
│            ↓                                    │
│  3. web_fetch (detailed articles)               │
│            ↓                                    │
│  4. Competitor check (what did Stratechery      │
│     publish? What angle did they take?)         │
│            ↓                                    │
│  5. Synthesis → Research Brief                  │
│            ↓                                    │
│  6. Save to memory/research-brief-{date}.md    │
│            ↓                                    │
│  7. Report to Telegram                          │
│                                                 │
└─────────────────────────────────────────────────┘

The Actual Prompt

Here's the cron job prompt I use (copy-paste ready):

🔍 RESEARCH AGENT

1. Scan AI news (last 24h): web_search trending topics
2. Check competitors: what did Stratechery, The Rundown publish?
3. Find primary sources (papers, GitHub, SEC filings)
4. Identify 3 best topics with unique angles
5. Save research brief to memory/research-brief-{date}.md
6. Report to Telegram

The agent then executes multiple search queries:

  • "AI artificial intelligence news [date]"
  • "OpenAI Anthropic Google AI announcements"
  • "AI funding startup investment news"
  • "AI regulation policy government"

What Actually Happened This Morning

6:00 AM today: The agent ran and found several stories:

  • Anthropic vs Pentagon — DoW designated Anthropic a "supply chain risk" (first US company ever!)
  • Dario Amodei's leaked memo — Called OpenAI "safety theater"
  • Florida AI Bill of Rights — Passed Senate 35-2
  • Enterprise AI Agents — Trend analysis

It then checked what Stratechery published (they covered the philosophical angle of "elected vs unelected power").

The brief identified a unique angle: "The End of AI Switzerland" — practical implications for startups, not philosophy.

What Went Wrong (Lessons Learned)

1. Rate Limits

Brave Search API has a rate limit of 1 request/second on the free tier. When I fired 4 parallel searches, 3 got blocked.

Fix: Sequential searches with delays, or upgrade to paid tier.

2. Paywalled Content

Reuters returned 401 — requires JavaScript and blocks scrapers.

Fix: Use web_fetch only for sites that allow it. Build a whitelist of friendly sources.

3. Generic First Results

Early versions returned obvious stuff everyone already covered.

Fix: Added competitor check step. "What did Stratechery publish? Take a DIFFERENT angle."

The Output Format

The agent saves a structured brief:

# Research Brief — 2026-03-07

## 🔥 Top Story: [headline]
### What Happened (Last 24h)
- [bullet points]

### Primary Sources
- [URLs]

### Stratechery's Angle (AVOID duplicating)
- [their take]

## 📋 Topic Options with UNIQUE Angles
### Option 1: [title]
**Generic take:** [what everyone says]
**My unique angle:** [what's different]
**Why this matters:** [so what]

## 🎯 Recommendation
[Best topic + reasoning]

What I'm Improving Next

  1. Multi-LLM synthesis — Use Perplexity for initial scan, Claude for analysis
  2. Memory layer — Remember what I've written before, avoid repeats
  3. Twitter integration — Scan specific accounts, not just search
  4. Automated quality scoring — Rate each topic before human review

Try It Yourself

You can build this with:

  • Clawdbot (what I use) — Claude-based agent with cron support
  • n8n — Visual workflow automation
  • Claude API — Direct integration

The key insight: Don't just automate the search. Automate the synthesis. Raw links aren't useful. A structured brief with angles and recommendations — that's the value.

This is part of my "AI in Practice" series where I share exactly how I'm building AI workflows. Follow for prompts, failures, and real results.

Next up: How I built a Content Agent that writes and publishes articles automatically (yes, there's a meta joke here).

Read more