Automated-AI-Web-Researcher-Ollama

Open source alternative to ChatGPT Pro and Tavily

3K stars
MIT
Active recently
Last commit 51 days ago

Last verified Oct 17, 2026

Automate web research with a local LLM. Breaks any query into focus areas, scrapes relevant sites, and saves findings with source URLs.

Visit website GitHub3K

image of Automated-AI-Web-Researcher-Ollama

Read the full breakdownWho it is for, how it works, trade-offs, the quick start and the FAQ

who it's for

Who Automated-AI-Web-Researcher-Ollama is for#

Researchers doing multi-source web research sweeps

The tool handles the mechanical part of desk research: given a topic, it finds, retrieves, and compiles relevant web content from multiple sources in one session. The researcher reviews the compiled document and summary rather than spending hours on individual searches.

Skip if:

Your sources require academic database access such as paywalled journal PDFs. DuckDuckGo cannot reach paywalled content, so the tool only covers publicly accessible web pages.

Developers extending a local Ollama setup

If you already have Ollama running and want to add automated web research without adding cloud dependencies, this tool integrates directly with your local model. Configure the model name once in llm_config.py and run it from the command line with no additional accounts or API keys.

Skip if:

Your project needs a programmatic API or structured JSON output for downstream processing. The tool writes a text file and produces console output; it does not expose an API endpoint.

Privacy-conscious individuals researching sensitive topics

All queries, search terms, scraped content, and model inference stay on your machine. Nothing is logged to an external service. For people researching medical, legal, or other sensitive subjects, no data routing to external servers is a material advantage.

Skip if:

You need results from sources that block automated scraping or require JavaScript rendering. The tool uses basic HTTP scraping and cannot retrieve content from JavaScript-heavy or paywalled sites.

the problem

The problem it solves#

Running thorough web research manually is slow and fragmented. You search, read, search again, and lose track of sources. Conversational LLMs compress that process but at a cost: they synthesize without searching, producing confident answers unsupported by current sources.

The alternative is commercial AI search APIs. Services like Tavily give LLMs real-time search with structured results, but they route every query through their servers and charge per call. For high-volume research, sensitive topics, or anyone already running Ollama locally, that combination of cost and data routing is the core pain. You want automated, multi-step research without sending your queries to an external service to get it.

how Automated-AI-Web-Researcher-Ollama solves it

How it solves it#

Prioritized research focus areas

On receiving your query, the Ollama model generates five specific research focus areas, each ranked by relevance. It investigates the highest-priority area first, then works down the list, so the most important angles are covered even if you stop the session early.

Automated web search and scraping via DuckDuckGo

For each focus area, the LLM formulates targeted search queries, runs them through DuckDuckGo, selects the most relevant pages from the results, and scrapes content from those pages. No external search API key required.

Sourced research document output

All retrieved content and source URLs are saved into a text file during the session. When you quit, the LLM reads the full document and generates a summary answering your original query. The file remains after the session for your reference.

Self-improving search cycle

After covering the initial five focus areas, the LLM generates a new round of focus areas based on what it found, often discovering related angles it did not anticipate. This loop continues until you stop it, making longer sessions progressively more thorough.

Post-research conversation mode

When the research session ends, the tool enters Q&A mode: you can ask specific questions about the findings, and the model answers from the compiled research content rather than from its training data alone.

Runtime session controls

During a running session, type 's' to show status, 'f' to see the current focus area, 'p' to pause and get a progress assessment from the LLM, or 'q' to quit and trigger the final summary.

strengths · trade-offs

Strengths and trade-offs#

Strengths

Fully local: no API costs or data routingEvery query, search result, and scraped page is processed on your own machine via Ollama. There are no per-query fees and no data sent to external services. For researchers handling sensitive subjects or running high query volumes, this is the defining advantage over managed APIs like Tavily.
MIT licensed with no usage restrictionsThe MIT license means you can run it commercially, modify it, and distribute it without restriction. There are no tiers, no feature gates, and no subscription required.
No third-party API keys requiredDuckDuckGo is used for web search, which requires no registration or API key. Ollama handles LLM inference locally. The full research loop runs without signing up for or paying for any third-party service.
Active repository with strong community interestThe repository has over 3,000 GitHub stars and 274 forks, with the last commit in September 2026, indicating active use and ongoing development from the community.

Trade-offs

-Prototype status with manual configuration requiredThe README explicitly describes this as a prototype. Configuration requires editing llm_config.py directly to set your model name, base URL, and context size. There is no web UI, no config wizard, and no default model: you must set these before any research session runs.
-Model context length constraintsExtended research sessions retrieve large amounts of text. Models without a large context window will truncate or fail at the summary step. The README recommends phi3 models with 128k context specifically for this reason. Shorter-context models produce degraded results.
-No Docker image: manual Python setup requiredInstallation requires cloning the repository, creating a Python virtual environment, and manually installing dependencies. There is no Docker image and no one-command deployment. Basic Python environment management is a prerequisite.

versus alternatives

Automated-AI-Web-Researcher-Ollama vs alternatives#

Automated-AI-Web-Researcher-Ollama vs Tavily

Both tools give LLMs the ability to research the web, but they operate on different deployment models. Tavily is a managed API service for AI agents: developers call its endpoint to get search results, citations, and summaries from a hosted service. Automated-AI-Web-Researcher-Ollama is a standalone Python script that runs an entire research loop locally, using DuckDuckGo for search and Ollama for inference.

Feature	Automated-AI-Web-Researcher-Ollama	Tavily
License	MIT	Proprietary
Self-hosting	Yes (Python + Ollama)	No
LLM integration	Local Ollama model	External LLM API
Search backend	DuckDuckGo (no key needed)	Proprietary search index
API for programmatic use	No (CLI only)	Yes (REST API)
Data stays local	Yes	No
Pricing	Free (self-hosted)	Free tier + paid plans

Automated-AI-Web-Researcher-Ollama is the stronger choice when data privacy is the deciding factor: queries, search terms, and all scraped content are processed locally and never leave your machine. It is also the better path when you want no recurring costs and already have Ollama running. The self-improving research loop can produce thorough results for open-ended questions that benefit from multiple search cycles.

Tavily is the better choice when your application needs a programmatic REST API rather than a CLI script, when you need reliable structured output for downstream processing, or when you cannot manage a local Ollama setup. Tavily integrates directly into LLM agent frameworks like LangChain and LlamaIndex, making it significantly easier to use in production AI agent deployments where consistency and infrastructure management matter.

install · quick start

Quick start#

bash

Install the project on Linux or macOS after setting up Ollama on your local machine.
```bash
git clone https://github.com/TheBlewish/Automated-AI-Web-Researcher-Ollama
cd Automated-AI-Web-Researcher-Ollama
python -m venv venv
source venv/bin/activate
pip install -r requirements.txt
```

tech stack · detected from GitHub

What it's built on#

Languages: Python

frequently asked

FAQ#

Does Automated-AI-Web-Researcher-Ollama require an internet connection?

Yes, for web searches and scraping. The LLM inference runs locally via Ollama, but the research loop uses DuckDuckGo to search the web and HTTP requests to retrieve content from websites. You need internet access during active research sessions.

Which Ollama models work best with this tool?

The README recommends phi3:3.8b-mini-128k-instruct or phi3:14b-medium-128k-instruct, specifically because their 128k context window handles the large amounts of scraped text that accumulate during extended sessions. Models with shorter context windows will struggle with the final summary step when research volume is high.

Is this an open source alternative to Tavily?

Yes, for local use cases. Tavily is a managed search API for LLM agents with structured results and a hosted service. This tool gives you similar research capability with no API costs and no data leaving your machine. The tradeoff is that Tavily is more reliable for production use and requires no infrastructure setup on your part.

Is there a newer successor to this project from the same author?

Yes. The README links to Academic-AI-Literature-Reviewer-Ollama, described by the author as a successor with a much larger codebase focused on academic articles rather than general web search. If your research need is academic literature specifically, that project may be more suitable. This tool remains active for general web research.

How do I stop a running research session early?

Type 'q' and press Ctrl+D. This stops the research loop and triggers the LLM to review all content collected so far, generating a summary of your original query before the session ends. You can also type 'p' to pause and assess progress without stopping.

also worth a look

Similar open-source tools#

hyperresearch

Alternative toChatGPT Pro

Deep research agent with a persistent vault for Claude Code

3,900 GitHub stars
Licence MIT

Jina AI

Alternative toTavily

Open source search APIs and MCP tools for RAG and agent workflows

886 GitHub stars
Licence Apache-2.0

Marker

Alternative toUnstructured.io

Convert documents to clean markdown and JSON with open source AI.

40,566 GitHub stars
Licence Apache-2.0

Chat2DB

Alternative toDataGrip

AI text-to-SQL client for 40+ databases; local-first, self-hosted

28,327 GitHub stars
Licence Apache-2.0 with additional commercial restrictions (LicenseRef-Chat2DB)

MinerU

Alternative toLlamaParse

Document parsing for LLM workflows, OCR, and agent reading.

82,142 GitHub stars
Licence Apache-2.0 with commercial threshold and attribution terms

Duckle

Alternative toFivetran

Self-hosted ETL on DuckDB with 385 connectors and no per-row billing.

1,522 GitHub stars
Licence Apache-2.0

by category

Open source tools in these categories#

407

4,904 GitHub stars
Licence Apache-2.0
#AI Agents

Omnara

Alternative toClaude Managed Agents

Open-source agent deployment API. Self-host or use Omnara Cloud.

2,957 GitHub stars
Licence Apache-2.0
#AI Agents

Inbox Zero

Alternative toSuperhuman

AI email assistant that drafts replies and clears your inbox

12,712 GitHub stars
Licence AGPL-3.0
#Email

Feature your project

The projects we add and every new guide. Free, unsubscribe any time.

Find open source alternatives to paid software

Automated-AI-Web-Researcher-Ollama vs Tavily

Feature	Automated-AI-Web-Researcher-Ollama	Tavily
License	MIT	Proprietary
Self-hosting	Yes (Python + Ollama)	No
LLM integration	Local Ollama model	External LLM API
Search backend	DuckDuckGo (no key needed)	Proprietary search index
API for programmatic use	No (CLI only)	Yes (REST API)
Data stays local	Yes	No
Pricing	Free (self-hosted)	Free tier + paid plans

Who Automated-AI-Web-Researcher-Ollama is for#

Researchers doing multi-source web research sweeps

Skip if:

Developers extending a local Ollama setup

Skip if:

Privacy-conscious individuals researching sensitive topics

Skip if:

The problem it solves#

How it solves it#

Prioritized research focus areas

Automated web search and scraping via DuckDuckGo

Sourced research document output

Self-improving search cycle

Post-research conversation mode

Runtime session controls

Strengths and trade-offs#

Strengths

Trade-offs

Automated-AI-Web-Researcher-Ollama vs alternatives#

Automated-AI-Web-Researcher-Ollama vs Tavily

Quick start#

What it's built on#

FAQ#

Similar open-source tools#

Open source tools in these categories#

AI & Machine Learning

Developer Tools

Data & Analytics

Find open source alternatives to paid software

Who Automated-AI-Web-Researcher-Ollama is for#

Researchers doing multi-source web research sweeps

Skip if:

Developers extending a local Ollama setup

Skip if:

Privacy-conscious individuals researching sensitive topics

Skip if:

The problem it solves#

How it solves it#

Prioritized research focus areas

Automated web search and scraping via DuckDuckGo

Sourced research document output

Self-improving search cycle

Post-research conversation mode

Runtime session controls

Strengths and trade-offs#

Strengths

Trade-offs

Automated-AI-Web-Researcher-Ollama vs alternatives#

Automated-AI-Web-Researcher-Ollama vs Tavily

Quick start#

What it's built on#

FAQ#

Similar open-source tools#

Open source tools in these categories#

AI & Machine Learning

Developer Tools

Data & Analytics

Find open source alternatives to paid software