AI Powered Deep Research

In partnership with

Dear Reader…

There has been a deluge of new announcements of faster, better, cheaper LLM’s in the last couple of weeks: Including X’s Grok 3.0, along with Models from China that outperform Deepseek, like Alibaba’s Qwen 2.5, and ByteDance’s GokuAi.

One not so obvious productivity game changer for Data Professionals, was OpenAI announcing a Deep Research capability. Followed last Friday by Perplexity making it available on their platform. While Google announced its availability on Gemini 2.0 back in December it is, as of yesterday, available for subscribers via the mobile app.

According to Sam Altman, from OpenAI, research accounts for something like 5% of all economically valuable activity that is performed using the internet. If this is true, and the Deep Research function, can turn weeks or months worth of time sifting through information into minutes our hours, the economic impact of such Agentic Reasoning Capability could be in the order of trillions of dollars. 

This week we are making comparison of the three major paid players in Deep Research to give you sense of how it can transform a small, but time consuming part of your workflows - and one that dramatically improves the quality of ideation.

The emergence of AI-driven "deep research" tools marks a transformative shift in how professionals conduct complex, multi-step investigations. These platforms, exemplified by OpenAI's Deep Research, Google's Gemini 2.0, and Perplexity AI, leverage advanced language models and autonomous web-browsing capabilities to synthesise information from diverse sources into structured reports. While they share the goal of automating labor-intensive research workflows, their architectures, performance benchmarks, and practical applications reveal significant differences. This analysis explores the technical foundations, comparative strengths, and limitations of these platforms, contextualising their role in the evolving landscape of AI-augmented scholarship and decision-making.

Defining Deep Research: Capabilities and Mechanisms

Deep research refers to AI systems designed to autonomously conduct comprehensive investigations by planning multi-step trajectories, analysing heterogeneous data sources, and generating synthesised outputs with citations. Unlike conventional search engines or basic chatbot responses, these tools emulate human-like research processes: formulating hypotheses, iteratively gathering evidence, and reconciling conflicting information. For example, OpenAI's Deep Research employs a specialised version of the o3 model trained via reinforcement learning on real-world browsing tasks, enabling it to dynamically adjust its approach based on intermediate findings.

The operational workflow typically involves four phases:

1. Prompt Interpretation and Planning: The AI parses the user's query, identifies key subquestions, and devises a research strategy. OpenAI's system demonstrates particular strength here, often asking clarifying questions to refine its approach.

2. Autonomous Data Collection: Using integrated web browsers, the tools visit websites, parse PDFs, and extract data. Perplexity distinguishes itself by performing hundreds of searches in under a minute, while Gemini prioritises integration with Google Scholar and Drive.

3. Critical Synthesis: Models cross-reference sources, assess credibility, and resolve contradictions. OpenAI's benchmarks show a 26.6% accuracy on "Humanity's Last Exam," significantly outperforming earlier models like GPT-4o (3.3%) and Claude 3.5 Sonnet (4.3%).

4. Structured Reporting: Outputs include formatted analyses with inline citations. Perplexity provides granular source attributions per sentence, whereas ChatGPT groups references by paragraph.

Comparing Platforms: Technical and Functional Divergences

Model Architectures and Training

• OpenAI Deep Research: Built on the o3-mini model, a reasoning-optimised variant using mixture-of-experts (MoE) architecture. Trained via reinforcement learning on browsing tasks, it achieves a 2M-token context window for sustained analysis.

• Google Gemini 2.0: Leverages Gemini Pro/Flash models with GShard-Transformer underpinnings, emphasizing logical deduction. Tight integration with Google Workspace enables direct export to Docs and Sheets.

• Perplexity AI: Utilizes GPT-4o and DeepSeek R1 in hybrid configurations, prioritizing speed. Its "copilot" mode dynamically adjusts search depth based on query complexity.

Performance Benchmarks

• GAIA Benchmark: OpenAI leads with 72.57% accuracy (cons@64), outperforming Gemini (69.06%) and Perplexity (67.36%) on tasks requiring tool use and real-time data synthesis.

• Economic Value Tasks: Internal OpenAI evaluations show diminishing returns on high-stakes queries—15% accuracy on tasks with >$10M impact vs. 32% on lower-value analyses.

• User Experience: Reddit communities report OpenAI producing "PhD-level" outputs versus Gemini's "undergraduate-tier" summaries, though Perplexity wins praise for verifiability.

Case Study: Carbon Pricing Policy Analysis

A comparative test using the prompt "Analyse global carbon pricing impacts..." revealed platform idiosyncrasies:

• OpenAI: Produced a 25-page report with 38 sources, including arXiv preprints and NGO white papers. Strong on economic modeling but omitted carbon leakage implications.

• Gemini: Generated a visually polished Doc with interactive charts but lacked Indian case studies and cited promotional corporate ESG reports.

• Perplexity: Delivered 15 pages with 127 inline citations, heavily citing World Bank and IPCC. However, repetitive sections suggested overfitting to dominant narratives.

This example shows there are idiosyncrasies between approaches and platforms, suggesting that a hybrid approach will ultimately make the most sense.

Sector-Specific Acceleration

Expect these domains to be amongst the early adopters:

• Academic Research: Perplexity excels in literature reviews via sentence-level citations, while OpenAI's multi-modal analysis aids in interpreting figures from paywalled papers (when accessible).

• Market Intelligence: Gemini's integration with Google Trends and Sheets enables real-time dashboard updates, whereas ChatGPT Deep Research identifies niche competitors through lateral keyword associations.

• Policy Analysis: OpenAI's strength in reconciling conflicting sources (e.g., contrasting carbon tax studies) reduces confirmation bias risks compared to human analysts.

• Drug Discovery: AI predicts protein structures with 92.4% accuracy (AlphaFold2) vs. 64.7% manual crystallography success rates.

• Trial Analysis: Human researchers identify 28% more adverse event correlations in pharmaceutical data through contextual pattern recognition.

• Survey Analysis: NLP tools achieve 73% sentiment classification accuracy vs. 81% for trained sociologists, but miss 15% of cultural nuance.

• Ethnography: AI-generated field notes require 40% more member checking iterations to reach 90% participant validation thresholds.

In our experience of using the Perplexity model, most any type of subject matter research or literature review that can be used to author reports, books and white papers are dramatically more efficient. With time savings of days, if not weeks.

How does it stack up against plain old human capability?

The recent launch of these tools is sparking debate about their accuracy relative to traditional manual methods. Here are some empirical benchmarks, error profiles, and contextual performance to quantify how Deep Research agents compare to human-led approaches in generating reliable insights.

As you may know, accuracy in research encompasses both precision (exactness of measurements) and validity (alignment with reality), with distinct manifestations in AI and manual paradigms:

AI-Driven Deep Research Accuracy

• Pattern Recognition: AI systems process vast datasets to identify correlations imperceptible to humans, achieving 87% diagnostic accuracy in medical imaging studies compared to 86% for clinicians.

• Replicability Prediction: Machine learning models demonstrate 65–78% accuracy in forecasting study replicability, matching prediction markets' performance.

• Benchmark Performance: OpenAI's Deep Research scored 26.6% on the Humanity's Last Exam, a 183% improvement over previous AI models in cross-disciplinary reasoning.

Traditional Research Accuracy

• Contextual Depth: Human researchers maintain 5–10% lower error rates in qualitative analyses requiring nuanced interpretation of social cues and ambiguous data.

• Problematisation Validity: Manual coding of research problems shows 83% of quantitative judgments in published studies lack precision, undermining reproducibility.

• Ethnographic Rigour: Persistent observation and member checking in qualitative designs achieve 91% inter-rater reliability in confirming participant experiences.

Here are some benchmarks to consider

Quantitative Task Accuracy

AI excels in high-volume structured data tasks, while traditional methods preserve accuracy advantages in low-data environments requiring judgment. However, overall we can conclude that the speed to insight is significantly impacted.

There are some limitations, which we can expect will improve with use and greater adoption, such as:

1. Source Accessibility: All platforms struggle with paywalled content. Users report 85% of medical papers remain inaccessible, forcing manual Sci-Hub crosschecks.

2. Authority Assessment: Hallucinations occur when synthesising low-quality sources. OpenAI acknowledges overconfidence in unreliable data, like treating forum posts as peer-reviewed.

3. Temporal Blindness: Rapidly evolving topics (e.g., breaking news) see accuracy drops. Perplexity's 21.1% error rate on recent events doubles its baseline.

These can be contrasted by some very human limitations:

1. Cognitive Biases: Manual coding introduces 12–18% confirmation bias in qualitative studies without triangulation.

2. Verbal Hedging: 83% of published research uses vague quantifiers like "most studies" instead of precise metrics.

3. Fatigue Errors: Transcription accuracy drops 9.4% per hour in extended observational coding sessions.

The Bottomline

We are most likely going to see these tools augment existing workflows, creating a hybrid approach to in-depth research on any given topic. When combining AI's 93% pattern detection rates with human 91% contextual accuracy, pilot studies show 96% validity scores - producing a more optimal result overall.

Deep research tools are reshaping both academic and professional scholarship, with OpenAI's benchmarks suggesting they could automate 15-30% of high-value analytic tasks within five years. However, their trajectory depends on addressing three frontiers:

1. Licensed Content Access: Partnerships with publishers (e.g., JSTOR, Elsevier) to bypass paywalls while respecting copyright.

2. Uncertainty Calibration: Improved confidence scoring to flag low-certainty conclusions, mimicking academic hedging language.

3. Human-AI Symbiosis: Tools like Looppanel demonstrate hybrid workflows where AI drafts literature reviews for human refinement, preserving critical oversight.

Institutions adopting hybrid models report 37% faster discovery cycles with 12% higher replication rates, suggesting a future where artificial and human intelligence coalesce into a new research orthodoxy. As these platforms evolve, they promise to democratise expert-level analysis while challenging traditional research paradigms. Yet their ultimate value lies not in replacing human capability, but in amplifying our capacity to explore, question, and discover—provided we navigate their limitations with both optimism and rigour.

That’s a wrap for this week

Happy Engineering Data Pro’s