Digital Polymaths are coming to Augment you!

In partnership with

Dear Reader…

As we move headlong into 2025, it is worth reflecting on how AI is impacting the role of Data Professional. At a macro level according to Accel Venture Capital Data, Business Intelligence & Analytics hires are up 50% on 2023, and hires for AI Engineers are up a staggering 285%.

Last year at this time Gartner argued that over the course of five years AI was likely to be as disruptive as World War II, in that it will revolutionise our lives by similar orders of magnitude.

While AI may not be displacing professionals in the workplace yet, the use demand for AI Engineers and the shear volume of investment particularly in Vertical AI applications is set to change your daily routine very soon.

As we look forward the rise of the expert polymath AI Assistant could be the catalyst that will flip this adage on its head: Making ten years of innovation possible in just one year. Bringing together the advances that we have been documenting over the last few weeks, the expert polymath AI assistant becomes a valuable workplace companion when they converge.

1. Near-Infinite Context Windows

In the most read post of the year from the 20th November we looked at how Large Language Models (LLMs) have made remarkable strides in extending their context windows. From the modest 2,048 tokens in GPT-3, we've witnessed a leap to an impressive one million tokens in Gemini 1.5 Pro. This exponential growth in context capacity is paving the way for more sophisticated and nuanced data processing capabilities.

The implications of this advancement are profound. You can now work with vast amounts of information within a single prompt, enabling more comprehensive analysis and reducing the need for data segmentation. This expanded context allows for better understanding of complex relationships within datasets, leading to fewer hallucinations and far more accurate insightful predictions.

2. Text-to-Action: Bridging Language and Execution

In the 27th November edition we explored how Text-to-action technology represents a significant leap in how we interact with and control data systems. It allows for the conversion of natural language commands into specific actions or responses, bridging the gap between human intent and machine execution.

In the realm of data engineering, this capability translates to more intuitive and efficient data management processes. Engineers can issue complex data manipulation commands using natural language, streamlining workflows and reducing the technical barriers to advanced data operations.

3. Recursive Self-Improvement: The Engine of Continuous Enhancement

On 11th December we discussed how Recursive self-improvement (RSI) is set to be a game-changer in how AI systems augment the profession. This mechanism allows AI agents to autonomously refine their learning algorithms, driving continuous improvements in model performance and efficiency. This means AI systems will be able to autonomously optimise data pipelines, ETL processes, and data quality checks, reducing the need for manual intervention. Moreover, RSI-enabled AI agents can continuously update and refine data governance policies based on evolving regulatory requirements and best practices.

To bring this into focus, let’s explore what a day in the life of a data professional is going to look like over the course of the coming years…

👋🏽 Meet Your New Polymath Colleague

Picture this: It's a crisp morning in 2025, and you're settling into your workstation with your coffee. But instead of diving straight into Stack Overflow or frantically messaging colleagues, you're greeted by your AI polymath assistant - a sophisticated AI system that combines deep expertise across multiple disciplines with the ability to understand and execute complex tasks.

This isn't just another chatbot. Your polymath assistant represents a quantum leap in AI capabilities, trained on vast repositories of knowledge and able to reason across multiple domains simultaneously. Through recursive self-improvement, it's constantly getting better at making itself smarter, leading to exponential growth in its capabilities.

This means you can lean on the Agent for both depth and breadth of understanding as you tackle day-to-day tasks. Everything from Data Science questions, to ethical dilemmas, to optimisation and Governance questions can be directed at your very own modern day Leonardo Divinci.

🪨 Morning: Shifting a Big Rock

9:00 AM - Complex Data Modeling

Your first task of the day is designing a new data model for integrating multiple data sources. Instead of spending hours pouring over documentation, you simply describe your requirements to your polymath assistant - Leonardo.

"Hey LEO, I need to design a Data Vault model for our new customer data integration project. We have data coming from our CRM, billing system, and support tickets."

LEO, drawing on its comprehensive understanding of Data Warehouse Architecture principles, immediately starts breaking down the problem:

"Let's structure this using the three core components of Data Vault 2.0:

• Hubs for business entities, like customers, and products

• Satellites for descriptive attributes and temporal tracking

• Links to capture relationships between entities”

The assistant then generates a visual model on a canvas, explaining how it ensures:

• Zero-impact changes for future modifications

• Support for multi-speed data loading

• Built-in security features

• Automated pattern-based structures

While this is a simplification of this scenario, it illustrates the emerging possibilities of having a Polymath Assistant at your finger tips. Even more relevant is how you will be able to interact via voice conversation, transforming the interaction into a more intuitive collaboration.

🏃🏽‍♀️Midday: Text-to-Action Magic

11:30 AM - Query Generation

Thanks to advances in text-to-action capabilities, your LEO can translate natural language into complex technical implementations[4]. When you need to query your newly created Data Warehouse, you simply explain what you're looking for:

"I need to analyse customer churn patterns by combining support ticket sentiment with billing history."

The assistant, leveraging its nearly infinite context window, maintains awareness of your entire data model while generating optimised SQL queries. It doesn't just give you the query - it explains the logic, suggests performance optimizations, and highlights potential data quality considerations.

Stay up-to-date with AI

The Rundown is the most trusted AI newsletter in the world, with 1,000,000+ readers and exclusive interviews with AI leaders like Mark Zuckerberg, Demis Hassibis, Mustafa Suleyman, and more.

Their expert research team spends all day learning what’s new in AI and talking with industry experts, then distills the most important developments into one free email every morning.

Plus, complete the quiz after signing up and they’ll recommend the best AI tools, guides, and courses – tailored to your needs.

Sign up to start learning.

🤯 Afternoon: Deep Dive and Integration

2:00 PM - Business Merger Problem Solving

You're facing a challenge integrating data from a newly acquired company into your existing Data infrastructure and models. The new data source has inconsistent business keys and a different temporal tracking method. Here's how LEO would approach the problem:

1. Data Architecture Analysis: The assistant quickly analyses the existing data structures and the new data source, identifying discrepancies in business key definitions and temporal tracking methods.

2. System Performance Optimisation: It suggests creating a staging area to transform the incoming data, ensuring minimal impact on the existing Raw Vault performance. The assistant generates optimized SQL queries for this staging process.

3. Business Process Analysis: Leo examines the business processes of both your company and the acquired one, identifying areas of overlap and divergence. It proposes a mapping strategy that aligns the new data with your existing business entities.

4. Statistical Modeling: To address the inconsistent business keys, the assistant performs a statistical analysis to identify potential matching algorithms. It suggests a probabilistic matching approach using machine learning techniques to link records across systems.

5. Security Best Practices: Recognizing the sensitive nature of merging data from different organisations, LEO outlines a comprehensive data governance strategy. It proposes encryption methods for sensitive attributes and suggests access control policies.

6. Implementation Plan: Based on its multidisciplinary analysis, the AI assistant generates a detailed implementation plan. This includes:

   • A modified hub structure to accommodate the new business keys

   • Additional satellites to capture the different temporal aspects

   • Link tables to maintain relationships between existing and new entities

   • ETL processes for data cleansing and transformation

7. Code Generation: The assistant then produces the necessary SQL code to implement these changes, leveraging Data Vault 2.0 best practices and automation techniques.

8. Documentation: Finally, it generates comprehensive documentation of the changes, including data lineage information to maintain auditability.

Here's where the true power of a polymath assistant shines. When you encounter a complex business integration issue, LEO doesn't just offer a single perspective. It combines knowledge from:

• Data architecture principles

• System performance optimisation

• Business process analysis

• Statistical modeling

• Security best practices

• Ethical considerations

🚀 The Game-Changing Difference

What makes a Polymath Assistant truly revolutionary in the context of time to value is its ability to:

🤔 Think Across Domains Unlike human experts who typically specialise in one area, your polymath assistant seamlessly integrates knowledge from multiple disciplines. When solving a data engineering problem, it might combine:

• Mathematical optimisation techniques

• Software architecture patterns

• Business domain knowledge

• Regulatory compliance requirements

• User Experience Design considerations

• Enterprise specific policies and procedures

📚 Learn and Improve Through recursive self-improvement, your assistant becomes increasingly adept at:

• Understanding your specific needs

• Anticipating potential issues

• Suggesting innovative solutions

• Optimising its own responses

🏪 Execute and Implement With advanced text-to-action capabilities, the assistant can work for you 24 hours a day to:

• Generate code and queries

• Create documentation

• Design data models

• Automate routine tasks

• Suggest improvements

The potential productivity boost will be interesting to see as these agents start to augment our lives both professionally and personally.

🔭 Looking Ahead

As these AI assistants continue to evolve through recursive self-improvement and expanded capabilities, they'll become increasingly sophisticated partners in your daily work. The combination of nearly infinite context windows, advanced text-to-action capabilities, and deep multi-domain expertise will transform practice of data management. We don't think they will replace us, but rather augment our day-to-day workflows. What is most likely is, that for those that embrace this technology, it will elevate your role. With a polymath assistant handling the heavy lifting of implementation details, you can focus on:

• Strategic architecture decisions

• Innovation and experimentation

• Business value optimisation

• Complex problem-solving

• Team collaboration and mentoring

• Exploring new innovations, optimisations and models

While your AI polymath assistant might have encyclopedic knowledge and lightning-fast processing capabilities, we firmly believe that your human creativity, intuition, and strategic thinking that is the value your bring, coupled your passion for solving real-world problems. This is a transformation that may take a year or ten, but it is surely happening. We are excited for what 2025 will bring, and how data professionals will shape the future of how organisations will understand and utilise their information assets.

That’s a wrap for this week

Happy Holidays Dats Pro’s