SOURCE: Kay Sever | March 25, 2026
After being on an AI learning curve for the past several months, I have decided it’s time to write about my observations, insights and possible ways to selectively apply AI within the optimization framework to create a better result. Therefore, our topic for 2026 will be the intersecting points of Optimization and AI as they relate to productive assets, the organization and the management system. We are told that AI will “know all”, so we expect to find it helpful no matter the question asked or problem to be solved. The more you can learn about AI models (LLMs: large language models) and their strengths and weaknesses, the clearer your vision will be about how to incorporate an AI model into your workflow.
Last month I said that this month’s topic would be problem solving. Again, because of what I learned about AI since then, I have changed the topic to integration and accuracy because executives and leadership teams need to understand AI’s current limitations to manage their expectations about the benefits AI offers. They must also reach agreement on how they want to integrate AI into departments, specific functional tasks and workflows before AI can be used effectively anywhere in a company… big decisions that require sound information.
When we read news about AI, often the topics cover AI data centers, the power they require and differences in processing speed, accuracy, depth of LLM “knowledge”, etc. Comparisons are continually being made between LLMs like Grok, ChatGPT, Gemini, Claude and others. Ultimately, the most important thing to end users is how AI can be integrated into research, analysis, workflow, problem solving and decision making to improve the performance of the companies you manage or work for.
Because of the speed of development and rapid (almost daily) changes in LLM capabilities, it is easy to be pulled into the details of the capabilities, but it can be difficult to gain a “big picture perspective” of how this whole AI thing is really going to work and what unforeseen circumstances may occur as you move forward with AI integration.
AI Integration Challenges
With all the present unknowns about AI capabilities in the “real world workplace”, your team will need to be able to answer several questions and reach consensus on the answers before you move forward:
- EMPLOYEES:
- What do you want AI to do for your employees to enhance their productivity and capabilities? What tools are they lacking that AI could provide at a low cost?
- What kind of training would be required to make employees competent at AI queries? This is a critical point – the more specific the query, the more accurate or focused the answer will be, which means that knowing how to “ask” AI for information or an analysis is key to getting an answer that can be used for decision-making.
- DATA SHARING/CONFIDENTIALITY:
- How do you want to integrate AI with your current data systems (i.e., what data do you want to share with AI). From everything I have read up to this date, once data or information is shared with AI, two things happen: it is no longer confidential or proprietary and you no longer own it (per leading AI developers). We hope this changes in the future.
- Do you want AI to refine or summarize your company reports? As of this date, if you ask AI to refine an article/report you have written, AI now owns that article – you or your company no longer owns it. As of this writing, an AI agent can share that information with other AI agents or anyone without your permission or knowledge.
- DATA SOURCES:
- If you want to replace some functions performed by employees with AI “agents” that do transactions, analyses and reporting, there are important things to know before you begin eliminating people. If AI performs transactions, analyses or reporting, you may not know what information the AI used or if the answer is correct. You cannot assume that AI will access the same sources of data or information that a trained employee would use to perform a task. Key information could be left out of an analysis without your knowledge. See more information about this in the next section below.
- It is possible to tell AI which data sets, files, etc. are to be used for an analysis or report. Specific instructions for data sources must be included in the task description and the instructions must be written using language that expands or limits the AI’s scope for the task. Links to those data sources must be built before the AI request is composed.
Accuracy Concerns – I Asked AI How Accurate It Was – Four Questions
I have analyzed production data (operations and maintenance), costs, revenues and optimization data for over 40 years. Without exception, I double-checked my work to insure I was generating data and charts that were 100% accurate so they could be used to quantify the impact of needed process changes, justify capital expansions, reduce non-value-added costs, and expose hidden profit opportunities that were real.
Because accuracy is such a high priority in my work, understanding AI’s accuracy was of keen interest to me. To understand more about the accuracy of LLMs, I decided to ask a major LLM four questions about AI data sources and accuracy of AI answers and analyses. Here are my questions and the answers exactly as they were stated by AI:
QUESTION #1: If AI is asked to research a topic, is there any way to know the sources it used? Is there a metric or indicator that would prove that an LLM or AI agent performed “comprehensive, multi-source research” versus just using information it had in its database? I asked this question in behalf of engineers who might research the latest methods for tailings processing, blasting techniques, metal recoveries, flotation chemicals/processes, etc.
ANSWER:
“No, there is no single foolproof signal that proves an LLM or AI agent performed “comprehensive multi-source research” versus relying mostly on “parametric memory”. However, in 2026 you can usually make a high-confidence judgment by looking at a combination of “observable behaviors” and structural clues in the answer.”
-
- Explicit in-line citations, verifiable links, news outlets mentioned.
- Mention of very recent events (last 90 days) of prices, SEC filings, etc.
- Diversity of sources reflected in the synthesis.
- Inconsistent/non-repetitive answers when you ask the exact same question 5 to 10 times, which indicates “live” search, not just referencing a stagnant database.
- Model shows its work with intermediate steps that imply search.
- Correct handling of fast-changing topics with very recent threads.
- High semantic entropy / confidence qualifiers on new topics – may say “sources are conflicting, found limited data”, etc.
- Quick diagnostic checklist:
- Are there multiple inline citations to different domains published in 2025-2026?
- Does the answer contain specific facts that weren’t widely known before Oct 2025?
- If I ask the exact same question 5 times, do I get meaningful different sources?
- Does it correctly reflect very recent news/numbers/drama in the last 30 days?
- If 3-4 yes answers, very likely did broad live/agentic research. If 0-1 yes answers, most likely relying on internal knowledge.
QUESTION #2: Is there any way to confirm that a specific LLM or AI agent had access to online articles during its research process? I asked this question on behalf of anyone hoping to find information published in on-line professional journals and industry publications (i.e., Mining Engineering, Aggregates and Mining Today, etc.)
ANSWER:
“No, there is no definitive 100% reliable way to confirm that a specific LLM or AI agent had access to particular online articles during its research process. The strongest indicators of on-line sources used are:
- Explicit in-line citations to exact articles used.
- Unique non-generic details pulled directly from the articles (statistics, quotes, etc.)
- Inconsistent/non-repetitive answers when you ask the exact same question about an article 5 to 10 times, indicating “live” retrieval.
- Articles referenced published in the last 30 days.
- Explicitly ask for all sources to be listed.
- Articles that present views that are counter to “common knowledge”.
- The response weaves in multiple angles from articles.
QUESTION #3: How do I know if a mathematical calculation was done correctly and can I depend on the answer being right every time? I asked this because it is very likely that AI will be asked to do mathematical calculations for engineers, accountants, analysts and executives. Important decisions will be made using those numbers. We have to know if the numbers are right.
ANSWERS:
“All the large LLMs are better at arithmetic and mathematics than they were in 2023-2024. However, they are still not perfectly reliable for certain calculations”:
- Very large numbers / high-precision arithmetic
- Multi-step calculations with many carryovers or borrowings
- Problems that require exact symbolic manipulation beyond when token patterns capture well
- Cases involving floating-point precision
- Extremely long chains of reasoning
“How to tell if a specific answer the LLM gave you is correct”:
- Copy the steps into Python or any scientific calculator.
- Re-ask the exact same question 3-10 times and compare answers. Variation means uncertainty.
- Ask for a step-by-step breakdown of the calculation and verify each line.
- Reverse engineer/check backwards – plug the answer into the equation and see if it can be solved.
- Ask a second LLM the same question and compare answers.
- Look for the language used to return an answer: “The result is exactly _____”
QUESTION #4: Do LLMs have access to all books, articles and research papers ever published? I asked this question because we heard at the beginning of this AI push that AI would be like a giant encyclopedia and would hold the world’s knowledge. For this reason, it was touted as a teaching aid for our children. If we ask AI to research a problem or the latest technology, we need to understand its content limitations.
ANSWER:
“No, absolutely not. No LLM, not today in 2026, and not in any foreseeable future, has access to all books, articles and research papers ever published.” Current content includes the following (per the LLM I queried):
- Books – Publishers block most “scraping” of books because of copyright. For books published in years 1980-2025, 5-25% are accessible.
- Academic research papers – 40-70% published between 1990-2025 are accessible.
- Articles on websites:
- Very popular websites (60-95% accessible)
- Medium popularity news, blogs, substacks (40-80% accessible)
- Niche websites, industry wikis (5-30% accessible)
- Non-English websites (5-25% accessible)
- Training datasets – often pay-walled (<10-20% accessible)
- Content published in the last 30 months (0-30% accessible)
- This means LLMs contain maybe 5-20% at most of the total “crawlable” public web and far less of the total internet.
- The web is not static – 300-500 million pages are published/day (rough 2026 estimate).
- Legal and technical barriers keep growing and AI crawlers are “blocked”.
- This LLM summarized its answer: “No LLM (even the biggest ones) has been trained on more than a small minority of all websites and articles ever published on the internet… The idea of an LLM that has “seen the whole internet” is still science fiction in 2026.”
My Summary
Many very important decisions will be made based how AI is implemented and what the end goals are. If we understand its current limitations, that understanding
- Helps us manage our expectations about what AI can do and can’t do at the present time.
- Helps us make better decisions about how, where and when to integrate it into work processes, at least in the near term.
- Helps us determine if it is a good replacement for humans in the workplace at the present time.
I learned a lot because I asked these four questions. I hope this article gives you some new perspectives to consider as you negotiate your path forward with AI.
Next month’s topic will depend on what I learn about AI in the next 30 days.
Kay Sever is an Expert on Achieving “Best Possible” Results. Kay helps executive and management teams tap their hidden profit potential and reach their optimization goals. Kay has developed a LIVESTREAM management training/coaching system for Optimization Management called MiningOpportunity – NO TRAVEL REQUIRED. See MiningOpportunity.com for her contact information and training information.
To comment on this story or for additional details click on related burros above.
- About Us
Kay has worked side by side with corporate and production sites in a management/leadership/consulting role for 35+ years. She helps management teams improve performance, profit, culture and change, but does it in a way that connects people and the corporate culture to their hidden potential. Kay helps companies move “beyond improvement” to a state of “sustained optimization”. With her guidance and the MiningOpportunity system, management teams can measure the losses caused by weaknesses in their current culture, shift to a Loss Reduction Culture to reduce the losses, and “manage” the gains from the new culture as a second income stream.
