With the recent advances in generative artificial intelligence tools, such as ChatGPT, there is a need to understand what AI can and cannot do. Generative AI tools can only reflect the current state of human knowledge and available information. For a subject that is not controversial—examples are the results of every Super Bowl or the details and structure of a specific type of legal agreement—the results of a query are unique and accurate. I think of it as Google on steroids. But when the topic is controversial or in evolution, the generative AI tool can exhibit the same cognitive biases and errors that we exhibit as human beings. To demonstrate this point, I wanted to explore the application of science and healthcare that I have been passionately studying over the past 20 years.

My motivation comes from learning that our global healthcare system is unsustainable. After the first decade of my studies, I learned that our healthcare system is unsustainable because it is founded on the principles of reductionist science. Over the past century, we have learned that reductionist science is not a valid scientific paradigm. Fortunately, a newer paradigm based on systems and data science is available to apply to healthcare and our world in general.

As an introduction, I first questioned ChatGPT to learn how it was created and its benefits and weaknesses in its application to science and healthcare. The answer describing how it was created is simple and understandable. However, it does admit a potential for errors.

I then started with a series of questions about our current scientific paradigm shift from reductionism to systems science. The first question is to clarify the fact that randomized controlled trials (RCTs) are a reductionist science tool.

Then I confirmed the assumptions that are required for RCTs to be valid.

Clearly, none of these assumptions are true. So, after confirming that RCTs are not valid, I wanted to confirm that we are going through the first scientific paradigm shift since the Renaissance, when reductionism was discovered.

Remember that the ChatGPT answers are only patterns based on predictions from all available information created by humans. In this case, there is a lot of denial and cognitive dissonance about letting go of the principles of reductionism and tools like RCTs. So, after answering in the affirmative that our world is going through a scientific paradigm shift, the ChatGPT cognitive dissonance kicked in.

After spending nearly 20 years studying this scientific paradigm shift, I knew that this last statement was false, so I asked a straightforward question.

With this more direct question, the ChatGPT answer was correct, directly contradicting the previous statement.

And finally, I asked if it could recommend or predict the best value treatment or product for a specific patient problem.

With a clear understanding of the limitations of generative AI, we should recognize the importance of a human-computing symbiosis that is the foundation of systems and data science principles that can produce AI that we desperately need in healthcare—that is, actual intelligence. Applying the principles of systems and data science using human-computing symbiosis is how innovations and highly accurate predictive algorithms can be generated to lead to a sustainable healthcare system and improve our world.

For healthcare to be sustainable, we must change our thinking and break free from our reductionist mindset that craves certainty and control. The concepts of certainty and control are harmful illusions. The mindset of systems science can overcome this reductionist thinking and allow our human potential for empathy, creativity, discovery and innovation to be unleashed.

We’re all in the global healthcare system together, and we need one another, learning together with the incredible computing capabilities of a variety of analytical tools applied to real patient care and networking those learnings and algorithms that will allow for a sustainable healthcare system based on value for patients to emerge. Who doesn’t want this? Why wouldn’t they?

Ramshaw is a general surgeon and data scientist in Knoxville, Tenn., and co-founder and CEO at CQInsights.