My personal experience with ChatGPT and a vision for the LLM future

This week, I would like to share my personal experience using ChatGPT for medical purposes and another interesting article that offered an abstractive but intuitive way of thinking about the future of GenerativeAI.

I had to visit the dentist this week to get some pending procedures done. Sitting in the dentist's chair is never a pleasant experience and I'm always looking forward to the end when the doctor recommends that I have a lot of ice cream! The next morning I received a text message from the clinic staff asking how I was feeling and if there was any pain or discomfort. This is not the first time that I've been to this practice but each time I get this message it always feels nice that someone asked about me. On previous occasions, I've had bleeding overnight where they also provided advice or solutions and I find this experience to be a simple yet personalized way of post-care treatment. Given my experience building chat automation, it was clear that ChatGPT or other Large Language Models (LLMs) could be used to automate this interaction but I was wondering whether I would be able to tell the difference and appreciate it as much if I knew.

That's where I came across this study where the authors compared the responses to medical questions provided by ChatGPT with the responses by certified medical practitioners to the same questions on the subreddit. The comparison was done manually by a team of licensed healthcare professionals who concluded that ChatGPT responses were preferred in 78.6% of the cases. I found this to be very interesting because it means that people with medical questions would have found responses from ChatGPT to be more empathetic than the responses they received from the doctors on the forum. In my case, it means that if my dental practice should choose to automate their post-care follow-up, I might find the experience even more delightful! There were several comments and opinions expressed by other readers about the article pointing out that since ChatGPT provides longer responses, it might appear more empathetic than short responses from doctors who are generally busy. Another interesting comment was from a reader -

Having recently undergone major treatment at a well-known university hospital, I can say that once AI has been trained, it has to be better than the current system. The doctor I was referred to saw me pretreatment twice. After hospitalization I never saw her again, nor did she contact me. Every week there was a new doctor leading the team and after discharge every visit was by a different APN and once by a different doctor. The current system is broken and there is no patient-physician relationship. Anything we can do to improve the situation is worth a try.

... which points towards a feeling of not being valued or cared for - something that could potentially be managed by an LLM-powered solution with access to hospital records. So indeed, ChatGPT may not replace a doctor but could potentially help make the patient experience better.

The other time I relied on ChatGPT this week was much more complicated and critical because I was trying to help my father who has been diabetic for several years and has been struggling with managing his blood sugar levels. He has recently started taking insulin injections to maintain his blood sugar levels and has suffered multiple instances where his blood sugar levels go low, also known as hypoglycemia which can be quite dangerous. He was finding it very difficult to identify patterns like when this happens and why. We got him started with a Continuous Glucose Monitor (CGM) which helps to paint a reasonably accurate picture of what happens to his blood sugar when he eats a meal and until the time the insulin starts acting. That was one part of the solution, but I relied heavily on ChatGPT to understand and make sense of the metrics and data the CGM was providing. In my conversation with ChatGPT, I was able to deep-dive into how different types of insulin act and help the body maintain blood sugar levels within the recommended range. I was able to understand the daily sugar level graphs from the CGM, what each spike or fall meant, and understand the metrics. For example, it helped me recognize how the value reported by the CGM which indicates the real-time level of blood glucose is correlated with the HbA1C that is measured in a blood report and often referred to by doctors. I had a long and detailed conversation with the use of documents, graphs, and images and it helped us understand what was happening and gave indications for what kind of actions could be taken and what impact those actions could have. It didn't function like a doctor but it was helping us understand the mechanism within the body. Based on follow-up consultations with the doctor, we are making the necessary changes and have seen positive outcomes so far. There are regular follow-up appointments but this conversation helped me understand what happens in the body and gave me a lot of confidence which might have taken much longer for a doctor to explain to us.

The use of ChatGPT in critical medical care is going to be tricky but my experiences this week indicate that while they will never replace doctors, there is a role they can play in making the patient experience more informative, empathetic, and delightful.

This brings me to the broader question of the future of LLMs and where we might see their applications. I read a very interesting article where Daniel tries to answer this, not by looking at the technology itself but by understanding humans and what we desire. He hypothesizes that as humans we will always want safety (we want our family and ourselves to be secure), success (we want to feel happy and achieve things) and we are social (connect with others like us). And he tries to imagine how LLMs can be used to help humans achieve these goals. His core concept is that of a Personal Digital Assistant (DA). This is like your own Jarvis that knows a lot about you and helps you with several tasks, from something as simple as booking a flight to something more critical like keeping your loved ones safe. He further expands on this concept and illustrates how your DA would connect with various APIs or daemons - offered by businesses (e.g. flight prices API from Booking.com) and also people (personal interests daemon by Sam the bartender) and would assist you - be it planning a trip or striking a conversation. While the idea sounds futuristic, you can see that many of these elements already exist. Most businesses provide APIs since they want users to use that information to make decisions. Most people also provide this information at the moment - like their career history on LinkedIn or hobbies on Facebook because they want to connect with others like them. What LLMs unlock is the ability to bring together these vast amounts of information that we already have access to but do it faster and more intuitively.

I like Daniel's style of thinking and loved reading his article and thesis. He also addressed some of the biggest threats around cybersecurity and the sharing of personal data which to me indicates that the market for these DAs would be rather fragmented. One way to think about how the solutions could evolve would be a set of simple assistants (or Role-Based DAs as Daniel describes it) which already exist, the most popular example being Github CoPilot. In that sense, I am building my own Podcast DA with Podsumo - an assistant that listens to my favorite podcasts and then informs me about the most interesting parts. This is already personalized to me and helps me thrive. I can also run it locally to overcome the threat of sharing personal data. Maybe this is the direction that Apple is going in with the release of the MLX framework, and having everything run on the device. If I were to imagine the next step in my own DA journey, it might be a Writing Assistant DA that connects with my Podcast DA to retrieve important quotes or themes that I can refer to while writing a newsletter like this.

One thought that came to mind as I was reading the article was - does this not exist today? Google already knows when your flight is booked and matches that with your location and traffic conditions to inform you that you should be leaving soon. I was quite impressed with this feature when it came out. I thought that Google Assistant would be able to do all this and much more - given it knows your search history, cookie-based interests, maps, etc. But why did it not work well? Was it because of issues with retrieval and latencies - you need to know exactly what piece of information to use and when. Will the use of Retrieval Augmented Generation (RAGs) speed up that process? I think Perplexity is doing a fantastic job with search-based retrievals. Or, did it not work well because it was coded with a set of rules and scenarios, IFTTT-style which is where LLMs can have greater leverage without the need for explicit rules? Or, did it not work well because we as humans are inherently different from each other in how we organize our lives - our email lives in Gmail but also in Outlook, our messages are in texts but also on Whatsapp and we book flights not just for ourselves but our families; the Google Assistant just could not accommodate all these scenarios. Is that something your personalized DA, powered by an LLM running locally, connected to your apps and data be able to solve? Time will tell.