Name: A.I. for business: Advice from a top investor at Google's Gradient Ventures
Uploaded: 2023-09-06
Duration: 13 min 33 s
Description: Darian Shirazi, a general partner at Google's A.I.-focused Gradient Ventures, discusses ChatGPT blindspots, the risk of A.I. hallucinations, and how bigger firms should be approaching the large language model revolution.

Darian Shirazi, a general partner at Google's A.I.-focused Gradient Ventures, discusses ChatGPT blindspots, the risk of A.I. hallucinations, and how bigger firms should be approaching the large language model revolution.

A.I. for business: Advice from a top investor at Google's Gradient Ventures

Tech

Darian Shirazi discusses ChatGPT blindspots, the risk of A.I. hallucinations, and how bigger firms should be approaching the large language model revolution.

Darian, thanks so much for being here with us. Um You have an interesting perspective, you know, you've been on the investment side, obviously, also on the founder side though, and the bigger tech company side. So I want to start out by just asking you from what you're seeing out there from your experience. Where do you think, which industries, which types of businesses are best positioned to benefit from the kind of productivity um advancements that we're hoping to see from A I I think every industry is gonna sort of have some kind of impact from large language models and artificial intelligence. We're seeing a number of verticals that are being impacted and, and impacted positively in terms of being automated. The first one that comes to mind for me is the legal tech realm. We're seeing a lot of law firms start to adopt LLM SI was the first investor in case text which basically automated the process of summarizing briefs and looking at red lines and looking at different things within law law reviews and Reuters bought them for 650 million this month. Uh So we're seeing a lot of disruption in legal tech, we just led a series in a company called File Read, which is automating e discovery with large language models. And the team came from Caltech, they sort of built the technology themselves and now have large contracts with law firms that are looking to have an edge in litigation. So I think that's one vertical that I would say is the first that's, that's already experiencing a strong sort of innovative, innovative lever behind them and and seeing growth from LM. And is there anything specific to that industry that um kind of makes the the needs and the the use case is very clear and the ro I there um but also the kind of cultural acceptance of it. Are you seeing that there too? Yeah, legal Tech has historically been a difficult place for venture backed companies to succeed. So it's kind of a role reversal where we're seeing that as a place where there's already been a big liquidity event, there continues to be a lot of active investment. I think that comes down to the fact that the majority of the content that lawyers deal with is text based. And so these large language models are particularly good at summarizing information that lawyers necessary would have to read through extensively. So I think the technology is very well suited for that vertical to start, but there are also others. So healthcare, I think is another one, there are a number of companies that are starting to do summarization of insurance data to understand whether a specific diagnosis code is actually going to be mapped to a real insurance plan. And what the cost breakdown would be that usually has to be done with labor reading through the insurance materials and then assessing what the insurance breakdown would be. Sometimes that leads to a dispute. We're also seeing in a lot of other areas such as computer vision. I spoke to a company today that is now using uh image models to basically detect whether a skier has fallen from a ski lift or not when they're about to get off of a ski lift. So they don't need a lift at the top of the lift. It's kind of a weird use case, but it gives you a sense as to the level of detail that some of these companies are going to that the ski industry is suddenly starting to adopt image models and the company is selling a generic LM testing platform. But one of their first customers happens to be a ski lift company. So I thought that was kind of interesting to see that even these long tail weird use cases are already seeing adoption with from these large language models, image models, et cetera. The more uh the more obvious ones to us I think are general productivity. We're seeing a lot of companies such as notion which is an incredible platform. They've integrated A I and A lot of their customers are now using A I to generate content and automate and speed up their work. Um We're seeing Google adopt it, Microsoft etcetera. General productivity will be obviously impacted positively in many ways by LM. But I think that legal and health tech are gonna be two areas that we wouldn't expect. And it's only the beginning, every single software company is gonna have to change their mindset because we're no longer buying these solutions that sort of allow you to put things in the cloud. We're also buy, people are gonna want to buy solutions that are automatically intelligent that can give them the correct answer. And that's gonna disrupt and change every single vertical as a result of that will be significantly more jobs. I think, I don't think it's gonna take away jobs. I think it's gonna make it such that there will be more of a need for training data to make these applications even more intelligent over time. Unless you're working a ski lift, maybe the lift will have more time to ski. But I thought that use case was particularly funny and interesting. So I wanted to not where your mind goes to right away when you think of language models or A I is more broadly. Um I wanna ask you about blind spots. Um What do you think are going to be some of the, the, the biggest blind spots and the biggest hurdles for companies, you know, you're talking about industries with uh the legal profession, with health, health care um that aren't necessarily known as early adopters of new technologies. So how are big companies looking at actually deploying some of this stuff? Yeah, I think, I think that there are a couple of challenges. One is citations is that currently we really don't have any sense within chat GP T or GP T 3.5 and four. What, where the content is coming from, where the specific answer is actually coming from. And that is a very complicated project, one that I know that is being worked on by a lot of other A I researchers and something that I think is going to have to be resolved. Because ultimately, if a large language model learns from this interview, fortune should get credit for it and I should get credit for whatever I've said as well. I believe unfortunately, the current models don't allow for that and we have no idea where it actually learned the specific answer. So that's one where I think we're going to need to see a lot of innovation. I think second blind spot is definitely data privacy is that currently large language models are learning information that may be personally identifiable. We need to figure out how to filter that information out. We're seeing a lot of pitches these days of companies that are helping with that process to make sure the training data isn't training data that can't be used in a large language model. I also think that answers being incorrect, being able to tune these models and tune the responses to say, hey, this is incorrect or correct. Ultimately, if we have a health care company that's based on L MS and it's helping a surgeon when they're in the middle of surgery and the answer is incorrect, that could have a really bad outcome. We can't have those kinds of outcomes. And so those are really three areas that I think are big blind spots that need to have significant research done before. A lot of the applications that will emerge over the next 10 to 20 years are actually viable. You don't want any hallucinations for health care applications. I'm guessing that sounds exactly. Yeah. Um Do you have any advice for again, just going back to some of the bigger companies that are, you know, testing it out if not fully deploying some of these A I powered applications and how not to just fall into some of the hype not to, you know, in some of this, you don't want to just throw spaghetti at the wall, right? You want to get it right. There's a lot at stake. Um What's, what's your advice on that front? I think the key thing is to realize that there's going to be many different steps along this journey, the same way that there have been in a lot of super technology super cycles. And I think in the case of the large language model community, companies should really start to adopt different models but not be completely invested in one ecosystem yet should really think about. OK, let me test open A, I, let me test all the different models that are out there before making a fully baked decision that so I'm gonna base my whole next generation of my business on this. And I think that there are many different platforms that are gonna help with the process of testing and tuning and ensuring that they actually pick the right models for the right application. So I think it's it's definitely early days, but there are a few key things that can really improve the experience for customers across different industries. Right, right away, I think Legal Tech is one, we also invested in a company called Range, which is in the wealth management space that allows for people that has automatic advice on where to invest money based on market changing conditions and allows people to ask questions about their money. There's definitely sectors or areas that are going to benefit immediately. And I think it's just take it step by step and not go all in on a platform just yet until there's been more development in this entire ecosystem. So um speaking of some of your investments, uh I'm sure that you are getting all sorts of pitches from all sorts of companies that have tacked on A I to their, their name and description. Um How are you sifting through all of these different investing opportunities and, and what do you look for as an A I investor? Yeah, it's, it's, it's busy. I mean, today I saw eight new companies and that's like probably the average day. I, I would say that everyone's trying to build something based on these large language models or building their own foundational models or customizing for different industries and applications. It's difficult. I we, we have a relatively small team and we're very specific about the people that we bring on because you have to have the right blend of technical skill set as well as business minds set in order to identify the right the right opportunity. I have a rule set that I usually follow for founders. I really, I really want to make sure that I can trust the founder and that I can trust them probably for the next 10 years because ultimately, that's the journey that we're signing up for. When we invest in a company, we expect to be with that company for 10 years or maybe longer. If you look at every major IP O that's out there. That's how long it really took from founding to exit. I think that second is I'd really like to know that they're gonna be resilient. There's gonna be many different waves, there are gonna be different phases within this A I uh renaissance. So to speak and we need to make sure that founders are willing to sort of roll with the punches. And if there happens to be sort of a disillusionment around A I in the next couple of years, we need to know that they're going to dig in and really build the right company and continue to execute on their vision and plan and crisis is common in start ups. You have to be able to survive through all of the different crises that come your way, whether they're, you know, people related or technology related or customer related, there are just so many different challenges that founders are going to experience. And then I think the third is, do I really believe that the founding team can learn and learn quickly over time and as they sort of evolve, the company go from being this project where they're looking to find product market fit and then becoming a company where they're monetizing and working with customers. And then ultimately scaling the business, are they sort of the founders with the right level of plasticity to survive all three phases is a really big, big deal for me because I don't want to be in the business of finding a professional CEO for a company. I want to work with the founders from the beginning to the end. And so I really look for that in the early stages. So far, I've been pretty good at finding these different characteristics and I think the rest of our team has as well and we just continue to look for those things and quickly screen for it. Um As far as like technical viability, those are things that are sort of innate in our team. Like we understand what is possible and not possible in A I. And I think that's what's given us a leg up in a lot of these conversations with portfolio companies vis A vis other investors that maybe haven't been investing in A I for the last six years. OK. Um So last question for you, I know most of the companies you're talking to are probably not trying to do what open A I is doing and they're not trying to, you know, build from scratch these large language models. But um a lot of these companies probably still require quite a bit of capital, compute power, all of that, right? A lot of resources. Um Do you think that in this particular case with generative A I that bigger companies could have the upper hand versus the startups and who have, you know, historically been the innovators? How, how do you look at that? Well, I think it used to be the case for General cloud but in the A I era, I think that's actually changing. I mean, NVIDIA has become the kingmaker in terms of allocating GP US. They've allowed companies, they're allocating GP US to companies like core we and LAMBDA lab. LAMB is one of our investments and they really do or they really are trying to spread the amount of compute power across all these different companies. I actually think that software innovations are going to allow for compute to be faster and cheaper over time. So I would say that in the A I era, there's actually more democratization than there was in the previous cloud era and that's been promising, especially as a venture investor. So I I would, I would take the opposite of that, that I think that this is definitely going to be an area where another, you know, trillion dollar company or multiple trillion dollar companies will emerge in because they're able to access different types of resources from NVIDIA and that there's more democratization of the compute power over time. OK, fascinating stuff, Darian. Thank you so much for being with us. Absolutely. Thank you for having me.