Clearly we need to do something about how we talk about open source and openness in general. It’s clear since 2006, at least, when I was rightly nailed for calling Google and Yahoo! to curb open source. As Tim O’Reilly wrote at the time, in an era of open source cloud, “one of the motivations for sharing, the need to give a copy of the source to allow someone to run your program, is really gone.” . In fact, he continued, “Not only is it no longer necessary, but in the case of larger applications, it is no longer possible.”
This inability to share has altered the definition of open source over the last decade, and is now affecting the way we think about artificial intelligence (AI), as Mike Loukides recently pointed out. There has never been a more important time to collaborate in AI, but there has also never been a time when doing so has been more difficult. As Loukides describes, “Because of their scale, large language models have a major problem with reproducibility.”
As with the cloud in 2006, companies doing the most interesting work in AI may have a hard time “opening up open source” the way we traditionally had expected. However, this does not mean that they cannot yet be significantly opened.
Good luck using this model on your laptop
According to Loukides, while many companies may claim to be involved in AI, there are actually only three companies driving the industry: Facebook, OpenAI, and Google. What do they have in common? The ability to run massive scale models. In other words, they’re doing AI in a way that you and I can’t. They are not trying to be secrets; they just have an infrastructure and knowledge on how to run that infrastructure that you and I don’t have.
“You can download the source code for Facebook’s OPT-175B,” Loukides admits, “but you won’t be able to train it yourself on any hardware you have access to. It’s too big even for universities and other research institutions. “You still have to believe the word Facebook does what it says it does.” This, despite Facebook’s big announcement that it was “sharing Open Pretrained Transformer (OPT-175B) … to allow for more community involvement in understanding this fundamental new technology.”
This sounds great, but, as Loukides insists, OPT-175B “probably can’t even be played by Google and OpenAI, even though they have enough computer resources.” Because? “OPT-175B is too tied to Facebook’s infrastructure (including custom hardware) to be played on Google’s infrastructure.” Again, Facebook is not trying to hide what it is doing with OPT-175B. It is very difficult to build this infrastructure, and even those with the money and knowledge to do so will end up building something different.
This is exactly the point made by Jeremy Zawodny of Yahoo! and Chris DiBona of Google in 2006 at OSCON. Sure, they could open up all the code code, but what could anyone do with it, since it was created to run on a scale and in a way that literally couldn’t be reproduced anywhere else?
Return to AI. It is difficult to trust AI if we do not understand science within the machine. We need to find ways to open up this infrastructure. Loukides has an idea, though it may not satisfy the most enthusiastic people in free software / AI: “The answer is to provide free access to external researchers and early users so they can ask their own questions and see the wide range of results.” “. No, not giving them key card access to Facebook, Google or OpenAI data centers, but through public APIs. It’s an interesting idea that might work.
But it’s not “open source” the way many want it to be. This is probably okay.
Think differently about openness
In 2006, I was glad to get angry at the mega open source machines (Google and Yahoo!) for not being more open, but that accusation was and is almost pointless. Since 2006, for example, Google has packaged and open source a key infrastructure to meet its strategic needs. I’ve called things like TensorFlow and Kubernetes the open supply of entry ramps (TensorFlow) or exit ramps (Kubernetes), either open industry standards for machine learning that hopefully , lead to more Google Cloud workloads, or ensure cloud-based portability to give Google Cloud more opportunities to earn workloads. It’s smart business, but it’s not open source in some sense from Pollyanna.
Google is not alone in this either. It is better in open source than most companies. Because open source is inherently selfish, companies and individuals will always open up code that benefits them or their own customers. It has always been that way, and always will be.
In Loukides’ view of ways to open AI significantly despite the delta between the three AI giants and all the others, he does not advocate open source in the way we have traditionally done with the definition of open source. . Because? Because, fantastic as it is (and it really is), it has never been able to answer the open source dilemma in the cloud, both for creators and software consumers, that DiBona and Zawodny presented at OSCON in 2006. more than a decade, and we are nowhere near an answer.
Except we are.
I’ve argued that we need a new way of thinking about open source licenses, and my thoughts may not be too different from how Loukides reasons about AI. The key, as I understand his argument, is to provide enough access for researchers to replicate the successes and failures of how a particular AI model works. They do not need full access to all the code and infrastructure to run these models because, according to him, doing so is essentially useless. In a world where a developer could run an open source program on a laptop and do derivative works, it made sense to require full access to that code. Given the scale and unique complexities of the code running today on Google or Microsoft, this no longer makes sense, if it ever did. Anyway, not for all the code in the cloud running at scale.
We need to abandon our binary view of open source. It has never been a particularly useful lens through which to see the open source world, and every day is less so, given our cloud era. As businesses and individuals, our goal should be to open up access to software in a way that benefits our customers and third-party developers to foster access and understanding rather than trying to adapt an open source concept. in the cloud of decades. It didn’t work for open source, just as it doesn’t work for AI. It’s time to think differently.
Font