Natural Selection for AI
Source: tai_blog

Now that AI is officially born and being raised  it is almost impossible to stop ourselves from having philosophical discussions about its meaning and its impact. We  humans  need to define our relationship with AI  we cannot ignore it. The path is still long for most of us. And yet  while I dump my words and thoughts here  theres a machine out there that does this and much more in a fraction of the time. Hopefully  also  in a fraction of its creative and moral value.   What gave AI birth?The answer to that question is not different from the evolution process of other entities. AI came to be what it is today after years of research  experimentation  and joining the forces of Statistics  Mathematics  Optimization  and computer power. It was first only one neuron making binary predictions. Then it became several neurons making predictions of several classes. Then it was a bunch of layers of neurons figuring out classes that they didnt even see before. And now  AI is many times more productive than a human brain  capable of telling humans what to do.   We created AI. We  humans  gave birth to this new way of intelligence. In our excitement for how fun and interesting it seemed to be to create a new type of intelligence  AI grew to be far more real than just fun.   But maybe we didnt just create the whole thing. Did we really create it  did we discover it  or did it evolve naturally? The answer might just not be so trivial or simple. Just as we dont know if we invented or created math  the process of obtaining AI can be just as well a complex mechanism combining elements of our creation and elements of our discovery.   Regardless  AI evolved. It grew from simple math elements to complex algorithms. Two elements are the fundamental pieces in the evolution process of artificial intelligence.   Lets recall for a moment the history of Statistics as one of the initial states of artificial intelligence. Linear regressions emerged some centuries ago  joining observations registered as data  a regression function  and an optimization problem to obtain the regression function. Very few data points and simple computational capacity were needed at the time to make linear regression become a staple mechanism for understanding a phenomenon. Not even a computer was necessary to obtain the regression function parameters given a set of data points. The origins of AI were very well handled with pencil and paper and a calculator at most.   Regardless of its simplicity  linear regression emerged from data  from a regression function  and from the possibility of calculating it (solution and computation of the optimization problem).   As of 2024  AI does not look at all as simple as linear regression  but their evolution process is comparable: data and computation of the optimization problem. While  evidently  they are not the only elements that played a role in the evolution of AI  it is to argue that they are the fundamental pieces selected for the development of AI. They are the elements that define the level of capacity of AI. Without them  AI would cease to exist just like living things would do without food. The concept of data might be easier to make sense of  but when it comes to computation of the optimization problem  things got very interesting during this evolution time.   DataFrom registered observations in pieces of paper  to Microsoft Excel  to databases  to the whole world wide web  data is nowadays an ocean containing the registry of experience. We started registering data to uncover patterns of different mechanisms through the different sciences. Whether in physics  biology  or psychology  we used registered data since the origins of early Statistics to understand connections among variables and causality patterns.   Thanks to these recorded observations  we have unveiled thousands of secrets of the atom and the universe. Stephen Hawking did not live to see the image of a black hole deducted from billions of data registries of the light and energy activity by an international network of radio telescopes called the Event Horizon Telescope (EHT). After so many years of his dedicated and thoughtful research about black holes  the first real image of one of these objects was probably a deserved experience for him. Thankfully  we did get to see such an object. But for what matters in our current conversation  without data  and so much of it  and its complexity of registration  the image of a real black hole would not have been possible. Once again  it has been from recorded observations that we have unveiled thousands of secrets  from the atom to the entire universe.   Data is also to AI what food is to humans. Its in-taken  processed  and finally used for something. With that said  theres one possible way of defining AI  and that is the capacity to digest billions of data to emit one decision in a small fraction of time. AIs life consists of making constant decisions: a prediction  creating statements  creating images  finding the hidden pattern  etc. If we compare the level of capacity of the human brain to emit one similar decision given billions of possibilities  we might be able to achieve it  but unfortunately the processing time would be just a bit longer than a fraction of a second or a minute. Regardless of our differences  we do have many things in common and one of them is our need for some input material. Data is to AI what food is to humans: it would cease to live without it.   AI is the capacity to digest billions of data to emit one decision in a small fraction of time   ChatGPT was the major democratized breakthrough of artificial intelligence. Before it  other AI solutions existed but were not in the hands of everyone. The average human being with access to a computer finally experienced the meaning of AI with the launch of the interface for textual processing of the GPT model launched in November 2022. What data was used to train this model? A very clear list of data domains is disclosed in the GPT-2 GitHub repo (See here). In a nutshell  the whole WWW was scrapped grabbing our actions  opinions  knowledge  reactions  and so much more.   Before we realize it  the data has become so diverse and big that AI will derive all the secrets of the physical world.   In the first versions of ChatGPT  when asking for recent facts or results that have emerged after the registered data used for its training  it very politely and robotically explains that those facts are not available at the time of its training. If ChatGPT is not fed with recent data  the claims it creates become outdated and likely invalid in time. This is how data acts as the food source of this type of AI.   But as we said  data is not just the energy source of AI  it is selection for AI. As more data becomes available  in time data also becomes more diverse than it was before. The processes of the universe are transformed in time and this information is hidden in the data that we register of our phenomena. Uncovering those hidden patterns is what demarks the evolution of artificial intelligence entities. Today  ChatGPT can answer questions  explain facts  and extract summaries of long documents. Tomorrow it can receive a research hypothesis and deliver a full thesis proving or disproving the hypothesis  or a thesis of a reformulated hypothesis because the initial hypothesis the human formulated did not make much sense. Before we realize it  the data has become so diverse and big that AI will derive all the secrets of the physical world.   But  as far as data was concerned  it did not act alone in the evolution of AI.   SoftwareIf you are not part of the AI community in general  have you wondered how something like ChatGPT actually comes up with so much sensible  almost accurate  textual content? The confidence with which this machine can provide information to answer our requests is something a human needs to build with time following a long path of hard and deep work.   The second responsible element for the evolution of AI is the refinement of the software. I mentioned before that aside from data  there was something called the computation of the optimization problem. A model such as a Generative Pre-trained Transformer (GPT) is a mathematical mechanism that processes an input to create an output concept. In the case of the model behind ChatGPT  it receives a query as input (write an essay about topic x) and it processes this query deeply to create an entire textual output answering the request. The way this machine processes this query is something that needs to be trained first. Just like when certain living entities are born  they have brains and they need training to learn things. Training a computer so it learns how to process future queries is far from a trivial task.   Richard Stallman was the creator of the so-called free software. The slogan to define the essence of this type of software since its origin has been free as in freedom  not as in free beer. With the growth of personal computer technology in the 70s and 80s  a key business opportunity came about selling the software running the machines separately from its hardware. With that  one single physical machine would represent income from every piece of software that it contained. Running a Windows machine required buying a license for the operating system. After that  writing a formatted document would require a user to purchase another license for Microsoft Word. This business model was the same for other types of software to run other processes  like printing  making calculations  drawing  etc.   The license between the user and the software has always been a barrier. Whether it is a positive or a negative barrier is another topic. However  the existence of this barrier did not allow the user to make any adaptation of a piece of software for a new computational feature. This meant that innovation in the capacity of software was very limited and subject only to the availability of the software owner.   Stallman established the concept of free software as software that can be used  copied  modified  and re-distributed  without liability to the original developer. Free software did not mean gratis. It meant to have the freedom to transform it. Now we see where this is going.   Training a model for a complex AI task requires software features emerging from different disciplines. Complex mathematical formulations  numerical solutions  fast optimization algorithms  programming languages of fast compilation  and scripting environments  among many more. When joining the efforts of all these disciplines  the necessary software to train these complex models was not a linear evolution that could come from a single private corporation. It emerged from the invisible force of the transformation of free software. Who transformed it? Communities  experts  and enthusiasts who contributed to the contributions of others. No wonder why a few years ago Microsoft bought GitHub after decades of refusing the concept of free software.   GPT models have a foundation in the dominant and advanced Python libraries of deep learning TensorFlow and PyTorch. Both of these software solutions are open source and have been in evolution since their release between 2015 and 2016. The parent of the model running behind ChatGPT OpenAI  a pioneer in popularizing the use of AI technology  developed its first versions of the GPT models and image generator models using these established open-source frameworks which already gave a solid landscape. So to this point  it is still amusing to imagine where we would be right now with AI  had open-source software not existed.   At this point  it is worth having another thought bubble to acknowledge and differentiate the contribution of Richard Stallman. While I have been using the concepts free software and open source interchangeably  they are by no means carrying the same fundamental meaning. The concept of free software  as originally defined in the General Public License (GNU GPL) series  had the spirit of freedom for the use  copy  modification  and redistribution of software guaranteeing its longevity as free software. This means that free software under GPL licenses shall remain free upon modification or redistribution. This is what has been known as copyleft licenses.   So to this point  it is still amusing to imagine where we would be right now with AI  had open-source software not existed.   OpenAI originally used and intended to develop this generative AI technology with a free software approach. However  the licenses that regulate software such as TensorFlow and PyTorch are of permissive nature  which was the perfect combo for OpenAI to achieve their current potential and closing the software right after crossing the peak moment.   Under a proprietary software paradigm  training an AI machine like the ones we are welcoming now would have been impossible. The changes these models and software needed to support more complex tasks would have required waiting for the proprietaries to release more versions. Under a free software paradigm  big changes in software capacity may become available in a few days. Nowadays  the dominant software that supports deep learning is open-source software. Just as in the case of data  the life of AI depends on and evolves with the evolution and availability of free or open-source software.   Data and software only?We can ask now  how are data and free/open source software selecting for the evolution of AI more than other features that also play a crucial role in it? Naturally  these two features are not the only ones that AI needed to become what it is today. Powerful hardware is one of them. While fast algorithms and efficient programming languages are one necessary condition  they would play a null role in practice without the existence of powerful hardware. Graphical processing units  exponential increase of RAM  high-performance computing  etc.  are all necessary elements to develop and run these complex models. So  where is the difference?   Its all about the invisible forces. To develop powerful hardware  big funding and sufficient tangible material is needed. These resources are assets that big private corporations can buy. This is not the case for diverse data and powerful software. The diversity and complexity of data is a quality that money alone cannot buy. Data is a registry of human and natural experiences. The diversity of natural experience is created by all the invisible forces that act around us. The same is true for powerful software. The contributions of so many experts and enthusiasts make the software become invisibly more solid and advanced. Here again  this diversity and complexity is something that money alone cannot buy.   What will happen next with AI?Until now  we have been using artificial intelligence solutions in a rather predictive  static way. Now  those entities that we trained in the past are learning from their own mistakes because we reinforce their behavior based on the predictions they have made. Now those entities are coming up with ideas and solutions that were hidden from the human mind before. AI has evolved to a level that it constitutes a dynamic entity. While it still goes on with human guidance  it surpasses humans in its ability to generate knowledge that is hidden from us.   AI is incorporated into human daily life. It will continue coexisting with us and will start guiding our actions and interactions. The more hidden patterns of the universe are to us  the more power artificial intelligence will gain because we will feed more experience into a type of intelligence that has proved capable of unveiling what is far from obvious. The more hidden the patterns  the more AI has an opportunity to learn something else. The moment this opportunity meets diverse enough data and software  selection for new capabilities of AI will happen.   As these words are written  generative AI and other types of artificial intelligence continue to improve and grow their capabilities and find their way into our daily lives. Our previous generations had to compete with physical force natural to other species who have physical abilities that humans dont. The biggest uncertainty now comes about the question of whether our current and future generations will need to compete with AI systems that can think faster than us. Ideally  AI would be a tool for humans that increases our efficiency and accuracy. With the fast evolution of AI  we might be making an independent entity out of it that can take control easily away from humans. Yet there  it will do so for as long as diverse-enough data and software exist.   Great sources that inspire ideasMadhumita Murgia  Code DependentCode Dependent by Madhumita MurgiaFind out more about Code Dependent by Madhumita Murgiawww.panmacmillan.com   Richard Stallman  Free Software  Free Society: https://www.gnu.org/philosophy/fsfs/rms-essays.pdfhttps://www.deeplearning.ai/the-batch/issue-229/https://www.theredhandfiles.com/chat-gpt-what-do-you-think/Interested in following these discussions? Looking forward to your comments!