# Is Open Source the Future of AI? A Data-Driven Approach
## 4 Results
The upper panel shows the distribution of models across parameter sizes (0–130 billion), with bars color-coded by architecture. The lower panel displays the maximum average benchmark scores for each parameter size, color-coded by the creation date. Both panels share the same x-axis (parameter size) to facilitate comparison. This figure illustrates the relationship between model size, architecture, performance, and development timeline. This is encouraging for the open-source community, as high-performance models are becoming increasingly accessible to people without large amounts of computing resources. The most challenging task was measuring Hugging Face's contributions to advancing the AI field, particularly the impact of individuals focused on enhancing existing base models rather than creating new ones from scratch. Given that the base model data of fine tunes are available for most leaderboard entries, we aimed to evaluate how significantly these contributors improved the most popular models through fine-tuning and other optimisation techniques. This analysis highlights the crucial role that community contributions play in refining and advancing AI capabilities, even when starting from pre-existing foundational models. Merges were excluded from this analysis because it was not possible to attribute specific contributions to the enhancement of the model. Often,  
Figure 10: This figure illustrates the relationship between parameters and average score over time. The x-axis represents the month, and the y-axis represents the ratio of parameters to average score. Bars are annotated with specific model releases. This chart depicts the change in parameters/average score over time, highlighting key model releases. Figure 9: This figure displays the total number of downloads (in millions) for models across varying parameter sizes. The x-axis represents the parameter size, while the y-axis shows the total downloads in millions. This chart illustrates the distribution of download popularity relative to model size. merges did not include the base models or included only one. Additionally, some merged models were created from other merged models, with the same base model appearing in multiple merge steps, making it difficult to assess the improvement to specific base models. Based on Figure 11, the total percentage improvement of each base model was analysed. Notably, Llama1 (7b parameters) and Mistral 7b parameters show the highest percentage improvements, demonstrating significant community engagement and successful fine-tuning efforts. In contrast, models like Phi 3 show minimal improvements, suggesting they may already be highly optimised, less susceptible, or less appealing for further fine-tuning by the community. When compared with Figure 6, we can observe that the models with the highest improvements, such as Llama and Mistral, also have a substantial number of models derived from them. This correlation suggests that the community's focus on these models has led to a higher number of derivative models and to significant performance enhancements. This highlights the synergistic effect. where popular base models attract more contributors, leading to more refined and optimised versions. Interestingly, while Mistral and Llama3 have seen significant improvements and much experimentation, Llama1 shows the most improvement despite having a relatively lower number of models built upon it. This phenomenon could be attributed to several factors. Firstly, Llama1 might have been an early model that laid the groundwork for subsequent models, making it a foundational base model with much to improve on. Secondly, the community might have identified specific areas for improvement in Llama1 that were more straightforward to address, leading to substantial enhancements with fewer derivative models. Additionally, the model is older and at that time the community was still evolving, which reflects the amount of derivative work. However, in relative terms, it still represented a significant portion of all the models uploaded at that time.