As concerns grow over deepening dependencies on a handful of Big Tech companies and the outsized influence of the U.S. and China, UN High Commissioner for Human Rights Volker Türk used his speech during July’s inaugural Global Dialogue on AI Governance to call on humans to “take back our agency” over AI. But what does agency over AI really mean? What policies and practices can allow people to shape AI in ways that respect, rather than undermine, human rights?
This blog post unpacks concrete initiatives that show how openness in AI development can increase people’s agency over AI and advance the development of public interest technology. At the same time, we reflect on how openness is not the same thing as mere access, and how short-term pragmatism can pave the way toward long-term independence.
Why is AI development so concentrated?
Last year, the UN formally established an Independent International Scientific Panel on AI, intended to “advance scientific understanding and ensure that international deliberations are informed by the best available evidence.” The panel’s preliminary report, published this past July, contained a stark warning about the concentration of AI capabilities in a small number of firms and countries, which it says “could enable authoritarian capture and undermine democratic accountability.” But how has this concentration of power come about?
One of the main reasons is that building a complete, multilayered “AI stack” is both an astronomical investment and a complex technical challenge. This technological stack comprises:
- The hardware layer, which provides the computing power that AI demands, mainly via advanced chips known as graphics processing units, or GPUs;
- The data layer, used for training and post-training;
- The model layer, which consists of the models themselves, such as GPT-4 or Claude, and the tools to service them;
- And finally, the application layer, which is the interface that users interact with.
Currently, the U.S. and China are the only two countries realistically capable of building a fully national version of the entire stack, where they build and control all necessary infrastructure within their borders. However, even then, they would continue to depend on other countries for some elements, whether that’s for sourcing the chips or the critical minerals used in the hardware layer. Meanwhile, most other countries’ reliance on U.S. or Chinese companies creates critical safety and accessibility vulnerabilities.
However, all is not lost for other countries and regions that want to build agency across the stack. There are several parts in the AI stack — notably the data, model, and application layers — where greater openness and geopolitical collaboration could help foster collective agency. In this way, we can work towards what Martin Tisné, founder and CEO of the Collaborative for AI in the Public Interest, calls a “global, open, resilient, non-aligned stack” adaptable to local needs.
How does openness support greater agency over AI?
When considering how to gain more agency over AI, it’s worth looking at the free and open source software (FOSS) movement; a longstanding and major driver pushing for increased agency over technology, from the software running the internet, to the tools millions of people rely on every day, such as the Firefox internet browser. Although definitions of FOSS vary from source to source, the essential principle means ensuring that anyone can access a given technology’s source code, study how it works, modify it, and share modified or unmodified copies with others.
The Open Source Initiative (OSI) has already applied this in the context of AI, with their open source AI definition (OSAID) according to which an open source AI system is one that enables four key freedoms:
- Use the system for any purpose and without having to ask for permission.
- Study how the system works and understand how its results were created.
- Modify the system for any purpose, including to change its output.
- Share the system for others to use with or without modifications, for any purpose.
None of the offerings from major AI tool providers, such as OpenAI or Anthropic, currently enable any of these freedoms. Until now, the overwhelming preference in AI development has been for closed systems, accessible only through online interfaces or application programming interfaces (APIs). This means that AI companies retain full control over which uses are permitted or not, can surveil how you use their tools, and forbid any inspection or modification of their AI models. Nevertheless, we are seeing a burgeoning ecosystem of open approaches to AI development.
How is openness being developed in the AI stack?
Looking at the model layer
There is a whole gradient of open alternatives for each element within the AI stack, starting with the model layer. Here, it is vital to distinguish between open weight and open source AI, since few models being released meet the OSAID criteria, even if they are open weight.
An open-weight model restricts access to some or all of the training data, while distributing its trained mathematical parameters so people can download, host, and run the model on their own local servers — which has considerable privacy and security benefits. Open-weight models can be modified through approaches such as “fine tuning,” which involves taking a generally capable model and further training it on task or domain-specific data. There is also an entire technique known as “abliteration” whereby people remove restrictions, censorship, and even safety guardrails from open-weight models. Even if open-weight releases fall far short of the ideal for openness, they do create some measure of transparency, modifiability, and (re)usability.
A recent Mozilla Foundation report on the state of open source AI maps the strong and growing ecosystem of open source collaboration, noting that open-weight AI models’s capacity is rapidly expanding and proving useful for task- and community-specific systems. While “closed models still lead at the frontier, on reasoning, and multimodality, the frontier is not what most workloads need.”
Open-weight models still depend on other layers of the stack to function, most notably the hardware layer — and it remains both expensive and technically complex to implement such hardware entirely locally. This can result in continued reliance on a small number of cloud providers even after deployment. Hopefully, as computing costs continue to fall, it will become easier for smaller, task-specific open models to host locally.
Compared with open source, open weight may not be the ideal strategy for fully reclaiming agency over AI development, but it can serve as a short-term, pragmatic step in the right direction especially for Global Majority countries with fewer resources. For instance, Chile’s National AI Centre (CENIA) has developed LatamGPT, a large language model (LLM) using Meta’s Llama 70B open weight model as the foundation to overcome the constraints related to having access to less infrastructure, money, time, technical expertise, or high-quality data than the large AI players. This is Latin America’s first regional LLM, trained with local data to provide an alternative for future small, task-oriented models to be trained from it — and it is itself an open-weight model, since it does not release complete training data.
According to LatamGPT’s own data sharing policies, anyone can openly access 41% of its training datasets, while another 37% remains accessible purely for research purposes. The remaining 21% of the datasets remain restricted, in line with the wishes of specific Latin American communities who provided the data. For instance, most Indigenous communities involved in the project preferred not to share data related to their linguistic and cultural heritage. This “tiered” approach to either allowing or restricting access to training data clearly has benefits for respecting dataholders’ rights and decisions.
Looking at the data layer
Traditionally, openness in the data layer of the AI stack refers to conditions that allow anyone to access and reuse datasets. Open data policies have been successful worldwide, with myriad governments and cultural institutions uploading vast quantities of data for public consumption online. However, in an era where dominant AI players use automated bots to extract data en masse from open datasets without compensating anyone, this concept needs refreshing.
So how can communities reclaim agency and control over their data, counteract data extractivism, challenge the concentration of power, and still support the development of openness in AI? To responsibly manage their data, communities need access to technical and legal tools that allow them to decide who can access their data, what to ask for in exchange, or even to control the conditions in which access happens. Mozilla’s Data Collective, for instance, builds infrastructure focused on consent and governance, enabling communities to manage access to their datasets. This then allows them to set usage terms, retain ownership, and even charge for access on their own terms. Such infrastructure is a prerequisite for dataholders seeking to avoid their resources being captured by dominant players.
Other Global Majority communities are focused on addressing AI performance gaps and language failures. In India, the Kayra project offers access to multimodal linguistic data while using data governance mechanisms to fairly compensate workers for datasets they collect or train. Masakhane, a project from Kenya’s Masakhane Research Foundation, works to “strengthen and spur NLP [Natural Language Processing] research in African languages, for Africans, by Africans,” embracing the principle that “Africans should be able to decide what data represents our communities globally, retain ultimate ownership of that data, and know how it is used.”
How can short-term pragmatism lead to long-term independence?
Mainstream discourse on AI development loves to frame it as a purely geopolitical battle between big players like the U.S. and China. But as the examples we describe above show, a different future is possible; one populated by a far more diverse and distributed set of AI solutions.
Governments and civil society seeking to increase AI agency must move beyond merely adopting and using AI tools as they are. Instead, they should actively explore, support, and build openness in AI. This means fostering multistakeholder engagement and cross-sector collaboration, creating community-focused data governance policies, and ensuring equitable access to models. In the short term, this will produce pragmatic AI solutions that people want and that they can trust.
But openness in AI is not sufficient by itself; developing open models or responsible data governance policies will not redistribute computational resources or prevent corporate capture of open standards. Openness requires concrete policy and investment to ensure that affected communities maintain decision-making power. In the long term, openness paired with policies and infrastructure intended to circumvent dependencies can spur the kind of innovation that loosens Big Tech’s grip on the AI ecosystem. In this way, it can help realize a vision for contextually rich, meaningful, and rights-respecting AI development that truly works in the public interest.