Blog

Beyond the Bricks/Knowledge/Libraries
Posted on

How AI could bring about the mass adoption of open access in mathematics – and beyond

by Glyn Moody

Numerous posts on Walled Culture have lamented the continuing failure to realise all the benefits from open access. Even when there is a move to open access, it is frequently to gold open access, which requires a payment from institutions when one of their academics publishes a paper with a journal. On the plus side, anyone can access that paper as soon as it is published; but on the downside, any money saved by libraries on subscriptions is needed to the cover the “article processing charges” (APCs) demanded by gold open access titles. The academic publishers have cleverly managed to subvert open access by embracing it on their own terms. They get to claim that they are supporting open access, as now required by many funding bodies, while ensuring that their high profit margins are maintained.

The chief reason this has happened is that academics feel they need to publish in journals with a high “impact factor” – the titles that are referenced the most in other academic papers – and these generally offer gold open access, if anything. Academics claim they must publish there because hiring and promotion decisions use an academic’s publishing record as a proxy for their scholarly worth. In general, only well-established academics, who are already widely-respected in their field and have tenure, or at least a secure position, are brave enough to resist this tendency. They can publish in titles that do not require APCs, which may not be as high profile as gold open access journals, but which are truer to the aspirations of the open access movement.

Repeated attempts to dismantle this system based on gold open access, which brings only some of the potential benefits of open access, have failed. However, in a completely unexpected turn of events, recent results in the field of artificial intelligence may finally provide the long-desired breakthrough, as least in the field of mathematics, and probably more widely.

A few weeks ago, OpenAI announced that an “internal OpenAI system” – that is, an advanced model that has not been released publicly – had made a significant advance in the field of fluid mechanics, which is “the branch of physics concerned with the mechanics of fluids (liquids, gases, and plasmas) and the forces on them”. The main mathematical framework for fluids is encapsulated in the Navier–Stokes equations. They are widely and successfully used in real-life applications, but there are still many open questions about them. OpenAI announced that its internal model had resolved the question “whether smooth three-dimensional fluid motion can break down”, one which had remained unanswered for some 90 years. The importance – and difficulty – of resolving that issue is reflected in the fact that it was one of the Millennium Prize Problems:

The Millennium Prize Problems are seven well-known complex mathematical problems selected by the Clay Mathematics Institute in 2000. Each of the problems has a prize of one million US dollars for the first correct solution, pledged by the Clay Mathematics Institute.

There is some controversy about who exactly should get credit for the new solution of the Navier-Stokes problem, but there is no doubt about its huge significance for mathematicians and mathematics. That flows in part from the forces OpenAI used to obtain its result. According to the company, some 10,000 AI agents worked together for about 88 hours, on a massive scale:

Across all attempted problems, the agents sent 4.9 million messages and used about 300 billion output tokens. In the process of resolving the Navier–Stokes problem, the agents sent 2.7 million messages and used approximately 130 billion output tokens.

This is not mathematics as we know it. Nor is the end result, which is a 166-page proof that even the world’s top mathematicians struggle to grasp fully. But we know that the result is in fact valid, since its logical correctness has been checked using what is called Lean formalisation. That an AI model was able to solve – apparently without much guidance – this long-standing problem suggests that many others will now follow. Indeed one of the top mathematicians playing a prominent role in the debate about the future shape of mathematics in the age of AI, Terence Tao, has written:

we will transition from an era of proof scarcity to an era of proof abundance. Most of our institutions — journals, priority conventions, hiring and promotion criteria, prizes, the very notion of a research program — were designed under the assumption of scarcity, and it should not surprise us if they behave poorly under abundance.

He also notes that advent of high-level AI mathematics will exacerbate many of today’s problems in the world of academic publishing. There will be a flood of complex, verified but barely comprehensible proofs that journals will struggle to process because of their volume and their sheer opacity. Another leading mathematician, Scott Aaronson, thinks that AI will take over the task of proving theorems almost completely:

in whatever years I have left, I don’t expect that I’ll ever again prove a theorem because I’m actually needed to prove it. If I do, it will only be for my or others’ enjoyment or edification.

Assuming there is indeed a shift to using AI to prove theorems, there will be little point in publishing them in high-profile journals in order to gain academic kudos. First, because if anything deserves credit it is the AI model, and secondly because anyone could use the same or a different AI model to obtain the result independently and then publish it elsewhere. This will lead to advanced AI proofs becoming commonplace and being routinely announced and posted online for anyone to access freely, since publishing the result in a journal would offer no extra value once the proof’s validity had been established.

To begin with, top mathematicians will probably offer their own explanations and ruminations on the implications of complex and inscrutable AI proofs. There might be unique value in those, and so people will publish those as a token of academic achievement, and hiring committees can use them to judge candidates. But this will be a temporary phase. Already AI models provide explanations of AI proofs; they are not very helpful currently, but considering the pace at which AI is progressing it is likely that in the near future they will become good enough to replace the human academics here too. Again, any sufficiently advanced AI model can be used for this explanatory role, making formal publication in journals pointless.

If both the solutions and in-depth explanations of them can be carried out by AI models, the key question becomes: what, then, will today’s highly-trained mathematicians do? This is naturally the subject of extremely lively debate in the mathematical community. Some of the thoughts are quite gloomy. The mathematician Max Weinreich wrote a paper entitled “The crisis of AI-generated mathematics”. Another mathematician, Daniel Litt, wrote of “The End of Mathematics”. But Litt has also written a more positive follow-up post entitled “A beginning for mathematics”, which offers a way forward for mathematicians in the AI age. He calls for the emphasis to shift to the human and social aspects of mathematics — the very things lacking from even the most impressive AI proofs:

The allocative aspects of our job (hiring, graduate admissions, etc.) are in dire need of reform if we want to retain human mathematical expertise. Broadly speaking I think we should focus on rewarding skill in the parts of our jobs that cannot be automated: the internal (e.g. understanding mathematics) and social-relational parts, and operationalizations that hew as closely to those aspects of the profession as possible. For example, talks and sustained mathematical discussion now demonstrate understanding much better than papers. Once AI systems improve at exposition and “digestion,” this will be even more the case.

What’s interesting here is that these are precisely the things that the open access community have been calling for – rewarding all aspects of the academic’s role, not just the extremely narrow one of publishing papers in a few leading journals.

It is still early days – OpenAI’s announcement of the solution to the Millennium Prize problem is only a few weeks back – but there is an emerging consensus among mathematicians that mathematics will never be the same again. AI proofs of many outstanding problems will become quicker and cheaper, which means that it would be wilful not to accept them. As the ability of AI models to explain and “digest” these complex proofs improve, that will necessarily encourage mathematicians to place more emphasis on the human side of their activities.

That, in turn, will mean that academic bodies responsible for hiring and promotion will need to de-emphasise – perhaps ignore completely – today’s obsession with publications in high-impact titles. And once that is gone, there will be no pressure for academics to publish in “prestigious” gold open access titles, since prestige will increasing come from other, broader activities – giving lectures, providing mentorship to younger colleagues etc. This will give them greater freedom to choose whatever kind of open access they want, including green and diamond open access, for whatever publications that still make sense in this new world of AI-assisted mathematics.

These changes are already taking place in the world of mathematics, which is the first to be impacted by the new capabilities of powerful AI models. It is not yet clear how quickly other fields will be affected. Academic fields closest to mathematics – physics, chemistry, engineering etc. – are likely to be the first to see similar dramatic breakthrough results from AI models. Moreover, if mathematics successfully shifts away from today’s emphasis on publications towards a broader appreciation of skills, it may be that other fields will recognise the benefits of doing so and start following suit – with or without pressure from AI.

Featured image by OpenAI.

Follow me @glynmoody on Mastodon and on Bluesky.

Beyond the Bricks/Media/Thoughts
Posted on

How copyright is being abused to censor Albania’s Flamingo Revolution

by Glyn Moody

For the last couple of years, the copyright world has been grappling with the latest generation of AI, as numerous posts on Walled Culture attest. That’s hardly surprising, since AI is currently delivering the greatest challenge to the foundations of copyright since the arrival of the computer, which was discussed at length in Walled Culture the book (free digital versions available). Although this is the main focus, the copyright industry continues to pursue some of its other traditional obsessions. For example, calls for ever-more stringent Net blocks are still being made. France is leading the way once again, reported here by TorrentFreak:

French lawmakers have officially adopted a new sports law that replaces manual domain checks with support for automated, real-time piracy blocking. The system allows for instant blockades during live sports broadcasts, with oversight checks from regulator ARCOM being applied afterward rather than before.

A new study commissioned by the European Parliament calls for something similar:

The study recommends a swift takedown obligation for streaming intermediaries, requiring these to take action within a certain time limit. As an example, it mentions 30 minutes, following the example of Italy’s Piracy Shield.

This takedown requirement should not only apply to Internet providers. Other intermediaries including third-party DNS resolvers such as Google and OpenDNS, VPN providers, and CDNs such as Cloudflare, should also take action.

The suggestion therefore is to adopt all the bad ideas that EU countries such as Italy, France and Spain have been trying out recently, with no sign that any of them actually help.

The copyright companies also continue to chip away at the surprisingly resilient Anna’s Archive. In January, the site lost its main .org domain; in April, Spotify secured secured a $322 million default judgment against the still unknown operators of Anna’s Archive. More recently, the site was hit with what it calls a “coordinated attack” that took it offline for a while, although it is not clear whether anyone in the copyright world was involved.

Against the background of those recurrent copyright topics, it is interesting to see a less familiar issue crop up recently. It concerns large-scale protests taking place in Albania. They were triggered by revelations that Jared Kushner, Donald Trump’s son-in-law, planned luxury tourism developments in two important Albanian natural habitats. One of them, the Vjosa–Narta wetland ecosystem, is the home of flamingos. This has led to the mass protests being given the name the Flamingo Revolution.

The protests against the proposed developments have been on a very large scale – some estimate 250,000 participants at times – and have rocked the Albanian government, which was caught completely off-guard by the widespread objections to the plans, and which has struggled to counter them and the widespread support they have received online. That makes the sudden mass suspension of Instagram accounts posting about anti-government protests in Albania because of alleged copyright infringement suspicious, to say the least. Martha Dimitratou, founder of Repro Uncensored, a nonprofit organisation that tracks digital censorship, spoke to Wired about what has been happening in Albania:

Dimitratou told WIRED they had verified around 100 cases of account restrictions connected to protests. “We continue to receive additional reports and, based on the volume and patterns we are seeing, estimate that the true number could be several hundred,” she says.

“We also identified that multiple posts on an account would receive copyright complaints, often five to 10 in a relatively short period, followed by escalating enforcement and, in some cases, the disabling of the entire account,” Dimitratou says. “Taken together, the alleged paid reporting, the repeated sequence of copyright complaints followed by account enforcement, the concentration of targets around the same movement, and the repeated targeting of replacement accounts strongly indicate coordinated activity,” Dimitratou added.

Arjan Koçi, who had multiple posts taken down, managed to contact the person behind those copyright complaints. Koçi asked why he was making these false claims against him:

To his surprise, Koçi received a response. “He asked for my contact and then texted me.” The person told Koçi that he had been paid $1,800 to report these accounts and would withdraw his complaints if Koçi paid him. “He said he would accept crypto as payment.”

This kind of coordinated copyright abuse is a well-known phenomenon:

“This tactic of mass copyright reporting, commonly called brigading, is an abuse of the systems that Meta has in place for people to report genuinely problematic, abusive, dangerous, or illegal content on their platforms,” said Kian Vesteinsson, deputy director for tech policy and digital rights research at Freedom House. “We’ve even seen this sort of brigading happen at scale in the context of a war or a conflict. Certainly, it’s a tactic that we saw carried out by the Myanmar military, for example, to go after people criticizing the government’s conduct,” he told WIRED.

As that indicates, this is not a new problem. Walled Culture the book discusses similar cases of copyright being abused for the purposes of censorship, notably as part of online “reputation management” services. These often use false claims of copyright infringement to remove material that is damaging or embarrassing to their clients. The application of similar censorship techniques on a large scale against politically inconvenient material in Albania shows how copyright continues to be perverted for purposes far from the purported ones of encouraging and rewarding creativity.

Featured image by Arbenllapashtica.

Follow me @glynmoody on Mastodon and on Bluesky.

Beyond the Bricks/Knowledge/Media
Posted on

US government argues that fair use applies to OpenAI’s training of LLMs in key copyright case

by Glyn Moody

Historically, legislation and court decisions have always lagged behind technological change. But the gap between the two is even starker in the world of generative AI. The large language model (LLM) that kicked things off, ChatGPT, was only released in November 2022; the pace of development since then has been dizzying. Meanwhile, nearly 200 lawsuits have been brought against AI companies, mostly claiming copyright infringement. Very few have been resolved so far, leaving this whole area in a kind of legal limbo.

One particularly important issue is whether it is fair use to use copyright material to train AI (at least in jurisdictions where fair use rights exist). The Electronic Frontier Foundation (EFF) has a long history of defending fair use, and of trying to thwart attempts to expand copyright protections – a theme that also runs through Walled Culture the book (free digital versions available). As a recent EFF post explains, companies are once again attempting to expand the reach of copyright’s legal protection because of developments in the AI world:

Copyright owners claim that unless courts abandon 300-year-old copyright principles—and give rightsholders the power to control non-infringing works created by others—an imagined flood of AI-generated works will devastate creative markets. Under this “market dilution” theory, building generative AI tools cannot be fair use because those tools might be [sic] encourage the proliferation of competing works.

As the EFF rightly notes, if the courts took this approach, it would have serious knock-on effects:

The “market dilution” theory would eviscerate not only the fair use doctrine, but also other limits on copyright that work specifically to prevent rightsholders from unfairly suppressing competition by claiming broad ownership over tropes, genres, styles, and so on. In other words, publishers would wield unchecked veto power over any expression that might conceivably compete with a work they own.

Support for the EFF’s view that fair use applies to the training of LLMs comes from a surprising quarter: the US Department of Justice, which has just issued what it calls a “Statement of Interest of the United States” in the case brought by The New York Times against ChatGPT’s creator, OpenAI, and Microsoft, alleging copyright infringement. The US government’s argument in support of OpenAI includes an invocation of national security. It claims that:

effective integration of AI will be critical for U.S. national security capabilities. Several years ago, the Government Accountability Office (GAO) warned that “[f]ailure to adopt and effectively integrate AI technology could hinder national security” because AI can be used for tasks such as “analyzing intelligence information,” “enhancing weapon systems (such as drones and robotic ships),” and “providing recommendations on the battlefield (such as where to target missile strikes).” Over time, it has become even more apparent that “[a]dvanced AI capabilities make our Nation stronger, but also introduce new national security considerations” requiring that “the best and most secure technology is deployed to rapidly confront any and all threats to our country.”

Copyright law is relevant here because:

Rules of law that make it significantly more difficult to develop a robust AI industry in the United States therefore threaten national security and give a competitive advantage to foreign adversaries who are not so encumbered.

Specifically, the US government is worried that the New York Times in its litigation seeks to narrow fair-use doctrine to exclude the training of LLMs. The Statement of Interest writes:

That result would be inconsistent with basic copyright law principles and severely hamper “the Progress of Science and useful Arts.” U.S. CONST. art. I, § 8, cl. 8. As discussed below, publications—including independent and start-up media, in addition to the New York Times itself—are leveraging LLMs for journalistic and other purposes. And beyond the subject matter of this litigation, LLMs are already helping researchers across fields achieve major breakthroughs. Constraining LLM development under a misunderstanding of fair use doctrine would thwart such creative and scientific progress while hindering American prosperity and economic mobility.

The document goes on to explain why requiring AI companies to license material in order to train their generative AI systems is not a good solution:

An erroneous fair use ruling would hamper competition in the market for LLMs, because only the largest technology companies might have the capital necessary to pay licensing fees. And such licensing fees would disproportionately benefit legacy media outlets due to the sheer volume of their written publications.

These are some of the things that are likely to be lost when there are licensing fees, according to the US government:

if not hindered by a strained understanding of copyright law, LLMs can and should help level the playing field between mainstream and independent publishers, for several reasons. Authors with limited resources can use LLMs to compete (by, for example, using an LLM to generate an image to accompany an article—which otherwise might require a photographer or license). And LLMs can direct users to dissenting sources that offer contrary information or perspectives. It is not in the public’s interest for the largest technology companies to have an oligopoly on LLM training due to licensing entry barriers that function primarily as large subsidies for old mainstream media companies.

Politicians in the European Union would do well to bear these issues in mind now that some are calling for collective licensing to be introduced there. Whether the court hearing The New York Times’ lawsuit against OpenAI will be influenced by the intervention of the government remains to be seen. But as time goes on, there is another reason why judges might look more favourably on fair use arguments and their equivalents in other jurisdictions. This emerges from an analysis by Andres Guadamuz on his TechnoLlama blog of another lawsuit involving alleged infringement of copyright by OpenAI. It concerns a ruling from the High Court of Delhi in India regarding the case of ANI Media v OpenAI:

Asian News International (ANI) is one of India’s largest news agencies, which supplies text and video syndication to broadcasters and publishers across South Asia. They sued OpenAI alleging copyright infringement during the training phase, as well as reproduction of its articles on the output phase. This decision is an interim order deciding ANI’s application for a temporary injunction against OpenAI, so while interesting on its own right, the trial continues.

The whole blog post is well-worth reading, but here’s the summary:

First, the jurisdiction question appears to be leaning in favour of allowing lawsuits to proceed where the effects of training are experienced. Second, this case proves once again that if you cannot produce a credible infringing output, the case will rest on the input phase, where judges seem less willing to damn an entire model for copies made during training. Third, I found the reliance on the private use exception to be an astute move in a jurisdiction that has no text and data mining exception [unlike the EU].

Guadamuz highlights an interesting aspect of the ruling:

I think we often underestimate how willing judges are to look at the actual usefulness of LLMs when ruling in these copyright cases. Judges are AI users too, and here we have the Honourable Amit Bansal, who appears to understand the technology well, and this evidently shows in the decision. I was really impressed with the clarity of the opinion, particularly in the memorisation discussion, something which I hope other judges will replicate when dealing with this question in the future.

As the use of generative AI and LLMs becomes widespread and even routine in today’s society, it seems possible that this will bring with it a certain presumption in their favour that will trump even the decades-long habit of treating copyright as something sacrosanct that must be preserved and protected at all costs.

Featured image by OpenAI.

Follow me @glynmoody on Mastodon and on Bluesky.

Audiovisual/Beyond the Bricks/Media
Posted on

Cinema as a community experience opens the door for low-cost films funded by fans

by Glyn Moody

Walled Culture the book (digital versions available free) is about the incompatibility of the traditional copyright system with the digital world, and its failure to reward creators fairly. That raises the question of what might replace it, and the book’s last chapter offers one solution in the form of true fans. These are people who love an artist’s work and are happy to support it directly. This allows those works to be made available for no cost, since they have already been funded by fans. That, in turn, means creations can be shared – no need for punitive copyright infringement laws here – thus spreading the word about the artist, and bringing in new true fans and additional financial support.

This system is already working well for books, music and graphical art. But a common criticism of the approach is that it could never work for films, which therefore require copyright protection for them to be made. In support of that claim, people often point to the extremely large budgets of many films, often running to hundreds of millions of dollars. Clearly, the argument goes, such sums could never be amassed through the donations of true fans.

One issue with that argument is the widespread practice of “Hollywood accounting”, which Wikipedia explains as:

the opaque or “creative” set of accounting methods used by the film, video, television and music industry to budget and record profits for creative projects. Expenditures can be inflated to reduce or eliminate the reported profit of the project, thereby reducing the amount which the corporation must pay in taxes and royalties or other profit-sharing agreements, as these are based on net profit.

In other words, those huge budgets may not reflect the real costs of making a film, but are often a scheme for depriving people such as actors and others of their promised share of the profits by minimising or even eliminating the latter. Moreover, a recent development certainly suggests that successful films do not requires such inflated budgets, discussed here on the BBC site :

Which film would you expect to be a summer hit – a family-friendly action-adventure which is based on tried-and-tested IP (intellectual property)? Or a creepy low-budget horror film dreamt up by a first-time director? Until very recently, most of us would have put our money on the first option.

But this summer, it’s looking as if low-budget horror will be triumphant, with two indie films, Backrooms and Obsession, vanquishing two megabudget extravaganzas, Masters of the Universe and Star Wars: The Mandalorian and Grogu.

The figures speak for themselves:

The difference between the appeal of tired IP and original movies is illustrated by this weekend’s US box office figures. Masters of the Universe, which derives from a 1980s Mattel toy range and cartoon about a loincloth-sporting, perma-tanned warrior prince, is a new release which made just $29m (£21.7m). The Mandalorian and Grogu, which came out in May, is faring poorly for a Star Wars film: this weekend it made $10m (£7.5m).

By contrast, Backrooms “took $81m (£61m) in the US on its opening weekend. It’s total in the US is now $135m (£101m).” Obsession did even better:

Not only did it have a healthy opening weekend, but it’s also the first film since ET the Extra Terrestrial in 1982 to have its takings go up rather than down in its second and third weekends in cinemas. Its total take is now $152m (£114m) in the US and $225m (£169m) globally.

Those figures are impressive, but what is really noteworthy is the gulf between the respective production costs of the two classes of film. According to the BBC article:

The budget of The Mandalorian and Grogu is reported to be $165m (£124m), and the budget of Masters of the Universe is reported to be $200m (£150m). Take into account the marketing costs, and those films have to make between $300m and $400m (or £225m and £300m) just to break even.

…

The budget for Backrooms, on the other hand: $10m (£7.5m). And Obsession? $1m (£750,000), tops. Suddenly, spending a fortune to make a fortune no longer seems like a winning formula.

The latter sum in particular is easily something that true fans’ funding could amass.

The dynamics behind the growing success of low-budget films – something the BBC article calls “a Hollywood earthquake” – are precisely the ones discussed in the closing chapter of Walled Culture the book. An article on the Guardian Web site puts it this way:

“Going to the movies has become a way to connect with friends and share the moviegoing experience in person. Somewhere you can turn your phone off for a few hours and just disconnect from the world,” says 22-year-old film TikToker Florence Rose.

It is about the sense of community, which lies at the heart of the true fans idea, one that extends beyond the cinema experience:

Apps such as Letterboxd – essentially the Goodreads of cinema – allow users to distil their reactions into sharp, ironic one-liners and scroll through the takes of other viewers. “Gen Z are yearning for community, and through apps like Letterboxd, which continues to see an increase in popularity, people can find that community of like-minded individuals and share their thoughts and feelings about films,” says Rose.

The rise of Letterboxd is indicative of the hunger for precisely this kind of community-based experience:

The platform, whose core audience is 18- to 24-year-olds, has grown from 1.7 million users in 2020 to 26 million in 2026, adding more than nine million since January 2025 alone. “When a movie is released in the cinema, online conversations are sparked, which drives more people to the cinema so they can feel like they can join in and be a part of those conversations,” Rose adds.

This move from the Hollywood-style blockbusters to low-budget films made by new directors – often emerging from the world of YouTube – and new production houses is great news for audiences eager for new experiences and a sense of community. It not only brings fresh talent to the art form, it also opens up the possibility of fan-funded films that manage the difficult trick of being thoughtful and popular.

Featured image by PxHere.

Follow me @glynmoody on Mastodon and on Bluesky.

Audiovisual/Beyond the Bricks/Knowledge/Media/Music
Posted on

Why the spread of powerful local AI models makes copyright enforcement even harder

by Glyn Moody

The central theme of Walled Culture the book (free digital versions available) is the growing incompatibility between traditional copyright law and the digital, online world. The culmination of that process was the EU Copyright Directive, passed in 2019. The Directive was supposed to be transposed into local laws by 2021, but a year after that deadline, many EU member states had failed to do so. Nor was that a case of mild slippage; a recent report from Communia provides an update on how the implementations are going. Here’s what it found for one of the most contentious areas of the Directive:

the introduction of the press publishers’ right under Article 15 has not been matched by consistent implementation of the mandatory limits intended to contain its scope. Despite the largely prescriptive nature of the Directive, a significant number of Member States have failed to fully implement these safeguards. In addition, some jurisdictions have chosen not to apply existing copyright exceptions to the new right, resulting in a broader scope of protection for press publishers than for other rightholders and further contributing to fragmentation across the internal market.

That fragmentation is deeply ironic, because one of the main justifications for a new copyright Directive was to bring consistency across the EU. As for the even more controversial upload filters, they have proved so difficult to implement that most governments have not even tried to lay down how they should be used:

Most Member States have limited themselves to restating the Directive’s requirement that lawful uploads must not be blocked, leaving the practical balancing of copyright enforcement and freedom of expression largely to platforms and courts. While a small number of jurisdictions have introduced stronger safeguards – such as ex ante protections against overblocking, transparency obligations, and mechanisms to address abusive claims – these remain the exception. As a result, the level of protection for lawful user expression continues to vary across Member States.

Again the much-vaunted consistency that the Directive would bring to EU copyright law is nowhere to be seen. If those failures underline that, as predicted, the EU Copyright Directive has turned out to be a bad law, badly implemented, arguably the arrival of generative AI has made many of its measures completely moot. As Walled Culture has reported, the idea that copyright is largely irrelevant in a world full of AI-generated material – something first suggested on this blog back in October 2022 – is now increasingly mainstream.

But things are still moving fast in the world of generative AI, with yet more profound implications for copyright. A recent post on the IPKat blog explores one of them: the rise of a powerful new generation of AI models that can be run on a personal computer – or even on a smartphone. Many of the latest models coming out of China are not just open source software, but open weight – that is, the models’ numerical values that get set when a model is trained are released, too, so that anyone can download, run, study, and modify them. As a good introduction to this new wave of Chinese AI innovation in Technology Review explains:

If these open-source AI models keep getting better, they will not just offer the cheapest options for people who want access to frontier AI capabilities; they will change where innovation happens and who sets the standards.

One big impact they are likely to have is on the enforcement of copyright, not least in the EU. The IPKat post notes:

The spectrum of engagement with [open source and open weight] models clearly challenges the copyright system. A system, in this Kat’s view, which, until recently, was oriented around individual acts of copying, with platforms cast as new points of interference to bridge technological enforcement gaps. However, [user-generated content] occurring on AI model marketplaces demands a shift in rhetoric and approach. Their ability to redistribute creative agency and control over the tools of cultural production should prompt us to reflect on how copyright law should respond to creativity that occurs through shared infrastructures.

The EU Copyright Directive’s core assumption that the main forums for sharing material would be a few, easily-controllable online giants like Google and Facebook, no longer holds. Instead, people are moving to world where millions of people are using the latest generation of open source AI tools collaboratively to generate creations. Those may or may not be based on existing copyright material, but there is no easy way to police that. As the IPKat post points out:

For users running these [new open source and open weight] models locally, they no longer need to pay per request, nor is their data shared with AI companies, and by extension, rightsholders through Article 53(c) of the AI Act.

The world of generative AI is so complex, and moving so quickly, that it is no wonder that even the relatively recent EU AI Act, which entered into force two years ago, is being left behind by the latest developments. And the EU Copyright Directive, which was drawn up nearly a decade ago, is the digital equivalent of the UK’s 1865 Red Flag Act, which governed “self-propelled vehicles”, and required “a man with a red flag was to walk at least 60 yd (55 m) ahead of each vehicle”.

Featured image by Peter Griffin.

Follow me @glynmoody on Mastodon and on Bluesky.

Audiovisual/Media
Posted on

Spain’s Net blocks have a flimsy legal basis, and lack both oversight and accountability

by Glyn Moody

A few weeks ago, Walled Culture wrote about Hadopi, France’s infamous copyright enforcement mechanism. The so-called “graduated response” – aka “three strikes and you are out” – has been around for over 15 years now, has cost French taxpayers a fortune, and has never achieved any of its aims. As the Walled Culture post suggested, the latest court ruling might finally put the benighted scheme out of its misery.

But even if it does, there is already another disproportionately heavy-handed attempt to enforce copyright up and running in the shape of Italy’s Piracy Shield. It works by blocking access to unauthorised material using court orders against Internet Service Providers (ISPs). That would clearly be problematic, even if it were implemented properly, and well run. It is neither, as academic research from the end of last year underlined. Since the massive extension of its powers a year ago, things have gone relatively quiet on the Piracy Shield front in terms of new developments, and there are no signs that the Italian government has taken the many criticisms to heart.

That’s depressing enough, but even more worrying is that alongside France’s Hadopi fiasco, and Italy’s troubling Piracy Shield, Spain too seems intent on letting the copyright industry attack the Internet’s basic plumbing and thereby disrupt other sites, as well as downgrading the experience of thousands of innocent users. It’s increasingly clear that what’s happening in Spain is now a serious a threat to the operation of the online world there, but one that is still little-known. That makes a post from the Disruptive Competition Project (DisCo), which comes from the Computer & Communications Industry Association (CCIA), particularly useful. Its title picks up on the similarities with Italy’s approach: “Repeating Failure: How Spanish Overblocking Ignores the Lessons of Italy’s Broken Piracy Shield”. Here’s what has been happening in Spain for over a year now:

Since early 2025, LaLiga (Spain’s top-tier football league) has been operating an aggressive and largely unchecked IP-address blocking regime in an attempt to tackle sports piracy. In 2024, LaLiga and several Spanish internet service providers (ISPs), some of whom have direct commercial interests in LaLiga broadcasting, sought a court order authorising the blocking of specific domain names.

They succeeded in this particular endeavour. However, LaLiga has thereafter continued to interpret this order as a green light to unilaterally select Internet Protocol (IP) addresses and domains to block, without ongoing court oversight or accountability.

It turns out that the legal basis of the current action is flimsy in the extreme:

The legal foundation of LaLiga’s anti-piracy enforcement model rests on a single, seven-page judgment issued by a Commercial Court in Barcelona on 18 December 2024. This ruling should have never been treated as setting a strong precedent, as it was not the result of a rigorous legal battle and didn’t seriously examine whether the blocking approach is lawful or proportionate under EU law.

As has happened so often in the world of copyright, the companies involved have taken a very narrow, and possibly flawed legal judgment and applied it as widely and broadly as possible, without asking further permission from the courts:

LaLiga now compiles and updates lists of IP addresses and domain names that it wants to see blocked. This process is completely opaque: these lists are not publicly disclosed, there is no independent technical or judicial validation before new addresses are added, and the court does not appear to continuously review any additions.

Once someone gets blocked, there is no redress mechanism to remove addresses from the list, even when they are no longer associated with infringing content, nor any appeal process for mistakenly blocked services.

It will come as no surprise to readers of this blog that this cavalier approach has already led to the overblocking of sites, just as has happened in Italy because of Piracy Shield’s poor implementation:

LaLiga’s approach has caused widespread disruption, preventing access to tens of thousands of legitimate websites in recent years – including critical services such as payment providers and a national health operator. This disproportionate cost is borne by innocent third parties and small businesses, reflecting a mistaken belief that the commercial interests of one dominant party should trump Spanish consumers’ rights to a functioning internet.

The second of those overblocking incidents is particularly serious, since it involves a healthcare provider, which potentially puts people’s lives at risk. According to an article on La Razón (via Google Translate):

During the weekend of December 13-14, access to the Madrid Health website was blocked. While the duration of the access interruption and the message displayed upon entering the website varied depending on the internet service provider blocking access at LaLiga’s request.

Despite the serious nature of this overblocking, which effectively places the enforcement of copyright above protecting people’s health, the Spanish government is trying to wash its hands of the problem:

The situation has become so severe that members of the Spanish Parliament have sought explanations from the government. While acknowledging that the blocks have significantly impacted legitimate websites, the Spanish Government maintains that this is a judicial matter falling under the jurisdiction of the courts.

However, the courts seem to be entirely on the side of LaLiga, since they didn’t even give Internet companies a chance to present their side of the story earlier this year.

In February 2026, the Commercial Court of Córdoba even granted [LaLiga] an ex-parte preliminary injunction against NordVPN and ProtonVPN. This means the court issued a binding order, based solely on LaLiga’s arguments, without notifying those virtual private networks (VPNs) or allowing them to present a defence beforehand. The VPNs only discovered the judgment through the media and are now obliged to implement the blocks enforced by LaLiga.

It’s true that the same court has just rejected LaLiga’s request for coercive fines, and accepted that there was a technical dispute over whether the blocking could be implemented. But the refusal to allow the VPN companies to present a defence remains a deeply troubling precedent, and runs contrary to basic principles of justice. Moreover, as TorrentFreak reports, this latest ruling may be only a temporary reprieve for the VPN providers:

The league confirms that the decision merely sets aside the coercive fines while the proceedings continue, stressing that it does not exempt NordVPN from implementing IP blocks where LaLiga can prove piracy is taking place.

It is the usual one-sided justice that is typical when copyright is involved. It seems that LaLiga is being given carte blanche to do whatever it likes here, and never mind the consequences for Internet companies and their users. As Hadopi fades, and Italy’s Piracy Shield carries on as before, the fear has to be that Spain’s unconstrained approach to copyright enforcement could end up being worse than both.

There is currently a rare opportunity to comment on the issue of EU copyright enforcement in the realm of sports and other live events. There is an open Call for Evidence from the European Commission, which:

aims to collect the information necessary to support the review of the 2019 Copyright in the Digital Single Market Directive, and to seek feedback on the challenges linked in particular to the exercise of copyright and related rights in the context of technological developments and potential ways to address them.

As well as that general review of the main EU Copyright Directive, discussed at length in Walled Culture the book (free digital versions available), and the issue of live streaming, the Commission is seeking people’s views on an important upcoming legislative proposal aimed at strengthening copyright (yet again) in the light of AI, which is planned for the first quarter of 2027. The Call for Evidence closes on 25 June, so you have a couple of weeks to hone your thoughts and submit them. Based on previous experience, we can probably expect the European Commission to ignore what anyone except the copyright industry thinks, but it’s worth a try.

Featured image by Petr Kratochvil.

Follow me @glynmoody on Mastodon and on Bluesky.

Knowledge/Libraries/Media/Music/Thoughts
Posted on

Why Google’s new AI-saturated search page will be a disaster

by Glyn Moody

Google didn’t invent full-text search of the Internet – that honour belongs to early pioneers such as WebCrawler, Lycos and AltaVista. But for the last 25 years or so, Google has been synonymous with online searching, providing the quickest and most effective way to find things online (although its results may be getting worse.) More recently, it has been adding to its search engine more features based on generative AI, first with its AI Overviews in 2024, and then a year later with its AI Mode in Search. Now it has announced the latest stage in that evolution with what it calls “A new era for AI Search”:

It’s more intuitive than ever, dynamically expanding to give you space to describe exactly what you need. Designed to anticipate your intent, it also helps you formulate your question with AI-powered suggestions that go beyond autocomplete. And you can search across modalities, using text, images, files, videos or Chrome tabs as inputs.

This new incarnation effectively turns search into a chatbot:

You can easily ask a follow-up question right from an AI Overview, and flow into a conversational back and forth with AI Mode. Your context stays with you, and as you explore more deeply, the links and supporting articles get even more relevant. This seamless experience is live today across desktop and mobile, worldwide.

As the the screenshot of the new interface above shows, the traditional search result links that are currently placed under the AI Overview have now been confined to a small panel on the right-hand side of the screen, which shows a cut-down version of today’s list. Users are encouraged to ask follow-up questions from the AI search chatbot, rather than exploring the links themselves.

What this is likely to mean in practice is that even fewer people will follow links to sites, something that was already happening last year; instead, they will engage with Google’s chatbot to gather information indirectly. This is terrible news for access to knowledge because it frames the Google AI search engine as the fount of all knowledge – one that will do all the hard work of finding information and combining it into an easily-digested answer that can be interrogated further. It can do that because it has already ingested billions of Web pages and other information sources as part of the Large Language Model (LLM) training process. But search engine users will no longer know what some of those sources are unless they painstakingly click on the links in the new panel.

Most people will not bother, because the AI-generated results will be good enough – or at least will appear to be good enough. Unless visitors to the site take the trouble to follow the links to the sources they won’t really know how reliable those results are. For example, it is possible that the sources are wrong, or misleading; moreover, Google’s LLM may itself introduce new errors and distortions. There is also the question of how Google will insert ads into this AI-generated information, and to what extent advertisers will be able to buy preferential treatment in results.

This new mediated approach is clearly terrible news for Wikipedia – an issue already discussed on Walled Culture earlier this year – and for creators. Google will use the information found in their works, but will not actively encourage people to visit the originals. For many people, summaries will be good enough, and they will never discover the greater riches of the sites and creations that Google’s LLM is based on. Worse still, the original creators such as Wikipedia may not even be mentioned in answers that involve aggregating information from a large number of sources.

Similarly, the new Google search is the publishing industry’s worst nightmare. Not only is Google drawing on material they have published, but it is pushing links to those sources into the background. It seem inevitable that the Web traffic to publishers will fall yet further, making already struggling business models based on advertising even more precarious. That will have knock-on consequences for the funding of many sites – particularly newspapers and magazines – and for the commissioning of work from journalists and other creative professionals. Users won’t even need to visit Google Search much in order to keep up-to-date with topics of interest thanks to Google Search’s new agentic capabilities that will do the work for them in advance:

With information agents, you can stay updated on whatever matters most to you. Your agent will intelligently look across everything on the web, like blogs, news sites and social posts, plus our freshest data, such as real-time info on finance, shopping and sports, to monitor for changes related to your specific question.

In this case, not only will people not visit sites, but the latter will be constantly bombarded by various AI bots seeking information on behalf of users – increasing site running costs, and making sites less usable by humans. Another key announcement from Google will lead to a further flood of agentic activities that will pose new challenges to businesses:

We’re also expanding agentic booking capabilities in Search to a wide range of new tasks, including local experiences and services. Just share your specific criteria — like finding a private karaoke room for six on a Friday night that serves food late — and Search brings together the latest pricing and availability with direct links to finish booking through the provider of your choice. And for select categories like home repair, beauty or pet care, you can ask Google to call businesses on your behalf.

What emerges from Google’s latest announcements is less of a search engine, and more of an immersive virtual environment that is designed to keep people engaging with Google’s services, asking them for information, advice and even delegating actions to them. There is no doubt that many users will find these new features attractive, not least because they can use “conversational voice features” in Gmail, Docs and elsewhere. These are the digital assistants that have been promised for many years, able to understand spoken commands, provide information verbally, and carry out complex operations on behalf of users without the need for any complex training. For many people, that will be a boon, and they will doubtless migrate from the traditional search page, which will still be the default – at least for now – to the latest AI-infused version.

But these impressive technical features come at a high price, even leaving aside issues such as the environmental impact of the huge server farms they require. With the latest incarnation of its search engine, Google is making the World Wide Web as we have known it for over 30 years invisible, and therefore increasingly irrelevant to most people, who will be happy to let Google become their universal user interface to everything. And yet Google still depends on the Internet to supply all the information it is analysing and repackaging. It risks killing the very thing that sustains it.

There’s another, more subtle issue. The new Google search features make finding information and carrying out actions very easy in many ways. Leaving aside the problem that this will require people to trust what is in effect a huge black box, where the internal workings cannot be examined, with all the loss of control this implies, there is another danger. People who use Google’s powerful new AI search services to offload many of their day-to-day actions may gradually lose the ability to understand the world and to act within it without that constant help. Such a dependence may be great for Google and its advertisers, but it surely cannot be a good thing for the future of society.

Featured image by Google.

Follow me @glynmoody on Mastodon and on Bluesky.

Beyond the Bricks/Media/Music
Posted on

Hadopi, perhaps the world’s worst copyright law, is moribund but not quite dead (yet)

by Glyn Moody

One of the best demonstrations that an obsession with protecting copyright’s intellectual monopoly drives politicians insane is the French law known as Hadopi, an acronym for ‘Haute Autorité pour la diffusion des oeuvres et la protection des droits sur internet’ (High Authority for the Dissemination of Works and the Protection of Rights on the Internet). The Hadopi mechanism has been trying – and failing – to police copyright’s intellectual monopoly in France for 15 years now, and it is one of the main villains in the Walled Culture book (free digital versions available).

Here’s how Hadopi’s “graduated response” approach worked when a revised version came into operation in 2010. Alleged infringers were warned twice; if another allegation was made within a year of the second warning, the subscriber’s Internet connection could be suspended. A fine of €1,500 could also be imposed. The first notices were sent out in September 2010; by December of that year, copyright companies were issuing between 25,000 and 50,000 infringement allegations per day. At the end of July 2013, Hadopi had issued 2 million first notices and 200,000 second notices. There were 710 investigations to ascertain whether those who had been accused three times should be referred to the prosecutors.

That gives an idea of the scale of the investigations into people’s everyday use of the Internet in France, and of the databases of personal data that were created. And yet the first and only disconnection order, issued in June 2013, turned out to be unenforceable, because the disconnection only applied to Web access – other services like email, private messaging, the telephone line or TV services had to be preserved somehow – and was later dropped.

By 2020, Hadopi had been in existence in various forms for a decade. Working from Hadopi’s annual report for that year, the French magazine Next INpact calculated that in total the agency had imposed €87,000 in fines. The cost of running Hadopi was picked up entirely by French taxpayers and came to €82 million. In other words, a system that had failed to discourage people downloading unauthorised copies of copyright material, had also cost nearly a thousand times more to run than it generated in fines.

As Walled Culture reported at the time, in 2023 the French digital rights organisation La Quadrature du Net brought a challenge to the Hadopi system, still running in theory, on the grounds that it was incompatible with the two EU laws defining Europe’s data protection regime, the General Data Protection Regulation and the ePrivacy Directive. Shockingly, in 2024 the Court of Justice of the European Union (CJEU), the EU’s top court, ruled that “the general and indiscriminate retention of [Internet Protocol] addresses does not necessarily constitute a serious interference with fundamental rights”. La Quadrature du Net did not give up. Alongside the case at the CJEU, it was also taking legal action in France:

In 2019, we asked the Conseil d’État to overturn Hadopi’s central decree, which authorises the storage of personal data needed for the graduated response system (IP addresses, civil identity and downloaded material). The case was referred to the Constitutional Council and in 2020 we had our first partial victory: the Constitutional Council restricted Hadopi’s broad access to personal data (the law at the time provided that it could access “all documents”). However, despite to our initial assessment, this did not necessarily mark the end of the Hadopi.

The defeat handed down by the CJEU in 2024 offered a glimmer of hope:

The outcome was disappointing, as we lost on the principle: the CJEU agreed to weaken its case law. It accepted that access to metadata might, in certain cases, not be subjected to prior independent review. However, it required numerous conditions to this possibility, relating to both the retention of such data and the requirements for prior independent review.

Those two issues – retention of metadata and the requirement for prior independent review – have now been acknowledged as problematic by the Conseil d’État in a new ruling:

the Conseil d’État finally agreed with us on these two points. Firstly, it found that the retention of metadata is not carried out in a manner that safeguards civil liberties. The CJEU required “watertight separation” of IP addresses and civil identity data (which can be understood as two distinct databases, or files, that can only be technically correlated after a formal request for access by Arcom). The Conseil d’État notes that “no legal provision imposes such retention, under these conditions, on electronic communications operators”.

Secondly, it also notes that access to this data is not subject to independent review. It fully endorses the conclusions already made by the CJEU, that Arcom [the body that took over Hadopi’s role] cannot be both judge and jury: it cannot request access and then review the legality of that access itself, even though it is an independent authority. However, like the CJEU, the Conseil d’État considers that this lack of review is only an issue from the third access to the data onwards, the stage at which a registered letter is sent.

As La Quadrature du Net notes, in practical terms, this latest ruling means that Hadopi is “stalled”:

The Arcom can no longer take you to court, as the requirements set by the CJEU are not satisfied. And it can only send you an email if it has first ensured that your internet service provider has stored your metadata with a “watertight separation”. It has now been downgraded to the function of a giant spam machine.

Hadopi is not quite dead yet: the French government could try to solve the two problems pointed out by the CJEU and confirmed by the Conseil d’État, by setting up yet more independent bodies to handle these specific aspects of Hadopi. That would involve throwing even more taxpayers’ money at an approach that has not only failed completely, but which is fundamentally misguided. Clearly, trying to keep the moribund Hadopi alive in this way would be an irrational and wasteful thing for the French government to contemplate; but given this is the world of copyright, it might well try to do it anyway.

Featured image by RawPixel.

Follow me @glynmoody on Mastodon and on Bluesky.

Audiovisual/Music
Posted on

Welcome to the future, where AI agents hunt down alleged online copyright infringement

by Glyn Moody

As readers of this blog have doubtless noticed, the latest hot tech – and investment – area involves “agentic AI”, where AI systems are allowed to operative autonomously on allocated tasks. There’s no doubt there are some exciting possibilities here, as well as some troubling issues concerning lack of control. It’s a rapidly-evolving area of research and experimentation, which makes it hard to discern what is happening, and what is likely to arrive in the near future. One handy way to obtain at least some idea of the possibilities is to look at patent filings. These are often speculative, floating ideas that may or may not turn into real products and services. But they do at least hint at some of the things that we may see.

For example, an interesting post on Music Business Worldwide has spotted some filings by Music IP Holdings, a unit formed last year by Universal Music Group. They concern the application of AI to various aspects of music licensing and copyright enforcement. The main patent is for a “Copyright Licensing Chatbot” :

designed to interrogate potential [music] licensees about commercial vs. non-commercial use, timeframe, and geographical scope before either clearing or escalating the request.

The filings state that the chatbot “questions the potential licensee on a number of usage attributes to identify both the appropriate licensing models and to confirm that the proposed usage of the content is aligned with the artist’s principles and overall requirements that need to be met before their material shall be licensed and used in a derivative work.”

Another idea revealed by the filings is an automated system for calculating what the licensing fees should be:

the filings describe a dynamic pricing engine that adjusts licensing costs in real time “based on demand, seasonality and market conditions” using reinforcement learning techniques to continuously optimize pricing based on “conversion rates, customer lifetime value, and market penetration goals”.

As Music Business Worldwide (MBW) notes:

The filings suggest this would be paired with personalized subscription plans, bundle offerings, loyalty rewards, and tiered pricing for AI music licensees – a model that, to MBW’s eye, at least, looks a lot less like traditional music licensing and a lot more like the demand-based dynamic pricing already familiar from the live concert ticketing business, applied to copyright permissions.

More flexible music subscription plans could be a good thing for licensees and general users, since they would allow a far wider range of options that better reflect individual circumstances. However, another proposed application of AI in the copyright world that appears in the patent filings seems a terrible idea. It involves what is called an “LLM [Large Language Model] Agent Copyright Crawler”

The filings describe the crawler as sampling content streams from the open web, detecting digital watermarks embedded in images and audio using machine learning, and cross-referencing that material against IP licenses “currently in force”.

…


The filings state that the “LLM Agent” can connect directly with a source of materials and “propose licensing terms”; it can thank users for having “properly licensed usage”; and – critically – it can send “one or more cease and desist letters to user or streamer”.

In other words, the vision here is of a potentially unlimited number of AI agents hunting down alleged copyright infringement across the Internet, and automatically sending threatening letters to those supposedly involved. As Walled Culture the book (digital versions available free of charge) noted, the move from manual to automated takedown requests has already led to a huge increase in the volume of such demands – Google receives many billions of them every year. Allowing more capable and complex AI agents to carry out searches and send takedown notices will exacerbate the situation further.

For example, a move from the fairly crude identification systems employed today to ones involving AI could lead to far looser matching – so as to cast the copyright infringement detection net more widely – with inevitably more false positives. Perhaps even more troubling, autonomous AI agents could draw on the power of large language models to hunt out personal information of the people who are allegedly involved. This might include their phone numbers, email addresses and social media accounts, which would allow cease and desist notices – or less formal warnings – to be sent to them directly and repeatedly. More unscrupulous services might even seek associated information about alleged infringers – where they work, for example – and threaten to send information about their activities to their employers.

This is not the first time that Walled Culture has discussed the potential for AI agents to be used to hunt down alleged copyright infringement: a slightly different application of the technology was discussed back in February last year. These scenarios might seem far-fetched, but the pace of development in the world of AI in general, and agentic AI in particular, means that nothing can be ruled out at this stage. Moreover, there is little hope that the legal system can keep up with the constantly-evolving AI landscape, so legal remedies are likely to be few and ineffectual. The specific patent filings spotted by Music Business Worldwide may never be turned into real-world products and services. But given the financial incentives to do so, we can be sure that many others are working on similar ideas, and that some of them will indeed be realised. And when they do, they are almost certain to make the world of copyright even worse than it is today.

Featured image from Raw Pixel.

Follow me @glynmoody on Mastodon and on Bluesky.

Knowledge/Libraries
Posted on

The world’s leading cancer charity to stop funding open access publishing because of hybrid journals

by Glyn Moody

As numerous posts on this blog have emphasised, the underlying idea of open access (OA) – allowing anyone to read and share published academic research for free – is great in principle, but in practice has failed in important ways. That’s because traditional academic publishers have subverted the open access model to such an extent that the costs for research institutions of publishing in OA journals have barely changed at all. And yet one of the other key aims of open access was to save money while widening availability. Against that background, a natural question to ask is: if open access has failed to deliver savings, why bother supporting it? Cancer Research UK, the world’s leading cancer charity, has evidently asked itself that question and come up with an answer, which it explains in a post entitled “Why we won’t be funding open access publishing any more”:

We need efficient scholarly communications to spread scientific ideas via a fair economic model. We currently don’t have that. The open access movement was bold and promising, but ultimately disappointing. Now is the time to stop and call for a new way to make publishing work…

…

Ceasing to fund open access in the way we currently do will save us £5.2m of donors’ money over the next three years. That’s a substantial amount which can be put towards cancer research.

The post by Dan Burkwood, Director of Research Operations and Communications at Cancer Research UK, explains what exactly the problem is:

We currently fund open access publishing for our researchers in a number of ways. Despite hopes that this would enable a flourishing of open access dissemination of science, most of the growth has occurred in hybrid journals. These are publications that combine OA articles with those behind a paywall – this means the publishers will still charge for university and institute libraries to access them, even though researchers have paid for their work to be published. For us, this means we currently use donated money to fund our researchers, institutes and centres to publish OA research articles, yet they still have to pay to access the majority of journals in which those articles appear. The publishers are – so to speak – having their cake whilst also eating it.

These so-called “hybrid models” are discussed at length in Chapter 3 of Walled Culture the book (free digital versions available). They were presented as a transitional approach towards journals that were fully open access, but in many cases that transition hasn’t happened, not least because the hybrid model is so profitable for publishers, who therefore have little incentive to move to fully open access titles. Burkwood rightly points to a key reason why academic publishers continue to wield such power: the academic world’s insistence on using published articles in prestigious titles as a metric of success.

Cancer Research UK are working to widen the way we evaluate research in order to mitigate the heavy focus on publication outputs. It’s clear to us that a broader view of an applicant’s career is vital to gauge potential success. By signing up to DORA (San Francisco Declaration on Research Assessment), we encourage our reviewers to assess the quality and impact of research through means other than just journal impact factor. Additionally, we invite applicants to submit a narrative CV, allowing a more holistic view of their track record, research outputs and career progression.

But as he acknowledges, “Despite our, and others, attempts to limit the emphasis of the ‘publish-or-perish’ mindset, it will take time for the culture to change.” In the meantime, he suggests:

If researchers have no access to publishing funds they can publish their work for open access at no cost, but the publication will sit behind a paywall for 6 months (under embargo) before being deposited on Europe PMC open access – this is known as green open access.

Green open access provides full and free access to papers, but only after an embargo period, typically six months, but sometimes longer (gold open access provides instant access, but requires payment by researchers’ institutions.) That makes green OA a poor subsitute for real, immediate open access.

The problem here is that such embargo periods have long been accepted as the norm, but that is only because a terrible blunder was made over two decades ago by the Research Councils UK (RCUK). In 2005, the RCUK stipulated that the work it funded would require open access publication. However, when the final version of the RCUK’s policy appeared in June 2006, it had a significant flaw, expressed in the following provision: ‘Full implementation of these requirements must be undertaken such that current copyright and licensing policies, for example embargo periods or provisions limiting the use of deposited content to non-commercial purposes, are respected by authors.’ As the leading open access scholar Peter Suber wrote at the time, this was a completely unnecessary concession:

Researchers sign funding contracts with the research councils long before they sign copyright transfer agreements with publishers. Funders have a right to dictate terms, such as mandated open access, precisely because they are upstream from publishers. If one condition of the funding contract is that the grantee will deposit the peer-reviewed version of any resulting publication in an open-access repository [immediately], then publishers have no right to intervene.

At the root of the issue of embargoes lies copyright. If researchers retained full control of the copyright of their articles, rather than assigning it to publishers, they could prevent any embargoes being applied to them.

Cancer Research UK’s decision is regrettable but understandable. The fear has to be that others will follow suit. While the hybrid model is not universal, it is widespread enough to undermine the open access idea. Until researchers refuse to publish in such hybrid titles, publishers will continue to profit from them. Given the unnecessary embargoes imposed on articles released under green open access, that leaves alternatives such as diamond open access, where there are no charges for anyone, an approach that has long been espoused on this blog.

Featured image by Cancer Research UK.

Follow me @glynmoody on Mastodon and on Bluesky.

Beyond the Bricks/Knowledge/Thoughts
Posted on

Why genAI means the end of copyright for software and the re-invention of open source

by Glyn Moody

Most of the discussions about the impact of the latest generative AI systems on copyright have centred on text, images and video. That’s no surprise, since writers, artists and film-makers feel very strongly about their creations, and members of the public can relate easily to the issues that AI raises for this kind of creativity. But there’s another creative domain that has been massively affected by genAI: software engineering. More and more professional coders are using generative AI to write major elements of their projects for them. Some top engineers even claim that they have stopped coding completely, and now act more as a manager for the AI generation of code, because the available tools are now so powerful. This applies in the world of open source software too. But a recent incident shows that it raises some interesting copyright issues there that are likely to affect the entire software world.

It concerns a project called chardet, “a universal character encoding detector for Python. It analyzes byte strings and returns the detected encoding, confidence score, and language.” A long and detailed post on Ars Technica explains what has happened recently:

The [chardet] repository was originally written by coder Mark Pilgrim in 2006 and released under an LGPL license that placed strict limits on how it could be reused and redistributed.

Dan Blanchard took over maintenance of the repository in 2012 but waded into some controversy with the release of version 7.0 of chardet last week. Blanchard described that overhaul as “a ground-up, MIT-licensed rewrite” of the entire library built with the help of Claude Code to be “much faster and more accurate” than what came before.

Licensing lies at the heart of open source. When Richard Stallman invented the concept of free software, he did so using a new kind of software licence, the GPL. This allows anyone to use and modify software released under the GPL, provided they release their own code under the same licence. As the above description makes clear, chardet was originally released under the LGPL – one of the GPL variants – but version 7.0 is licensed under the much more permissive MIT licence. According to Ars Technica:

Blanchard says he was able to accomplish this “AI clean room” process by first specifying an architecture in a design document and writing out some requirements to Claude Code. After that, Blanchard “started in an empty repository with no access to the old source tree and explicitly instructed Claude not to base anything on LGPL/GPL-licensed code.”

That is, generative AI would appear to allow open source licences like the GPL to be circumvented by rewriting the code without copying anything directly from the original. That’s possible because AI is now so good at coding that the results can be better than the original, as Blanchard proved with version 7.0 of chardet. And because it is new code, it can be released under any licence. In fact, it is quite possible that code produced by genAI is not covered by copyright at all, for the same reason that artistic output created solely by AI can’t be copyrighted. If the licence can be changed or simply cancelled in this way, then there is no way to force people to release their own variants only under the GPL, as Stallman intended. Similarly, the incentive for people to contribute their own improvements to the main version is diminished.

The ramifications extend even further. These kind of “AI clean room” implementations could be used to make new versions of any proprietary software. That’s been possible for decades – Stallman’s 1983 GNU project is itself a clean-room version of Unix – but generally requires many skilled coders working for long periods to achieve. The arrival of highly-capable genAI coding tools has brought down the cost by many orders of magnitude, which means it is relatively inexpensive and quick to produce new versions of any software.

In effect, generative AI coding systems make copyright irrelevant for software, both open source and proprietary. That’s because what is important about computer code is not the details of how it is written, but what it does. AI systems can be guided to create drop-in replacements for other software that are functionally identical, but with completely different code underneath.

Companies that license their proprietary software will probably still be able to do so by offering support packages plus the promise that they take legal responsibility for their code in a way that AI-generated alternatives don’t: businesses would pay for a promise of reliability plus the ability to sue someone when things go wrong. But for the open source world these are not relevant. As a result, the latest progress in AI coding seems a serious threat to the underlying development model that has worked well for the last 40 years, and which underpins most software in use today. But a wise post by Salvatore “antirez” Sanfilippo sees opportunities too:

AI can unlock a lot of good things in the field of open source software. Many passionate individuals write open source because they hate their day job, and want to make something they love, or they write open source because they want to be part of something bigger than economic interests. A lot of open source software is either written in the free time, or with severe constraints on the amount of people that are allocated for the project, or – even worse – with limiting conditions imposed by the companies paying for the developments. Now that code is every day less important than ideas, open source can be strongly accelerated by AI. The four hours allocated over the weekend will bring 10x the fruits, in the right hands (AI coding is not for everybody, as good coding and design is not for everybody).

Perhaps a new kind of open source will emerge – Open Source 2.0 – one in which people do not contribute their software patches to a project, as they do today, but instead send their prompts that produce better versions. People might start working directly on the prompts, collaborating on ways to fine tune them. It’s open source hacking but functioning at a level above the code itself.

One possibility is that such an approach could go some way to solving the so-called “Nebraska problem”: the fact that key parts of modern digital infrastructure are underpinned up by “a project some random person in Nebraska has been thanklessly maintaining since 2003”. That person may not receive many more thanks than they have in the past, but with AI assistants constantly checking, rewriting and improving the code, at least the selfless dedication to their project becomes a little less onerous, and thus a little less likely to lead to programmer burn out.

Featured image by Lynn Greyling.

Follow me @glynmoody on Mastodon and on Bluesky.

Media/Thoughts
Posted on

Copyright industry intensifies efforts to undermine core Internet plumbing: VPNs

by Glyn Moody

Last month Walled Culture wrote about an important case at the Court of Justice of the European Union, (CJEU), the EU’s top court, that could determine how VPNs can be used in that region. Clarification in this area is particularly important because VPNs are currently under attack in various ways. For example, last year, the Danish government published draft legislation that many believed would make it illegal to use a VPN to access geoblocked streaming content or bypass restrictions on illegal websites. In the wake of a firestorm of criticism, Denmark’s Minister of Culture assured people that VPNs would not be banned. However, even though references to VPNs were removed from the text, the provisions are so broadly drafted that VPNs may well be affected anyway. Companies too are taking aim at VPNs. Leading the charge are those in France, which have been targeting VPN providers for over a year now. As TorrentFreak reported last February:

Canal+ and the football league LFP have requested court orders to compel NordVPN, ExpressVPN, ProtonVPN, and others to block access to pirate sites and services. The move follows similar orders obtained last year against DNS resolvers.

The VPN Trust Initiative (VTI) responded with a press release opposing what it called a “Misguided Legal Effort to Extend Website Blocking to VPNs”. It warned:

Such blocking can have sweeping consequences that might put the security and privacy of French citizens at risk.

Targeting VPNs opens the door to a dangerous censorship precedent, risking overreach into broader areas of content.

Indeed: if VPN blocks become an option, there will inevitably be more calls to use them for a wider range of material. The VTI also noted that some of its members are considering whether to abandon the French market completely. That could mean people start using less reliable VPN providers, some of whom have dubious records when it comes to security and privacy. The incentive for VPNs to pull out of France is increasing. In August last year the Paris Judicial Court ordered top VPN service providers to block more sports streaming domains, and at the beginning of this year, yet more blocking orders were issued to VPNs operating in France. To its credit, one of the VPN providers affected, ProtonVPN, fought back. As reported here by TorrentFreak, the company tried multiple angles:

The VPN provider raised jurisdictional questions and also requested to see evidence that Canal+ owned all the rights at play. However, these concerns didn’t convince the court.

The same applies to Proton’s net neutrality defense, which argued that Article 333-10 of the French sports code, which is at the basis of all blocking orders, violates EU Open Internet Regulation. This defense was too vague, the court concluded, noting that Proton cited the regulation without specifying which provisions were actually breached.

ProtonVPN also argued that forcing a Swiss company to block sites for the French market is a restriction of cross-border trade in services, and that in any case, the blocking measures were “technically unrealizable, costly, and unnecessarily complex.” Despite this valiant defence, the court was unimpressed. At least ProtonVPN was allowed to contest the French court’s ruling. In a similar case in Spain, no such option was given. According to TorrentFreak:

The court orders were issued inaudita parte, which is Latin for “without hearing the other side.” Citing urgency, the Córdoba court did not give NordVPN and ProtonVPN the opportunity to contest the measures before they were granted.

Without a defense, the court reportedly concluded that both NordVPN and ProtonVPN actively advertise their ability to bypass geo-restrictions, citing match schedules in their marketing materials. The VPNs are therefore seen as active participants in the piracy chain rather than passive conduits, according to local media reports.

That’s pretty shocking, and shows once more how biased in favour of the copyright industry the law has become in some jurisdictions: other parties aren’t even allowed to present a defence. It’s a further reason why a definitive ruling from the CJEU on the right of people to use VPNs how they wish is so important.

Alongside these recent court cases, there is also another imminent attack on the use of VPNs, albeit in a slight different way. The UK government has announced wide-ranging plans that aim to “keep children safe online”. One of the ideas the government is proposing is “to age restrict or limit children’s VPN use where it undermines safety protections and changing the age of digital consent.” Although this is presented as a child protection measure, the effects will be much wider. The only way to bring in age restrictions for children is if all adult users of VPNs verify their own age. This inevitably leads to the creation of huge new online databases of personal information that are vulnerable to attack. As a side effect, the UK government’s misguided plans will also bolster the growing attempts by the copyright industry to demonise VPNs – a core element of the Internet’s plumbing – as unnecessary tools that are only used to break the law.

Featured image by pxfuel.

Follow me @glynmoody on Mastodon and on Bluesky.

Knowledge/Libraries
Posted on

Common Corpus, an open training set for AI, goes global – and so should support for it

by Glyn Moody

As many of the AI stories on Walled Culture attest, one of the most contentious areas in the latest stage of AI development concerns the sourcing of training data. To create high-quality large language models (LLMs) massive quantities of training data are required. In the current genAI stampede, many companies are simply scraping everything they can off the Internet. Quite how that will work out in legal terms is not yet clear. Although a few court cases involving the use of copyright material for training have been decided, many have not, and the detailed contours of the legal landscape remain uncertain.

However, there is an alternative to this “grab it all” approach. It involves using materials that are either in the public domain or released under a “permissive” licence that allows LLMs to be trained on them without any problems. There’s plenty of such material online, but its scattered nature puts it at a serious disadvantage compared to downloading everything without worrying about licensing issues. To address that, the Common Corpus was created and released just over a year ago by the French startup Pleias. A press release from the AI Alliance explains the key characteristics of the Common Corpus:

Truly Open: contains only data that is permissively licensed and provenance is documented

Multilingual: mostly representing English and French data, but contains at least 1[billion] tokens for over 30 languages

Diverse: consisting of scientific articles, government and legal documents, code, and cultural heritage data, including books and newspapers

Extensively Curated: spelling and formatting has been corrected from digitized texts, harmful and toxic content has been removed, and content with low educational content has also been removed.

There are five main categories of material: OpenGovernment, OpenCulture, OpenScience, OpenWeb, and OpenSource:

OpenGovernment contains Finance Commons, a dataset of financial documents from a range of governmental and regulatory bodies. Finance Commons is a multimodal dataset, including both text and PDF corpora. OpenGovernment also contains Legal Commons, a dataset of legal and administrative texts. OpenCulture contains cultural heritage data like books and newspapers. Many of these texts come from the 18th and 19th centuries, or even earlier.

OpenScience data primarily comes from publicly available academic and scientific publications, which are most often released as PDFs. OpenWeb contains datasets from YouTube Commons, a dataset of transcripts from public domain YouTube videos, and websites like Stack Exchange. Finally, OpenSource comprises code collected from GitHub repositories which were permissibly licensed.

The initial release contained over 2 trillion tokens – the usual way of measuring the volume of training material, where tokens can be whole words and parts of words. A significant recent update of the corpus has taken that to over 2.267 trillion tokens. Just as important as the greater size, is the wider reach: there are major additions of material from China, Japan, Korea, Brazil, India, Africa and South-East Asia. Specifically, the latest release contains data for eight languages with more than 10 billion tokens (English, French, German, Spanish, Italian, Polish, Greek, Latin) and 33 languages with more than 1 billion tokens. Because of the way the dataset has been selected and curated, it is possible to train LLMs on fully open data, which leads to auditable models. Moreover, as the original press release explains:

By providing clear provenance and using permissibly licensed data, Common Corpus exceeds the requirements of even the strictest regulations on AI training data, such as the EU AI Act. Pleias has also taken extensive steps to ensure GDPR compliance, by developing custom procedures to enable personally identifiable information (PII) removal for multilingual data. This makes Common Corpus an ideal foundation for secure, enterprise-grade models. Models trained on Common Corpus will be resilient to an increasingly regulated industry.

Another advantage for many users is that material with high “toxicity scores” has already been removed, thus ensuring that any LLMs trained on the Common Corpus will have fewer problems in this regard.

The Common Corpus is a great demonstration of the power of openness and permissive copyright licensing, and how they bring benefits that other approaches can’t match. For example: “Common Corpus makes it possible to train models compatible with the Open Source Initiative’s definition of open-source AI, which includes openness of use, meaning use is permitted for ‘any purpose and without having to ask for permission’. ” That fact, along with the multilingual nature of the Common Corpus, would make the latest version a great fit for any EU move to create “public AI” systems, something advocated on this blog a few months back. The French government is already backing the project, as are other organisations supporting openness:

The Corpus was built up with the support and concerted efforts of the AI Alliance, the French Ministry of Culture as part of the prefiguration of the service offering of the Alliance for Language technologies EDIC (ALT-EDIC).

This dataset was also made in partnership with Wikimedia Enterprise and Wikidata/Wikimedia Germany. We’re also thankful to our partner Libraries Without Borders for continuous assistance on extending low resource language support.

The corpus was stored and processed with the generous support of the AI Alliance, Jean Zay (Eviden, Idris), Tracto AI, Mozilla.

The unique advantages of the Common Corpus mean that more governments should be supporting it as an alternative to proprietary systems, which generally remain black boxes in terms of where their training data comes from. Publishers too would also be wise to fund it, since it offers a powerful resource explicitly designed to avoid some of the thorniest copyright issues plaguing the generative AI field today.

Featured image from PublicDomainPictures.net.

Follow me @glynmoody on Mastodon and on Bluesky.

Beyond the Bricks/Knowledge/Thoughts
Posted on

Copyright litigation over Anne Frank’s writings likely to impact the fate of VPNs in the EU

by Glyn Moody

“The Diary of a Young Girl” is a Dutch language diary written by the young Jewish writer Anne Frank while she was in hiding for two years with her family during the Nazi occupation of the Netherlands. Although the diary and Anne Frank’s death in the Bergen-Belsen concentration camp are well known, few are aware that the text has a complicated copyright history – one that could have important implications for the legal status and use of Virtual Private Networks (VPNs) in the EU. TorrentFreak explains the copyright background:

These copyrights are controlled by the Swiss-based Anne Frank Fonds, which was the sole heir of Anne’s father, Otto Frank. The Fonds states that many print versions of the diary remain protected for decades, and even the manuscripts are not freely available everywhere.

In the Netherlands, for example, certain sections of the manuscripts remain protected by copyright until 2037, even though they have entered the public domain in neighboring countries like Belgium.

A separate foundation, the Netherlands-based Anne Frank Stichting, wanted to publish a scholarly edition of Anne Frank’s writing, at least in those parts of the world where her diary was in the public domain:

To navigate these conflicting laws, the Dutch Anne Frank Stichting published a scholarly edition online using “state-of-the-art” geo-blocking to prevent Dutch residents from accessing the site. Visitors from the Netherlands and other countries where the work is protected are met with a clear message, informing them about these access restrictions.

However, the Anne Frank Fonds was unhappy with this approach, and took legal action. Its argument was that such geo-blocking could be circumvented with VPNs, and so its copyrights in the Netherlands could be infringed upon by those using VPNs. The lower courts in the Netherlands dismissed this argument, and the case is now before the Dutch Supreme Court. Beyond the specifics of the Anne Frank scholarly edition, there are important issues regarding the use of VPNs to get around geo-blocking. Because of the potential knock-on effect the ruling in this case will have on EU law, the Dutch Supreme Court has asked for guidance from the EU’s top court, the Court of Justice of the European Union (CJEU).

The CJEU has yet to rule on the issues raised. But one of the court’s advisors, Advocate General Rantos, has published a preliminary opinion, as is normal in such cases. Although that advice is not binding on the CJEU, it often provides some indication as to how the court may eventually decide. On the main issue of whether the ability of people to circumvent geo-blocking is a problem, Rantos writes:

the fact that users manage to circumvent a geo-blocking measure put in place to restrict access to a protected work does not, in itself, mean that the entity that put the geo-blocking in place communicates that work to the public in a territory where access to it is supposed to be blocked. Such an interpretation would make it impossible to manage copyright on the internet on a territorial basis and would mean that any communication to the public on the internet would be global.

Moreover:

As the [European] Commission pointed out in its written observations, the holder of an exclusive right in a work does not have the right to authorise or prohibit, on the basis of the right granted to it in one Member State, communication to the public in another Member State in which that right has ceased to have effect.

Or, more succinctly: “service providers in the public domain country cannot be subject to unreasonable requirements”. That’s a good, common-sense view. But perhaps just as important is the following comment by Rantos regarding the use of VPNs to circumvent geo-blocking:

as the Commission points out in its observations, VPN services are legally accessible technical services which users may, however, use for unlawful purposes. The mere fact that those or similar services may be used for such purposes is not sufficient to establish that the service providers themselves communicate the protected work to the public. It would be different if those service providers actively encouraged the unlawful use of their services.

That’s an important point at a time when VPNs are under attack from some governments because of concerns about possible copyright infringement by those using them.

The hope has to be that the CJEU will agree with its Advocate General’s sensible and fair analysis, and will rule accordingly. But there is another important aspect to this story. The basic issue is that the Anne Frank Stichting wants to make its scholarly edition of Anne Frank’s diary available as widely as possible. That seems a laudable aim, since it will increase understanding and appreciation of the young woman’s remarkable diary by publishing an academically rigorous version. And yet the Anne Frank Fonds has taken legal action to stop that move, on the grounds that it would represent an infringement of its intellectual monopoly in some parts of Frank’s work, in some parts of the world. The current dispute is another clear example of how copyright has become for some an end in itself, more important than the things that it is supposed to promote.

Featured images from Wikimedia: Anne Frank – Diary

Follow me @glynmoody on Mastodon and on Bluesky.

Beyond the Bricks/Knowledge/Libraries
Posted on

Wikipedia at 25 grapples with new challenges arising from generative AI

by Glyn Moody

Wikipedia celebrated its 25th birthday this month. Given the centrality of Wikipedia to so much activity online, it is hard to remember (or to imagine, for those who are younger) a time without Wikipedia. The latest statistics are impressive:

  • Wikipedia is viewed nearly 15 billion times every month.
  • Wikipedia contains over 65 million articles across more than 300 languages.
  • Wikipedia is edited by nearly 250,000 editors every month around the world. Editors are defined by one edit or more every month; only editors with a username are counted.
  • Wikipedia is accessed by over 1.5 billion unique devices every month.

That’s testimony to the global nature of Wikipedia. But there’s something else, not mentioned there, that is of great relevance to this blog: the fact that every one of those 65 million articles is made available under a generous licence – the Creative Commons Attribution-ShareAlike 4.0 licence, to be precise. That means sharing and re-use are encouraged, in contrast to most material online, where copyright is fiercely enforced. Wikipedia is living proof that giving away things by relying on volunteers and donations – the “true fans” approach – works, and on a massive scale. Anil Dash puts it well in a post celebrating Wikipedia’s 25th anniversary:

Whenever I worry about where the Internet is headed, I remember that this example of the collective generosity and goodness of people still exists. There are so many folks just working away, every day, to make something good and valuable for strangers out there, simply from the goodness of their hearts. They have no way of ever knowing who they’ve helped. But they believe in the simple power of doing a little bit of good using some of the most basic technologies of the internet. Twenty-five years later, all of the evidence has shown that they really have changed the world.

However, Wikipedia is today facing perhaps its greatest challenge, which comes from the new generation of AI services. They are problematic for Wikipedia in two main ways. The first, ironically, is because it is widely recognised that Wikipedia’s holdings represent some of the highest-quality training materials available. In a post explaining why, “in the AI era, Wikipedia has never been more valuable”, the Wikimedia Foundation writes:

AI cannot exist without the human effort that goes into building open and nonprofit information sources like Wikipedia. That’s why Wikipedia is one of the highest-quality datasets in the world for training AI, and when AI developers try to omit it, the resulting answers are significantly less accurate, less diverse, and less verifiable.

That recognition is welcome, but comes at a price. It means that every AI company as a matter of course wants to download the entire Wikipedia corpus to be used for training its models. That has led to irresponsible behaviour by some companies, when their scraping tools download pages from Wikipedia with no consideration for the resources they are using for free, or the collateral damage they are causing to other users in terms of slower responses.

Trying to stop companies drawing on this unique resource is futile; recognising this, Wikimedia Foundation has come up with an alternative approach: Wikimedia Enterprise, “a first-of-its-kind commercial product designed for companies that reuse and source Wikipedia and Wikimedia projects at a high volume”. In 2022, its first customers were Google and the Internet Archive, and this month, Wikimedia Enterprise announced that Amazon, Meta, Microsoft, Mistral AI, and Perplexity have also signed. That’s important for a couple of reasons. It means that many of the biggest AI players will download Wikipedia articles more efficiently. It also means that the Wikipedia project will receive funding for its work.

This new money is crucial if Wikipedia is to remain a high quality resource. And that is precisely why every generative AI company that uses Wikipedia posts for training should – if only out of self-interest – pay to do so. What is happening here echoes something this blog suggested back in May 2024: that AI companies should pay artists to create new works, and give away the results, because fresh training material is vital. Helping to pay for Wikipedia to create more high-quality articles that are freely available to all is a variation on that theme.

The other problem that generative AI causes Wikipedia is more subtle. The Wikimedia Foundation explains that alongside financial support, the project needs proper attribution:

Attribution means that generative AI gives credit to the human contributions that it uses to create its outputs. This maintains a virtuous cycle that continues those human contributions that create the training data that these new technologies rely on. For people to trust information shared on the internet, platforms should make it clear where the information is sourced from and elevate opportunities to visit and participate in those sources. With fewer visits to Wikipedia, fewer volunteers may grow and enrich the content, and fewer individual donors may support this work.

Without fresh volunteers, Wikipedia will wither and become less valuable. That’s terrible for the world, but it is also bad for generative AI companies. So, again, it makes sense for them to provide proper attribution in their outputs. That requirement has become even more pressing in the light of a new development. According to tests carried out by the Guardian:

The latest model of ChatGPT has begun to cite Elon Musk’s Grokipedia as a source on a wide range of queries, including on Iranian conglomerates and Holocaust deniers, raising concerns about misinformation on the platform.

That’s potentially problematic because of how Grokipedia creates its entries. Research last year found that:

Grokipedia articles are substantially longer and contain significantly fewer references per word. Moreover, Grokipedia’s content divides into two distinct groups: one that remains semantically and stylistically aligned with Wikipedia, and another that diverges sharply. Among the dissimilar articles, we observe a systematic rightward shift in the political bias of cited news sources, concentrated primarily in entries related to politics, history, and religion. These findings suggest that AI-generated encyclopedic content diverges from established editorial norms-favouring narrative expansion over citation-based verification.

If leading chatbots starts drawing on Grokipedia routinely for their answers, it is less likely that there are independent sources where the information can be checked, something generally possible with Wikipedia. It therefore becomes even more urgent for generative AI systems to provide attribution, so at least users know where information is coming from, and whether there are likely to be further resources that confirm a chatbot’s claims. Not everyone will want to do that, but it is important to offer it as an option.

Wikipedia at 25 is an amazing achievement in multiple ways, one of which includes serving as a demonstration that material can be given away for free, supported directly by users, and on a global scale. It would be a tragedy if the current enthusiasm for generative AI systems led to that resource being harmed and even destroyed. A world without Wikipedia would be a poorer world indeed.

Featured image by Wikipedia.

Follow me @glynmoody on Mastodon and on Bluesky.

Audiovisual/Beyond the Bricks/Knowledge/Libraries/Media/Music/Thoughts
Posted on

Walled Culture the book, three years on

by Glyn Moody

Walled Culture the book (free digital versions available) was launched just over three years ago. A few weeks afterwards, I talked with journalist and editor Maria Bustillos about the book and its background, as part of the Internet Archive’s Book Talk series. That interview has just been added to the Future Knowledge Podcast series in a shortened form, so this seems like a good moment to take stock of the key developments in the copyright world since the book’s launch.


Walled Culture’s first chapter looked at the impact on copyright of the transition from analogue to digital technologies, and the attempts by the copyright industry to lock down the new medium of the Internet. This section explored the enormous effort expended by companies in trying to stop people making copies of digital files, which meant privileging copyright above basic consumer and even human rights. Since the book was published, the work to strengthen and widen industry control has continued.

For example, in October 2022, there was a French proposal to impose a levy on the commercial use of works in the public domain – which undermines the very idea of what the public domain is. The following year, Germany proposed including copyright infringement as an example of “digital violence” – a reflection of the inflated importance that copyright is too often assigned as a result of industry lobbying. A few weeks later, the EU’s top court ruled that there is no right to online anonymity because copyright is more important. More recently, the EU has floated the idea of strengthening copyright yet further, by giving new rights to live streaming sites, to the detriment of the Internet and its users.

Chapter 2 of Walled Culture explored the consequences of the ever-longer copyright term and the attempt to apply analogue constraints in the digital realm. One is the issue of hostage works – works that are still in copyright, but which cannot be re-issued because it is not clear who owns the rights, and thus who could give permission for new editions. Even though there are laudable new efforts to tackle it, the problem is so huge that such small-scale heroism barely makes a dent in it.

The chapter also explored the many problems for libraries that arise from the shift to ebooks. It turns out that publishers have been even more greedy here than was evident when Walled Culture first appeared. In the UK, publishers lobbied against a 20% tax being applied to ebooks, arguing that removing it would allow people to enjoy lower prices. The UK government was persuaded, the tax was removed, but instead of passing on the savings to ebook purchasers, publishers kept prices largely the same, and pocketed the extra money themselves.

Another topic covered in the same chapter concerned the Marrakesh Treaty, which allows countries to bring in copyright exceptions to facilitate the creation of versions of works that can be accessed by the visually impaired. Even though such a treaty was first discussed 45 years ago, and finally agreed 13 years ago, it turns out that several countries are still undermining the goals and the intent of the treaty – an extraordinary betrayal of the blind, visually impaired, and dyslexic people around the world. The Marrakesh Treaty saga shows the copyright industry and its friends in governments around the world at their very worst, unashamedly placing copyright’s intellectual monopoly above other fundamental human rights.

Chapter 3 was a detailed exploration of open access – the idea that everyone, everywhere, should have free access to published knowledge, specifically to the research that is publicly funded. As Walled Culture the book explains, serious mistakes were made in pursuing this idea, with the result that the version of open access we have today has failed to bring with it all the predicted benefits. However, in the last few years, a glimmer of hope has emerged in the shape of diamond open access, where there are no charges for either the reader or the researcher. Walled Culture has reported on a number of the initiatives to support it.

The next two chapters looked at a very different, but highly important aspect of copyright: its enforcement. They describe more than a decade of attempts by the copyright industry to use harsh new laws to stop files being copied and shared online. Perhaps the most famous initiative is the French Hadopi, whose “three strikes” approach was meant to curb file sharing by throwing people off the Internet completely. It failed miserably, and at considerable cost to the public purse.

In the period since Walled Culture the book was published, a new state system for fighting alleged copyright infringements has risen to prominence, this time in Italy. Called Piracy Shield, it is administered by the country’s national telecoms regulator, Autorità per le Garanzie nelle Comunicazioni (Authority for Communications Guarantees, AGCOM). As this blog has tracked, AGCOM continues to extend Piracy Shield, even in the face of research that it is not working and that it brings with it serious “economic, technical and operational harms”.

In many ways, the heart of the Walled Culture book is Chapter 6, which describes the battle over the EU Copyright Directive, including some dramatic twists and turns in the narrative. As the chapter notes, the end result was deeply unsatisfactory, contradictory even. But it was passed, which meant that EU Member States were required to transpose it into local laws. The manner in which that was done is instructive.

For example, in Portugal, a new law implementing the EU Copyright Directive was not only passed without debate, it was passed without the public being told what the final form would be. The secretive, anti-democratic manner in which the new law transposing the EU Copyright Directive was rammed through is one of the worst examples of the contempt often shown in the field of copyright by politicians for the public they supposedly represent and serve.

In terms of major copyright changes, there were two main novelties in the EU Copyright Directive. The first was the so-called “link tax”, supposedly about saving the European newspaper industry and supporting good journalism. The reality has proved rather different, as a recent analysis showed, confirming earlier experiences in Spain and Germany. More recently, Canada introduced its own link tax, and research already indicates that it has been a failure from multiple viewpoints.

The other major change brought in by the EU Copyright Directive concerned the requirement for platforms to obtain licences for any copyright material that is uploaded to their servers, and to use filters to block material that is not licensed. Remarkably little has been heard about these upload filters since the Directive was passed and implemented. But that’s not because they are working well, or that people have stopped uploading copyright material to sites – the prime justification for upload filters – but because the copyright industry has moved on to bigger things. It no longer wants a few large platforms to filter uploads, it wants the entire Internet infrastructure to filter everything at all times.

In 2023, basic Internet infrastructure services – DNS and CDN – were accused of involvement in making unauthorised copies of material, even though they played no active role in its transmission. The same year, there was a French proposal to force browser providers to create the means to mandatorily block websites present on a government provided list. Then came a suggestion that the blocking should take place on the Internet’s core routers. That was just a proposal, but Italy has already brought in requirements for Piracy Shield blackout orders to be applied by DNS and even VPN providers.

Chapter 7 discussed the increasing consolidation and concentration of media power, to the detriment of creators, and the extremely dubious activities of many collective right organisations (CROs). The former has continued apace, and some CROs are finding new ways to increase their income. Similarly, the loss of culture as a result of copyright, discussed in chapter 8, has not diminished. Since the Walled Culture book was published, further examples of this unnecessary destruction have emerged in the world of films, classic TV programmes, video games, and even the Web itself. There are doubtless many others yet to be discovered.

The one bright spot in terms of recent developments is in the context of Chapter 9. This concluded by looking at the idea of true fans, and how they might offer an alternative and fairer approach to supporting creators. As numerous recent Walled Culture posts attest, this seems to be catching on, and in some surprising places.

The biggest development in the copyright world since the Walled Culture book was launched is undoubtedly generative AI. I didn’t mention it in the book, because the idea only entered the mainstream in November 2022, when ChatGPT was launched, and thus many months after Walled Culture was written.

There is a certain irony in the apparently unstoppable rise of genAI, and the ensuing panic by the copyright industry. As Walled Culture the book details, companies in the latter sector have been obsessed with preventing ordinary people from using the Internet to make even a single unauthorised copy of material – often threatening extreme legal and economic sanctions. And yet today’s hot genAI companies – startups like OpenAI, and established players like Google – have used vast swathes of the digital world in order to train their AI systems, and there is little the copyright companies have been able to do about it. To be sure there are plenty of lawsuits underway, and even a few fines. But the sums involved are likely to be relatively small compared to the extraordinary valuations placed on genAI companies – making them just another minor cost of doing business.

What’s really problematic about the rise of services based on generative AI is not that they are using huge amounts of copyright material – those will probably be licensed in one way or another – but that the creators of that material are unlikely to see much benefit from the payments made to publishers, recording companies and studios. Generative AI is portrayed as a novel and exciting chapter in technology, but actually, it’s not new, since much of the foundational work goes back decades. And as far as ordinary people and creators are concerned, it is definitely the same old copyright story: companies reaping the benefits and profits, while everyone else is expected to accept with gratitude the few scraps tossed their way.

Follow me @glynmoody on Mastodon and on Bluesky.

1 2 3 … 21
WordPress Cookie Plugin by Real Cookie Banner