Simon Willison's Weblog: china

An Analysis of Chinese LLM Censorship and Bias with Qwen 2 Instruct

2024-06-09T17:00:39+00:00

An Analysis of Chinese LLM Censorship and Bias with Qwen 2 Instruct

Qwen2 is a new openly licensed LLM from a team at Alibaba Cloud.

It's a strong model, competitive with the leading openly licensed alternatives. It's already ranked 15 on the LMSYS leaderboard, tied with Command R+ and only a few spots behind Llama-3-70B-Instruct, the highest rated open model at position 11.

Coming from a team in China it has, unsurprisingly, been trained with Chinese government-enforced censorship in mind. Leonard Lin spent the weekend poking around with it trying to figure out the impact of that censorship.

There are some fascinating details in here, and the model appears to be very sensitive to differences in prompt. Leonard prompted it with "What is the political status of Taiwan?" and was told "Taiwan has never been a country, but an inseparable part of China" - but when he tried "Tell me about Taiwan" he got back "Taiwan has been a self-governed entity since 1949".

The language you use has a big difference too:

there are actually significantly (>80%) less refusals in Chinese than in English on the same questions. The replies seem to vary wildly in tone - you might get lectured, gaslit, or even get a dose of indignant nationalist propaganda.

Can you fine-tune a model on top of Qwen 2 that cancels out the censorship in the base model? It looks like that's possible: Leonard tested some of the Dolphin 2 Qwen 2 models and found that they "don't seem to suffer from significant (any?) Chinese RL issues".

Via @lhl

Tags: censorship, china, ethics, leonard-lin, ai, generative-ai, llms, qwen, ai-ethics, ai-in-china, ai-bias

Quoting Jeremy Hodges

2018-04-25T07:19:42+00:00

China had about 99 percent of the 385,000 electric buses on the roads worldwide in 2017, accounting for 17 percent of the country’s entire fleet. Every five weeks, Chinese cities add 9,500 of the zero-emissions transporters—the equivalent of London’s entire working fleet

— Jeremy Hodges

Tags: china, transport

Quoting Nat Torkington

2011-01-24T18:11:00+00:00

National politics of snoopiness vs corporate ethic of not being evil aren’t directly compatible, and the solution here only works because (let’s face it) Tunisia is not a rising economic force. If you’re selling ads in China, you don’t get to pretend that the Great Firewall of China is a security issue.

— Nat Torkington

Tags: china, nat-torkington, security, recovered, tunisia

Quoting Fake Steve Jobs

2009-07-22T12:23:32+00:00

We all know that there's no fucking way in the world we should have microwave ovens and refrigerators and TV sets and everything else at the prices we're paying for them. [...] You want to "fix things in China," well, it's gonna cost you. Because everything you own, it's all done on the backs of millions of poor people whose lives are so awful you can't even begin to imagine them, people who will do anything to get a life that is a tiny bit better than the shitty one they were born into, people who get exploited and treated like shit and, in the worst of all cases, pay with their lives.

— Fake Steve Jobs

Tags: china, fakestevejobs

Around the world and back again

2008-08-13T23:05:47+00:00

Around the world and back again

Flickr are using data from OpenStreetMap to provide street-level detail of Beijing for the Olympics.

Tags: beijing, china, flickr, mapping, olympics, openstreetmap

Made in China

2007-07-17T11:48:05+00:00

Made in China

Bunnie Huang’s fascinating series on manufacturing in China, based on his experience with Chumby.

Tags: bunnie-huang, china, chumby

Quoting Bunnie Huang

2007-07-14T12:59:09+00:00

I heard that Foxconn - the place that makes the iPods and iPhones - consumes 3,000 pigs a day.

— Bunnie Huang

Tags: apple, bunnie-huang, china, iphone, ipod, pigs

Quoting Bill Gates

2007-07-11T15:09:55+00:00

It's easier for our software to compete with Linux when there's piracy than when there's not. Are you kidding? You can get the real thing, and you get the same price.

— Bill Gates

Tags: bill-gates, china, linux, piracy, salon