What are you worth? Your answer today may be based on what you earn, the price of your possessions, or how much your family loves you. But in the near future, you may well be rated in a whole different way – and your online behavior most certainly will be.
“We live in a world where judgment is replaced by numbers — by scores that calculate the value of a human being, using algorithms,” says Gerd Gigerenzer, director of the Harding Center for Risk Literacy at Max Planck Institute for Human Development in Berlin. Mr. Gigerenzer also heads a council of consumer experts that advises the German Ministry of Justice. The group is currently working on a report on this subject.
Evaluation by algorithms is progressing in China, where an experiment of what is called a “social credit system” is underway, indicates the professor of psychology and specialist in behavioral sciences. China’s system, which uses a combination of mass surveillance and big data to score citizens, is currently voluntary but will be mandatory by 2020. At this point, every citizen and business will be rated whether they want to or no.
“If you break social conventions, look for the wrong website, buy too many video games, cross the road at a red light, or even have friends with a low score, then yours is it. score will fall,” Mr Gigerenzer told the Handelsblatt’s sister publication, Der Tagesspiegel.
Jetzt die besten Jobs finden und
by e-mail benachrichtigt werden.
If you have something you don’t want anyone to know, maybe you shouldn’t do it in the first place. Eric Schmidt, former executive chairman of Google
In the various experimental versions of the system – deployed by local governments and selected companies – millions of poorly rated citizens are already labeled as “unqualified” to book flights or high-speed train tickets, let alone get a loan from the bank. The potential implications could be even worse. “If your score drops too much, your children will not be able to go to the best schools and many other limitations will apply,” warns the expert. A system like that ensures self-censorship even within families, he says, predicting that the plan is bound to be successful.
It looks like a drastic, Orwellian version of the future. But as Mr. Gigerenzer notes, we in Western countries are not as far from such a system as we would like to think. In Germany, he points out, it starts with the universal credit scoring system known as Schufa. Like its American counterpart FICO, Schufa is a private company that assesses the creditworthiness of around three-quarters of all Germans and more than 5 million businesses in the country. Anyone wishing to rent a house or lend money must produce their Schufa score in Germany – or their FICO score in the United States. Additionally, factors like “geo-score” can also lower your overall rating if you live in a low-income neighborhood, or even if many of your neighbors have poor credit.
In other areas, German health insurers will offer you lower premiums if you fall ill less often. They can offer you even better bonuses if you share data from your fitness tracker to show you’re doing your part to stay healthy. Anyone using websites like Amazon, eBay or Airbnb is invited to rate others and is rated themselves. Those who try to avoid being graded are looked down upon. A growing number of consumers will be denied certain services or, say, mortgages if they don’t show some sort of rating.
“We are in a crucial phase where we need to have a discussion about our values,” argues Mr. Gigerenzer. “Do we want to continue like this? Should we continue to rate people in areas such as finance, health, crime, rental housing, mail order businesses, etc.? ? And if you answered yes to that question, then here comes the next question: Should we allow all the collected data to be aggregated, so that we can arrive at a total score for each citizen?
In China, the answer is a resounding yes, and yes. Beijing is selling the idea of a social credit system as a way to build trust and fight crime and Corruption. If an institution or an individual has a good score, then you can trust them, the Chinese authorities trumpet.
And it works like a charm. Citizens welcome the fact that the rating system will help them determine whether the people around them are trustworthy or not, says the behavioral scientist. “And I believe that even in Germany there would be a certain group that would support the idea of ’digitally transparent’ people.”
Mr. Gigerenzer points out that Eric Schmidt, former executive chairman of Google, spoke controversially about the “right to know” in 2009. The entrepreneur, who recently became a technical adviser to the global giant, is a strong advocate of ” full transparency”. .”
“If you have something you don’t want anyone to know, maybe you shouldn’t do it in the first place,” Mr. Schmidt said.
“I see a remarkable similarity between Mr. Schmidt and the Chinese government,” notes Mr. Gigerenzer, adding that Baidu, China’s equivalent of Google, is already working with Beijing’s nascent social credit system.
Then again, while you like the idea of a social credit system, Gigerenzer thinks the technology involved isn’t quite up to par yet. Artificial intelligence could work very well with games like chess, or in other situations with well-defined parameters, he says. “But the situation looks totally different when we talk about real situations with a lot of uncertainties.”
He gives two examples. First, the program, Google Flu Trends, which was launched to much fanfare in 2008 and which attempted to determine the prevalence and location of flu cases by aggregating all the different online searches made by Google users about flu symptoms. But after the software nearly missed the peak of flu season in 2013, the program was quietly abandoned.
Second, Mr. Gigerenzer mentions COMPAS, a recidivism prediction algorithm used in various US states. The tool was developed to help judges determine sentencing by reviewing the criminal history of defendants and then predicting the likelihood of them reoffending. But further research found the risk-assessment algorithm was flawed in more than a third of cases, along with racial bias. In recent experiments, ordinary people with no experience in the field have done better at telling offenders’ fortunes than the algorithm could.
“It would be tragic if someone’s life were destroyed simply because others blindly trusted a commercial algorithm,” says Gigerenzer.
So what to do with this process, especially since it is already well under way?
This would have been an important topic for the recent The coalition talks to form the next German government, explains Mr. Gigerenzer. In the constant discourse on digital, the social and psychological dimensions are left aside, he continues. “A huge blind spot has developed.”
When its expert panel completes its report on “consumer scoring”, it will be presented to the Department of Justice. Gigerenzer hopes the report will also spark a lot more public debate on the subject.
There is growing criticism in Germany of what some see as a lack of transparency in the establishment of Schufa scores. In 2014, the Federal Court of Justice dismissed the lawsuit of a woman who wanted to know how the company had calculated her negative credit score.
But two non-governmental organizations, AlgorithmWatch and the Open Knowledge Foundation, launched the OpenSchufa initiative earlier this month. Supported by data journalists, they set out to crack Schufa’s algorithm. They rely primarily on financial information provided by the public.
“If we do nothing, one day a company or a government institution will gather all the information from different databases and compile a social credit score,” warns Mr. Gigerenzer. “And in the end, we will be in the same state as the Chinese. Right now, we are investing billions in digital technologies,” he continues. “Whereas we should invest so much in digital education so that humans know what algorithms can and cannot do. We cannot sit idly by as they are used to change our minds and our societies.
This interview was conducted by Heike Jahberg, who writes on consumer issues for the Handelsblatt’s sister publication, Der Tagesspiegel. This story was adapted for Handelsblatt Global by Cathrin Schaer. To contact the author: [email protected]