Civic Lab Online: How Does Machine & Algorithm Bias Affect All of Us?

Posted on February 8, 2023 at 6:00 am

About Civic Lab Online

Civic Lab Online provides information on issues facing our community for you to explore. Take a look at thought-provoking materials for teens and adults that allow us to engage in open conversation and grow together as a community. You’ll find all past topics on the Civic Lab Online web page.

Machine & Algorithm Bias: Fast Facts

Access to good information is necessary for us to make informed decisions, so search engines and software use patterns and mathematics to guess what users want. However, the algorithms that machines use can also reflect gender, class, and racial bias. Search engines may favor results with a financial incentive. Unintentional (and perhaps intentional) machine bias affects our everyday lives. Below are fast facts on how this happens.

In what ways could a computer be biased? Can’t algorithms avoid bias by not collecting irrelevant information?

Friedman and Nissenbaum’s 1996 study was one of the first to show that software can systematically and unfairly discriminate against certain individuals and groups. Pre-existing bias in society can affect system designs, technical bias can occur due to technical limitations, and emergent bias can arise after software is completed and released.

Artificial Intelligence (AI) bots created to connect jobs and job applicants didn’t collect information about the sex of applicants but did exclude most female applicants. A bot can be programmed to make connections between past successful male employees and the language male applicants typically use on resumes. If programmed to collect applicants who are more likely to apply, bots often rejected profiles of female job seekers who statistically apply for a limited pool of jobs they’re qualified for and instead prioritized men who applied widely for all kinds of jobs. 

Risk assessments calculated by an algorithm, or set of programmed computer calculations, are given to judges in court during sentencing to predict whether criminals will reoffend. Whether the algorithms work has only been examined in a few studies, primarily by those selling the software. When ProPublica obtained the risk scores of more than 7,000 people arrested in the same Florida county, only 20 percent of the people predicted to commit violent crimes had actually done so, and only 61 percent had committed any crime including misdemeanors. Black defendants were consistently mislabeled as more likely to reoffend and white defendants mislabeled as less likely, though they were asked about home life, employment, and education rather than race.

The algorithm that helps human Transportation Security Administration (TSA) agents assess scanned bodies for suspicious activity has two gender options, meaning that transgender travelers are far more likely to be flagged as suspicious in an airport because the millimeter wave scanner’s data on a typical body type doesn’t match.

What about just retrieving the information I want to know? Can a computer’s algorithms have bias then?

Search engines are defined as biased if they favor results of their partners and bury competitors. Google has had several antitrust (laws that determine legal business practices) battles over bias.

Google entered a two-year trial that ended in 2013 to determine whether their Universal Search function (including maps, news, and other options in results rather than just 10 blue links) was harming competitors by defaulting to Google products, such as Google Places rather than TripAdvisor. Google argued that other search engines such as Yahoo! and Bing have copied the Universal Search interface and favor their own results just as much. This practice continues primarily because the suggested solutions proved unconstitutional or impractical:

• To have the government somehow monitor Google algorithms for neutrality
• To constrain Google’s ability to partner with websites and apps
• To require Google to release more information about how their algorithms work, which would allow competitors to copy and spammers to manipulate them
• To invert Fair Use Doctrine (legally using quotes from content for reporting purposes) to restrict Google from crawling webpages (searching the Internet for content) to use in search results, but letting other search engines crawl Google’s content
• To restrict Google from Universal Search and return to the ten blue links model

Similarly, in April 2021 a United States Senate Subcommittee held a hearing to discuss Google’s and Apple’s roles in the mobile app ecosystem, and whether they are making competing apps such as Spotify, Match Group, and Tile difficult to access or unaffordable in their app stores.

Users often assume that search engines are impartial because they are automated, but search engine algorithms are created by humans and most companies include a human element to search results as well. Google and others have admitted that employees manually blacklist or whitelist certain websites, especially if the algorithm didn’t rank the company as expected. Human operators also evaluate and can remove information at the request of governments.

200 different factors are used by Google to calculate a page’s rank in search results. Not all of the factors are public knowledge, and neither is how they are weighted (how factors are valued versus others). Google’s worldwide data centers are also not in sync, so data center algorithms may differ depending on location.

Algorithms that make predictions, such as Bing’s incorrect predictions about the vote to leave the European Union or Google’s Flu Trend predictions that were off by 140 percent, sometimes fail because they can only collect data that people put on the internet, and enough people act offline and not online to skew results.

Social media algorithms often weigh novelty over popularity. A new term is more likely to become a “Twitter Trend” than a widespread movement that has existed for longer, even when more people are currently tweeting about the movement. This pushes the focus to viral memes and makes it more difficult to track slowly building news. It’s also why you’re more likely to order from businesses that post while you’re online. Search engines work the opposite way—a domain that has been established for longer will be ranked higher in results.

Personalization algorithms collect information on what an individual user does online in order to make predictions about what information will be most useful to them. When you search for “Spokane businesses,” the search engine will look at your location and give you results for Spokane, Washington, rather than Spokane, South Dakota. Google uses signals like your location, previous search keywords, and social media contacts; whereas Facebook and YouTube use your interaction with others (social gestures) and what you’ve watched, liked, and shared.

• Personalization algorithms hide information that they predict will be unwanted, which can mean that you stop seeing social media posts from friends you don’t contact very often or that certain websites will never show up in your search—all without you knowing you’re missing them.
• A recommendation mechanism collects information on what you’ve purchased or read, looking for similar items and curating online content such as ads specifically targeting you.
• Rather than returning results that might go against a user’s current beliefs, even if accurate, and prompt them to decide on a new course of action or thought, personalized algorithms return results that are comfortable for the user. When it comes to news and facts, users might end up in “filter bubbles” that give them information they want to hear over information that is correct.

Read. Watch. Listen.

READ

“The Ethical Algorithm: The Science of Socially Aware Algorithm Design.” Michael Kearns and Aaron Roth,2020. Accessed 3 Feb. 2023.

Algorithms have made our lives more efficient, more entertaining, and sometimes, better informed. At the same time, complex algorithms are increasingly violating the basic rights of individual citizens. Allegedly anonymized datasets routinely leak our most sensitive personal information. Statistical models for everything from mortgages to college admissions reflect racial and gender bias. Meanwhile, users manipulate algorithms to “game” search engines, spam filters, online reviewing services, and navigation apps. Michael Kearns and Aaron Roth offer a new approach: a set of principled solutions based on the emerging and exciting science of socially aware algorithm design.

“The Filter Bubble: What The Internet Is Hiding From You.” Eli Pariser, 2011. Accessed 3 Feb. 2023.

In December 2009, Google began customizing its search results for each user. Instead of giving you the most broadly popular result, Google tries to predict what you are most likely to click on. According to MoveOn.org board president Eli Pariser, Google’s change in policy is symptomatic of the most significant shift to take place on the Web in recent years—the rise of personalization. In this groundbreaking investigation of the new hidden Web, Pariser uncovers how this growing trend threatens to control how we consume and share information as a society—and reveals what we can do about it.

“How Search Engines Work: Crawling, Indexing, and Ranking – Beginner’s Guide to SEO.” Brittany Mueller and Moz Staff, 2023. Moz.com. https://moz.com/beginners-guide-to-seo/how-search-engines-operate. Accessed 3 Feb. 2023.

This article gives a quick overview of how search engines find, scan, and index web pages for specific terms and how they rank them in order to be found when you search online.

WATCH

Grey, CGP. “How Machines Learn.” YouTube, 18 Dec. 2017, www.youtube.com/watch?v=R9OHn5ZF4Uo. Accessed 3 Feb. 2023.

This humorous video explains how the algorithms that make online bots work can also be too complex for bots or the humans who created them to understand and yet still consistently improve the bots techniques to work better over time.

LISTEN

MIT Technology Review. “Podcast: In Machines We Trust – Hired by an Algorithm.” YouTube, 23 June 2021, www.youtube.com/watch?v=ztcVB_zh_M0. Accessed 3 Feb. 2023.

If you’ve applied for a job recently, then it is all but guaranteed that your application was reviewed by software—in most cases, before a human ever saw it. In this episode, the first in a four-part investigation into automated hiring practices, the hosts speak with the CEOs of ZipRecruiter and Career Builder and one of the architects of LinkedIn’s algorithmic job-matching system to explore how AI is increasingly connecting job searchers and employers. But while software helps speed up the process of sifting through the job market, algorithms have a history of biasing the opportunities they present to people by gender, race, and in at least one case, by whether you played lacrosse in high school.

Additional Information

What if I want to limit the bias of the information that is delivered to me? Or if I want to limit the information search engines collect about me? Do I have options?

It is possible to limit the information algorithms collect and associate with your contact information. Here are some ideas to consider trying out:

You can opt to use a private search engine, such as DuckDuckGo or Startpage, many of which are available online for free.

You can also use the incognito tab on your browser, and none of your searches while in incognito mode will be used to market to you.

You can opt out of Google personalized search results and/or check your Google settings and Google Dashboard to see what information is being collected.

If you stay logged in to your email while searching the internet, many email carriers make it a practice to track your searches and connect your search history with your name and email contact information. This is more difficult for browsers to do if you make it a practice to log out of your email before searching online.

First party cookies track your activity on a site and report back to the site. Third party cookies track your activity after you leave a site and report it back to the original site. On most computers and phones, you can go into Settings and choose whether you want to accept, block, or be prompted for first-party and third-party cookies.

Take a moment to look at your search results, especially for news or politics. Do they all seem to be from the same point of view? Do they have headlines that seem designed to make you anxious or upset? To help spot biased search results, you can make it a practice to visit a few different sites that are highly rated for the accuracy of their reporting and choose at least one that skews differently from your own viewpoint. This not only helps you spot biased search results but will make your search engine results less tailored to what the algorithm detects as your beliefs and comfortability with the search results.

Digital & Print Resources

Learn more about machine & algorithm bias with ProQuest, Gale in Context: Science & Gale in Context: Global Issues.

Print and Other Materials in Our Catalog
Search our catalog for books, large print, eBooks, and audiobooks.

Downloadable Documents

Machine & Algorithm Bias: Fast Facts
Machine & Algorithm Bias: Read. Watch. Listen.
Machine & Algorithm Bias: Additional Information
Machine & Algorithm Bias: Sources

Tags: adults, algorithm bias, civic lab, community, Discussion, Google, machine bias, privacy, search engines, teens