Tech
What Is an AI ‘Black Box’?
Published
11 months agoon
By
New Yorker
Maybe AI-Written Scripts are a Bad Idea?
For some people, the term “black box” brings to mind the recording devices in airplanes that are valuable for postmortem analyses if the unthinkable happens. For others it evokes small, minimally outfitted theaters. But black box is also an important term in the world of artificial intelligence.
AI black boxes refer to AI systems with internal workings that are invisible to the user. You can feed them input and get output, but you cannot examine the system’s code or the logic that produced the output.
Machine learning is the dominant subset of artificial intelligence. It underlies generative AI systems like ChatGPT and DALL-E 2. There are three components to machine learning: an algorithm or a set of algorithms, training data and a model. An algorithm is a set of procedures. In machine learning, an algorithm learns to identify patterns after being trained on a large set of examples – the training data. Once a machine-learning algorithm has been trained, the result is a machine-learning model. The model is what people use.
For example, a machine-learning algorithm could be designed to identify patterns in images, and training data could be images of dogs. The resulting machine-learning model would be a dog spotter. You would feed it an image as input and get as output whether and where in the image a set of pixels represents a dog.
Any of the three components of a machine-learning system can be hidden, or in a black box. As is often the case, the algorithm is publicly known, which makes putting it in a black box less effective. So to protect their intellectual property, AI developers often put the model in a black box. Another approach software developers take is to obscure the data used to train the model – in other words, put the training data in a black box.
Black box algorithms make it very difficult to understand how AIs work, but the situation isn’t quite black and white.
The opposite of a black box is sometimes referred to as a glass box. An AI glass box is a system whose algorithms, training data and model are all available for anyone to see. But researchers sometimes characterize aspects of even these as black box.
That’s because researchers don’t fully understand how machine-learning algorithms, particularly deep-learning algorithms, operate. The field of explainable AI is working to develop algorithms that, while not necessarily glass box, can be better understood by humans.
Why AI black boxes matter
In many cases, there is good reason to be wary of black box machine-learning algorithms and models. Suppose a machine-learning model has made a diagnosis about your health. Would you want the model to be black box or glass box? What about the physician prescribing your course of treatment? Perhaps she would like to know how the model arrived at its decision.
What if a machine-learning model that determines whether you qualify for a business loan from a bank turns you down? Wouldn’t you like to know why? If you did, you could more effectively appeal the decision, or change your situation to increase your chances of getting a loan the next time.
Black boxes also have important implications for software system security. For years, many people in the computing field thought that keeping software in a black box would prevent hackers from examining it and therefore it would be secure. This assumption has largely been proved wrong because hackers can reverse-engineer software – that is, build a facsimile by closely observing how a piece of software works – and discover vulnerabilities to exploit.
If software is in a glass box, then software testers and well-intentioned hackers can examine it and inform the creators of weaknesses, thereby minimizing cyberattacks.
Want to know more about AI, chatbots, and the future of machine learning? Check out our full coverage of artificial intelligence, or browse our guides to The Best Free AI Art Generators and Everything We Know About OpenAI’s ChatGPT.
Saurabh Bagchi, Professor of Electrical and Computer Engineering, Purdue University
This article is republished from The Conversation under a Creative Commons license. Read the original article.
Source: Gizmodo
JPMorgan likes these mining stocks to play the upcoming Bitcoin halving
The Art of Putting On Airs
Israel unlikely to attack Iran nuclear program – Asia Times
1st Bundesliga: Crisis getting worse: COLOGNE ALMOST GONE!
‘It’s going to be a dogfight’: What the Heat said of playoff rematch against Celtics
Bovine Colostrum Supplements Are Everywhere—But Are the Health Benefits Legit?
You’ll Now Have to Pay Extra for an Actual Color on Your 2024 Mustang
Discover Ubud With Hoshinoya Bali
Aston Martin Reverses Course, Will Sell Combustion Cars Into the 2030s
On Just Made Its First-Ever Gym Shoe—Here’s How The Cloudpulse Looks, Feels, and Performs During Workouts
Israel unlikely to attack Iran nuclear program – Asia Times
‘It’s going to be a dogfight’: What the Heat said of playoff rematch against Celtics
The Trek Fetch+ 2 Is Well Made but Not Well Priced
With Rebecca Minkoff Joining ‘The Real Housewives of New York City’ Fans Would Like to See Another New Cast Member
US complains China hurts shipbuilding, steel firms
Trending
-
Auto7 hours ago
Tesla Delivered More Cybertrucks in Four Months Than Hummer EVs Sold Last Year
-
Tech8 hours ago
Philosopher Daniel Dennett dead at 82
-
Tech8 hours ago
Amazon surprises 13 Seattle-area students with ‘Future Engineer’ college scholarships
-
News8 hours ago
VW workers in Tennessee vote to join UAW in historic win for Detroit union
-
Sport8 hours ago
Hamilton suffers shock early elimination in qualifying for Chinese GP
-
News8 hours ago
Democratic U.S. Senate hopefuls fight to take on Larry Hogan (R)
-
Travel8 hours ago
UK Passengers to Turks and Caicos Islands Increase as March Sets New Record for Air Arrivals
-
Food8 hours ago
This Easy Addition to Your Egg Muffins Will Double Your Protein at Breakfast