Accurate, Focused Research on Law, Technology and Knowledge Discovery Since 2002

Daily Archives: August 1, 2023

A jargon-free explanation of how AI large language models work

Ars Technica: “When ChatGPT was introduced last fall, it sent shockwaves through the technology industry and the larger world. Machine learning researchers had been experimenting with large language models (LLMs) for a few years by that point, but the general public had not been paying close attention and didn’t realize how powerful they had become. Today, almost everyone has heard about LLMs, and tens of millions of people have tried them out. But not very many people understand how they work. If you know anything about this subject, you’ve probably heard that LLMs are trained to “predict the next word” and that they require huge amounts of text to do this. But that tends to be where the explanation stops. The details of how they predict the next word is often treated as a deep mystery. One reason for this is the unusual way these systems were developed. Conventional software is created by human programmers, who give computers explicit, step-by-step instructions. By contrast, ChatGPT is built on a neural network that was trained using billions of words of ordinary language. As a result, no one on Earth fully understands the inner workings of LLMs. Researchers are working to gain a better understanding, but this is a slow process that will take years—perhaps decades—to complete. Still, there’s a lot that experts do understand about how these systems work. The goal of this article is to make a lot of this knowledge accessible to a broad audience. We’ll aim to explain what’s known about the inner workings of these models without resorting to technical jargon or advanced math. We’ll start by explaining word vectors, the surprising way language models represent and reason about language. Then we’ll dive deep into the transformer, the basic building block for systems like ChatGPT. Finally, we’ll explain how these models are trained and explore why good performance requires such phenomenally large quantities of data.”

Fortune Global 500

“The corporations on our annual list of the world’s largest companies showed their muscle in 2022, delivering record-high aggregate revenues of $41 trillion. But the Fortune Global 500 has been anything but static, as technological change and scientific breakthroughs threaten established leaders and elevate new winners. Walmart remains No. 1 for the 10th consecutive year,… Continue Reading

The Live Music Archive Lets You Stream

Open Culture: Download More Than 250,000 Concert Recordings–for Free: “The Internet Archive maintains an enormous Live Music Archive of concert recordings, not all of them by the Grateful Dead. There are more than 17,000 such recordings in its Grateful Dead collection — 2,000 more than when last we featured it here on Open Culture —… Continue Reading

We’re now finding out the damaging results of the mandated return to the office

Fortune: “And it’s not a pretty picture. A trio of compelling reports—the Greenhouse Candidate Experience report, the Federal Reserve’s Survey of Household Economics and Decisionmaking (SHED), and Unispace’s Returning for Good report—collectively paint a stark picture of this brewing storm. Unispace found that nearly half (42%) of companies with return-to-office mandates witnessed a higher level… Continue Reading

Falling Fruit – Map the Urban Harvest

“Falling Fruit is a celebration of the overlooked culinary bounty of our city streets. By quantifying this resource on an interactive map, we hope to facilitate intimate connections between people, food, and the natural organisms growing in our neighborhoods. Not just a free lunch! Foraging in the 21st century is an opportunity for urban exploration,… Continue Reading

New Attack Impacts Major AI Chatbots

Wired: “ChatGPT and its artificially intelligent siblings have been tweaked over and over to prevent troublemakers from getting them to spit out undesirable messages such as hate speech, personal information, or step-by-step instructions for building an improvised bomb. But researchers at Carnegie Mellon University last week showed that adding a simple incantation to a prompt—a… Continue Reading

How governments are looking to regulate AI

Economist Intelligence – EIU: The EU has taken the lead with the strictest rules, which will have the biggest impact globally Chinese rules are also strict, but their impact is domestic, and geared towards keeping control and power with the ruling party The US favours innovation, and its political system hinders any attempt at regulation,… Continue Reading

Global Climate Shift Index

“The Global Climate Shift Index™ is a tool that quantifies the local influence of climate change on daily temperatures around the world. Climate Central is an independent group of scientists and communicators who research and report the facts about our changing climate and how it affects people’s lives. We are a policy-neutral 501(c)(3) nonprofit. Climate… Continue Reading

Federal grand jury indicts Trump on four counts

Link to text of criminal indictment filed in the United States District Court for the District of Columbia 08/01/23 in case number 1:23-cr-00257-TSC, includes for co-conspirators (news reports identify them as Rudy Giuliani; John Eastman; Syndey Powell; Jeffrey Clark, Kenneth Chesebro, and yet unknown.)…. “From on or about November 14, 2020, through on or about… Continue Reading