Later On

A blog written for those whose interests more or less match mine.

Archive for the ‘Go’ Category

Excellent full-length documentary on AlphaGo and the match against the world champion

leave a comment »

I highly recommend this documentary even to those who do not play Go. I have no knowledge of (or interest in) football, but I loved the series “Friday Night Lights,” as so many do, not because of the football but because of the human drama. Football is really just the MacGuffin. The story is about the people, and it is absorbing because of that. So it is with this documentary.

Written by LeisureGuy

13 June 2020 at 9:24 pm

AlphaGo: The movie

leave a comment »

The full documentary:

Written by LeisureGuy

22 March 2020 at 10:02 am

Posted in Games, Go, Movies & TV

The Shaw Alphabet and Other Quixotic Solutions I Love

leave a comment »

I just published on Medium a brief article the lists a number of interests that regular blog readers will recognize.

Written by LeisureGuy

11 January 2020 at 2:29 pm

Posted in Daily life, Games, Go, Health

Free Go book as PDF: “Go Studies: A History of Adventure”

leave a comment »

I just learned of this site: ExploreBaduk.com, with Baduk being the Korean name for Go. And the first thing I saw was a free book on Go: Go Studies: A History of AdventureScroll down that page to get to the download buttons.

It’s really a terrific site, with a lot of very good content. Take look. Remember your New Year’s Resolution to learn Go. 🙂

Written by LeisureGuy

13 January 2019 at 1:43 pm

Posted in Books, Games, Go

How the Artificial-Intelligence Program AlphaZero Mastered Its Games

leave a comment »

James Somers writes in the New Yorker:

A few weeks ago, a group of researchers from Google’s artificial-intelligence subsidiary, DeepMind, published a paper in the journal Science that described an A.I. for playing games. While their system is general-purpose enough to work for many two-person games, the researchers had adapted it specifically for Go, chess, and shogi (“Japanese chess”); it was given no knowledge beyond the rules of each game. At first it made random moves. Then it started learning through self-play. Over the course of nine hours, the chess version of the program played forty-four million games against itself on a massive cluster of specialized Google hardware. After two hours, it began performing better than human players; after four, it was beating the best chess engine in the world.

The program, called AlphaZero, descends from AlphaGo, an A.I. that became known for defeating Lee Sedol, the world’s best Go player, in March of 2016. Sedol’s defeat was a stunning upset. In “AlphaGo,” a documentary released earlier this year on Netflix, the filmmakers follow both the team that developed the A.I. and its human opponents, who have devoted their lives to the game. We watch as these humans experience the stages of a new kind of grief. At first, they don’t see how they can lose to a machine: “I believe that human intuition is still too advanced for A.I. to have caught up,” Sedol says, the day before his five-game match with AlphaGo. Then, when the machine starts winning, a kind of panic sets in. In one particularly poignant moment, Sedol, under pressure after having lost his first game, gets up from the table and, leaving his clock running, walks outside for a cigarette. He looks out over the rooftops of Seoul. (On the Internet, more than fifty million people were watching the match.) Meanwhile, the A.I., unaware that its opponent has gone anywhere, plays a move that commentators called creative, surprising, and beautiful. In the end, Sedol lost, 1-4. Before there could be acceptance, there was depression. “I want to apologize for being so powerless,” he said in a press conference. Eventually, Sedol, along with the rest of the Go community, came to appreciate the machine. “I think this will bring a new paradigm to Go,” he said. Fan Hui, the European champion, agreed. “Maybe it can show humans something we’ve never discovered. Maybe it’s beautiful.”

AlphaGo was a triumph for its creators, but still unsatisfying, because it depended so much on human Go expertise. The A.I. learned which moves it should make, in part, by trying to mimic world-class players. It also used a set of hand-coded heuristics to avoid the worst blunders when looking ahead in games. To the researchers building AlphaGo, this knowledge felt like a crutch. They set out to build a new version of the A.I. that learned on its own, as a “tabula rasa.”

The result, AlphaGo Zero, detailed in a paper published in October, 2017, was so called because it had zero knowledge of Go beyond the rules. This new program was much less well-known; perhaps you can ask for the world’s attention only so many times. But in a way it was the more remarkable achievement, one that no longer had much to do with Go at all. In fact, less than two months later, DeepMind published a preprint of a third paper, showing that the algorithm behind AlphaGo Zero could be generalized to any two-person, zero-sum game of perfect information (that is, a game in which there are no hidden elements, such as face-down cards in poker). DeepMind dropped the “Go” from the name and christened its new system AlphaZero. At its core was an algorithm so powerful that you could give it the rules of humanity’s richest and most studied games and, later that day, it would become the best player there has ever been. Perhaps more surprising, this iteration of the system was also by far the simplest.

A typical chess engine is a hodgepodge of tweaks and shims made over decades of trial and error. The best engine in the world, Stockfish, is open source, and it gets better by a kind of Darwinian selection: someone suggests an idea; tens of thousands of games are played between the version with the idea and the version without it; the best version wins. As a result, it is not a particularly elegant program, and it can be hard for coders to understand. Many of the changes programmers make to Stockfish are best formulated in terms of chess, not computer science, and concern how to evaluate a given situation on the board: Should a knight be worth 2.1 points or 2.2? What if it’s on the third rank, and the opponent has an opposite-colored bishop? To illustrate this point, David Silver, the head of research at DeepMind, once listed the moving parts in Stockfish. There are more than fifty of them, each requiring a significant amount of code, each a bit of hard-won chess arcana: the Counter Move Heuristic; databases of known endgames; evaluation modules for Doubled Pawns, Trapped Pieces, Rooks on (Semi) Open Files, and so on; strategies for searching the tree of possible moves, like “aspiration windows” and “iterative deepening.”

AlphaZero, by contrast, has only two parts: a neural network and an algorithm called Monte Carlo Tree Search. (In a nod to the gaming mecca, mathematicians refer to approaches that involve some randomness as “Monte Carlo methods.”) The idea behind M.C.T.S., as it’s often known, is that a game like chess is really a tree of possibilities. If I move my rook to d8, you could capture it or let it be, at which point I could push a pawn or move my bishop or protect my queen. . . . The trouble is that this tree gets incredibly large incredibly quickly. No amount of computing power would be enough to search it exhaustively. An expert human player is an expert precisely because her mind automatically identifies the essential parts of the tree and focusses its attention there. Computers, if they are to compete, must somehow do the same.

This is where the neural network comes in. AlphaZero’s neural network receives, as input, the layout of the board for the last few moves of the game. As output, it estimates how likely the current player is to win and predicts which of the currently available moves are likely to work best. The M.C.T.S. algorithm uses these predictions to decide where to focus in the tree. If the network guesses that ‘knight-takes-bishop’ is likely to be a good move, for example, then the M.C.T.S. will devote more of its time to exploring the consequences of that move. But it balances this “exploitation” of promising moves with a little “exploration”: it sometimes picks moves it thinks are unlikely to bear fruit, just in case they do.

At first, the neural network guiding this search is fairly stupid: it makes its predictions more or less at random. As a result, the Monte Carlo Tree Search starts out doing a pretty bad job of focussing on the important parts of the tree. But the genius of AlphaZero is in how it learns. It takes these two half-working parts and has them hone each other. Even when a dumb neural network does a bad job of predicting which moves will work, it’s still useful to look ahead in the game tree: toward the end of the game, for instance, the M.C.T.S. can still learn which positions actually lead to victory, at least some of the time. This knowledge can then be used to improve the neural network. When a game is done, and you know the outcome, you look at what the neural network predicted for each position (say, that there’s an 80.2 per cent chance that castling is the best move) and compare that to what actually happened (say, that the percentage is more like 60.5); you can then “correct” your neural network by tuning its synaptic connections until it prefers winning moves. In essence, all of the M.C.T.S.’s searching is distilled into new weights for the neural network.

With a slightly better network, of course, the search gets slightly less misguided—and this allows it to search better, thereby extracting better information for training the network. On and on it goes, in a feedback loop that ratchets up, very quickly, toward the plateau of known ability.

When the AlphaGo Zero and AlphaZero papers were published, a small army of enthusiasts began describing the systems in blog posts and YouTube videos and building their own copycat versions. Most of this work was explanatory—it flowed from the amateur urge to learn and share that gave rise to the Web in the first place. But a couple of efforts also sprung up to replicate the work at a large scale. The DeepMind papers, after all, had merely described the greatest Go- and chess-playing programs in the world—they hadn’t contained the source code, and the company hadn’t made the programs themselves available to players. Having declared victory, its engineers had departed the field.

Gian-Carlo Pascutto, a computer programmer who works at the Mozilla Corporation, had a track record of building competitive game engines, first in chess, then in Go. He followed the latest research. As the combination of Monte Carlo Tree Search and a neural network became the state of the art in Go A.I.s, Pascutto built the world’s most successful open-source Go engines—first Leela, then LeelaZero—which mirrored the advances made by DeepMind. The trouble was that DeepMind had access to Google’s vast cloud and Pascutto didn’t. To train its Go engine, DeepMind used five thousand of Google’s “Tensor Processing Units”—chips specifically designed for neural-network calculations—for thirteen days. To do the same work on his desktop system, Pascutto would have to run it for seventeen hundred years.

To compensate for his lack of computing power, Pascutto distributed the effort. LeelaZero is a federated system: anyone who wants to participate can download the latest version, donate whatever computing power he has to it, and upload the data he generates so that the system can be slightly improved. The distributed LeelaZero community has had their system play more than ten million games against itself—a little more than AlphaGo Zero. It is now one of the strongest existing Go engines.

It wasn’t long before the idea was extended to chess. In December of last year, when the AlphaZero preprint was published, “it was like a bomb hit the community,” Gary Linscott said. Linscott, a computer scientist who had worked on Stockfish, used the existing LeelaZero code base, and the new ideas in the AlphaZero paper, to create Leela Chess Zero. (For Stockfish, he had developed a testing framework so that new ideas for the engine could be distributed to a fleet of volunteers, and thus vetted more quickly; distributing the training for a neural network was a natural next step.) There were kinks to sort out, and educated guesses to make about details that the DeepMind team had left out of their papers, but within a few months the neural network began improving. The chess world was already obsessed with AlphaZero: posts on chess.com celebrated the engine; commentators and grandmasters pored over the handful of AlphaZero games that DeepMind had released with their paper, declaring that this was “how chess ought to be played,” that the engine “plays like a human on fire.” Quickly, Lc0, as Leela Chess Zero became known, attracted hundreds of volunteers. As they contributed their computer power and improvements to the source code, the engine got even better. Today, one core contributor suspects that it is just a few months away from overtaking Stockfish. Not long after, it may become better than AlphaZero itself.

When we spoke over the phone, Linscott marvelled that a project like his, which would once have taken a talented doctoral student several years, could now be done by an interested amateur in a couple of months. Software libraries for neural networks allow for the replication of a world-beating design using only a few dozen lines of code; the tools already exist for distributing computation among a set of volunteers, and chipmakers such as Nvidia have put cheap and powerful G.P.U.s—graphics-processing chips, which are perfect for training neural networks—into the hands of millions of ordinary computer users. An algorithm like M.C.T.S. is simple enough to be implemented in an afternoon or two. You don’t even need to be an expert in the game for which you’re building an engine. When he built LeelaZero, Pascutto hadn’t played Go for about twenty years.

David Silver, the head of research at DeepMind, has pointed out a seeming paradox at the heart of his company’s recent work with games:  . . .

Continue reading.

Written by LeisureGuy

28 December 2018 at 11:10 am

Posted in Chess, Games, Go, Software, Technology

AlphaGolem

leave a comment »

In looking at the current state of AI and thinking of its future, John Henry and the steam drill come to mind. John Cornwell, director of the Science & Human Dimension Project at Jesus College, Cambridge, writes in Aeon:

The game of Go, which has a 3,000-year history in China, is played by two people on a board with counters, or stones, in black and white. The aim is to defeat one’s opponent by surrounding his territory. Metaphorically, the loser is choked into submission by the winner. At a match held in Seoul in South Korea, on 12 March 2016, the world Go champion Lee Sedol, observed by hundreds of supporters, and millions of spectators on television, slumped in apparent despair on being defeated by his opponent: a machine.

Go is a boardgame like no other. It is said to reflect the meaning of life. There are a prodigious number of potential moves – more, it is said, than all the particles in the known Universe. Serious Go players train virtually full-time from the age of five; they think of the game as an art form, and a philosophy, demanding the highest levels of intelligence, intuition and imagination. The champions are revered celebrities. They speak of the game as teaching them ‘an understanding of understanding’, and refer to original winning moves as ‘God’s touch.’

Lee’s face, as he lost the third of five games, and hence the match, was a picture of sorrow. It was as if he had failed the entire human race. He was beaten by AlphaGo, a machine that works on deeply layered neural nets that mimic the human brain and nervous system. The engineers and artificial intelligence (AI) experts who created AlphaGo admit that they do not understand how the machine’s intuition works. If melancholy is a consequence of loss, what was mourned that day was the demise of something uniquely special about human nature.

AlphaGo was designed at the AI research lab DeepMind, a subsidiary of the powerful Google corporation. DeepMind’s spokespeople say that this is just the beginning: they liken their research programmes to the Apollo Moon shot, or the Hubble telescope. The company has recruited 700 technicians, of whom 400 are post-doc computer scientists. They look ahead to the day when AI machines will be employed to solve the most impenetrable and recalcitrant problems in science, health, the environment … the Universe.

DeepMind scientists were thrilled with their success on 12 March 2016. Their glee recalled Dr Frankenstein’s – ‘it’s alive!… it’s alive!’ – in the 1931 movie directed by James Whale. Meanwhile, the emotions of Lee and his supporters bring to mind the pervasive atmosphere of melancholy in Mary Shelley’s novel; one commentator spoke of their ‘heavy sadness’. For his part, Lee had symbolically undergone the fate of Frankenstein’s brother William, throttled in the depths of a wood by the monster.

Cathartic foreboding is familiar in countless stories of hubris, from the original Prometheus myth to Frankenstein (1818) and on to the explosion of 20th- and 21st-century sci-fi literature and movies. But it is melancholy that haunts the imagined spectacle of humans rivalling God by devising creatures made in their own image. For Frankenstein’s monster, as for his creator Victor, the consciousness of having created a distorted human likeness lies at the heart of the unfolding misery and violence. ‘I am malicious because I am miserable,’ says the monster. ‘Am I not shunned and hated by all mankind?’ In the absence of any fatherly kindness from Frankenstein, his sorrow turns to hatred and murderous revenge: ‘If I cannot inspire love, I will cause fear.’

Alex Garland’s film Ex Machina (2014) is a recent manifestation of disastrous hubris in the creator-creature theme. It features Nathan, a billionaire genius AI scientist and contemporary Dr Frankenstein, who lives alone in a remote research facility where he constructs female robots. His latest artifact is Ava, a winsome AI android with suspected consciousness. Nathan wants to test her capacity for seduction. He recruits a young and impressionable computer scientist, Caleb, on the pretext of conducting a series of Turing tests: will Caleb mistake the machine for a human being? Will he fall in love with her? The answers, respectively, are no and yes.

Ava, for her part, manipulates Caleb for her own hidden, long-term aims. With the help of a fellow robot, she murders Nathan and escapes, leaving Caleb imprisoned and alone, facing starvation and death. Caleb elicits our contempt and pity. But Ava, despite her early expressions of frustrated longing (that suggest the sadness of a Lady of Shalott, ‘half sick of shadows’, but which are in fact a tactic of deceit) is a warped version of the prisoner who overcomes many obstacles to escape a Plato’s Cave of unreal androids. At the end of the film, Ava is helicoptered away from the facility to the world of real people. A sense of foreboding haunts the film from the outset, revealed in Nathan’s prognostication of AI’s future: ‘One day,’ he says, ‘the AIs will look back on us the same way we look at fossil skeletons from the plains of Africa. An upright ape, living in dust, with crude language and tools, all set for extinction.’

The enormity of AI’s challenge, and the melancholy it generates, was anticipated more than half a century ago by Norbert Wiener, the pioneer of cybernetics. Wiener was an atheist, yet in God and Golem, Inc(1964) he predicts a set of AI circumstances, theological and eschatological in their scope, with premonitions of dark physical and metaphysical risk. He laid down a principle that self-learning systems are capable, in theory, not only of unprogrammed learning, but of reproducing themselves and evolving. Crucially, they will relate in independent ways with human beings. Wiener believed the risks attendant on playing God were dramatically exemplified in the 17th-century legend of the Golem of Prague, a huge, conscious humanoid, made of clay and powered by cabbalistic magic to protect the Jews of the city. The Golem, named Josef, soon revealed its potential for calamity. When instructed to heave water, it could not stop its task, and flooded the house. (There are premonitions here of the seminar-room joke and thought-experiment, in which an AI machine is briefed to make paperclips and cannot be stopped: eventually it wrecks the infrastructure of the planet and destroys the human race.) The Golem turns against the very people it was intended to protect, and kills them.

Wiener also emphasised the ability of self-learning machines to play games. Every kind of relationship, he argues, is reducible to a game. He saw the Golem myth as a game, and he expands on the idea to suggest that the Book of Job, that most melancholy of biblical stories, is another archetypal game: God and Satan competing to win the soul of the suffering prophet. Similarly, Wiener sees the struggle between God and Satan in John Milton’s epic poem Paradise Lost (1667) as a celestial game: Satan the melancholic fallen arch-fiend, eternally stricken with wounded merit, competing with God for possession of humankind:

Abashed the devil stood,
And felt how awful goodness is, and saw
Virtue in her shape how lovely – saw, and pined
His loss.

And that game will one day be repeated, Wiener predicted, when a human being pits herself against the ultimate machine. Fifty years ahead of time, Wiener foretold that researchers would build a machine to defeat the human champion of the most difficult boardgame ever devised. But this would be just the prelude to much greater extensions of the machines’ prowess. Proposing a general principle in cybernetics, Wiener wrote: ‘a game-playing machine may be used to secure the automatic performance of any function if the performance of this function is subject to a clear-cut, objective criterion of merit’. By clear-cut, he meant definable in a finite number of words or matrices. The systems would, in time, engage in ‘war and business’ which are conflicts ‘and as such, they may be so formalised as to constitute games with definite rules’. He might have included environment, food security, development, diplomacy.

In his conclusion, Wiener speculated that formalised versions of complex human planning and decisions were already being established to ‘determine the policies for pressing the Great Push Button and burning the Earth clean for a new and less humanly undependable order of things’. He was alluding to the probability that the decision for nuclear war would be initiated by a self-learning machine. The notion of the automatic Doomsday Machine had been dramatised that same year in Stanley Kubrick’s film Dr Strangelove(1964). For all its mordant humour, the movie is profoundly dark, ultimately dominated by despair.

Go players speak of the top players’ special ‘imagination’, a talent or faculty that DeepMind’s designers also claim for AlphaGo. But in what sense can a machine possess imagination?

An early hint of AI ‘imagination’ and its rationale, can be found in a 2012 article published in Neuron, the journal of neurology: ‘The Future of Remembering: Memory, Imagining and the Brain’ is authored by a team led by the psychologist Daniel Schacter at Harvard University. The article was ostensibly about Alzheimer’s, and it argued that sufferers lose not only memory but the ability to envisage future events and their consequences. It claimed that imagination is key to both memory and forward-thinking.

Schacter and his colleagues cite the work of Sir Frederic Bartlett, professor of psychology at the University of Cambridge from the 1920s, to tell us what memory is not. In 1932, Bartlett claimed that memory ‘is not the re-excitation of innumerable fixed, lifeless and fragmentary traces, but an imaginative reconstruction or construction’. His research was based on an experiment whereby volunteers were told a Native American legend known as ‘The War of the Ghosts’. It takes about seven minutes to recite; the volunteers were then asked over lapses of days, weeks and months to retell the story. Bartlett found that the volunteers engaged their imaginations to recreate the tale in various ways, based on their own social and personal experiences. Memory, in other words, is not a retrieval of inert bits of information from a database, but a dynamic reconstruction or recreation: an exercise in imagination.

In their article, Schacter and his team argue that neuroscientific studies of imagination, memory, forward-thinking and decisionmaking have much to contribute to AI research. The significance of this statement, in retrospect at least, is the fact that one of the article’s authors was Demis Hassabis, then of University College, London. Hassabis had studied computer science at Cambridge, worked in the development of computer games (including the bestselling Theme Park) and gained a doctorate in cognitive neuroscience. He had been thinking hard about the direction of travel – from the brain to the machine. Certainly, he has said, since as early as 1997, it would be a strategy for his future research through the next two decades. In July 2017, as CEO and co-founder of DeepMind, he Tweeted: ‘Imagination is one of the keys to general intelligence, and also a powerful example of neuroscience-inspired ideas crossing over into AI.’

As Hassabis would explain on many occasions following the triumph of AlphaGo, the machine’s imagination consisted in its capacity to model future scenarios and the consequences of those scenarios at prodigious speeds and across a broad span of combinations, including its opponent’s potential moves. Furthermore, the operation of the neural nets meant that its ‘imagination’ was dynamic, productive, not inert and passive.

The significance of machines mimicking the biological action of the brain and nervous system, as Hassabis framed it, was a metaphorical reversal of the more familiar direction of travel. Before the great leap forward in noninvasive brain imaging through the 1980s and ’90s (the so-called Decade of the Brain), it had been routine, from the early modern period on, to invoke machines to explain the mind-brain function: think of . . .

Continue reading.

Written by LeisureGuy

14 November 2018 at 9:30 am

Two excellent movies on Netflix

leave a comment »

First, The Surrounding Game, which explores the appeal of Go (aka Baduk (Korea) and Weichi (China)).

Second, The Accountant, a good action movie.

Written by LeisureGuy

30 August 2018 at 4:24 pm

Posted in Games, Go, Movies & TV

I found this commentary on AlphaGo Zero v. AlphaGo Master fascinating

leave a comment »

I am fascinated by the analysis and discoveries.

Written by LeisureGuy

7 July 2018 at 1:48 pm

Posted in Games, Go, Technology, Video

Twitch played Go

leave a comment »

Pretty cool. Take a look.

Written by LeisureGuy

1 May 2018 at 4:01 pm

Posted in Go

“The Surrounding Game” now available on iTunes, Amazon Video, and YouTube

leave a comment »

I highly recommend the documentary. It live on  iTunes, Amazon Video, and YouTube!

It’s a terrific documentary. The first third is sort of a history and high points of Go, which provides a context for the viewer who doesn’t play Go, and then the documentary really hits its stride and gets better and better.

Here’s the trailer:

 

Written by LeisureGuy

15 March 2018 at 10:48 am

Posted in Games, Go, Movies & TV

Cool Go game: Player thinks aloud in a live game against an AI

leave a comment »

Very instructive.

Written by LeisureGuy

5 March 2018 at 4:52 pm

Posted in Games, Go, Video

A really stunning documentary, especially if you don’t play Go: “The Surrounding Game”

leave a comment »

It’s about Go, but I mean really about Go. The reason that it’s good for those who don’t know Go is that there’s an initial section providing information and context for the last part of the film, which is the greater part of the film, in every sense. It was as tense as you could want. And it just got better and better. Man. I’m impressed.

The Surrounding Game

Written by LeisureGuy

3 March 2018 at 8:15 pm

Posted in Games, Go, Movies & TV

No Children Because of Climate Change? Some People Are Considering It

leave a comment »

I have to say that I believe that my grandchildren will face a challenging future. We haven’t yet seen the mass migrations of people that will happen as coastal cities are flooded more than they already are, and we have seen that refugees are not treated well. Maggie Astor reports in the NY Times:

Add this to the list of decisions affected by climate change: Should I have children?

It is not an easy time for people to feel hopeful, with the effects of globalwarming no longer theoretical, projections becoming more dire and governmental action lagging. And while few, if any, studies have examined how large a role climate change plays in people’s childbearing decisions, it loomed large in interviews with more than a dozen people ages 18 to 43.

A 32-year-old who always thought she would have children can no longer justify it to herself. A Mormon has bucked the expectations of her religion by resolving to adopt rather than give birth. An Ohio woman had her first child after an unplanned pregnancy — and then had a second because she did not want her daughter to face an environmental collapse alone.

Among them, there is a sense of being saddled with painful ethical questions that previous generations did not have to confront. Some worry about the quality of life children born today will have as shorelines floodwildfires rage and extreme weather becomes more common. Others are acutely aware that having a child is one of the costliest actions they can take environmentally.

The birthrate in the United States, which has been falling for a decade, reached a new low in 2016. Economic insecurity has been a major factor, but even as the economy recovers, the decline in births continues.

And the discussions about the role of climate change are only intensifying.

“When we first started this project, I didn’t know anybody who had had any conversations about this,” said Meghan Kallman, a co-founder of Conceivable Future, an organization that highlights how climate change is limiting reproductive choices.

That has changed, she said — either because more people are having doubts, or because it has become less taboo to talk about them.

Facing an uncertain future

If it weren’t for climate change, Allison Guy said, she would go off birth control tomorrow.

But scientists’ projections, if rapid action isn’t taken, are not “congruent with a stable society,” said Ms. Guy, 32, who works at a marine conservation nonprofit in Washington. “I don’t want to give birth to a kid wondering if it’s going to live in some kind of ‘Mad Max’ dystopia.”

Parents like Amanda PerryMiller, a Christian youth leader and mother of two in Independence, Ohio, share her fears.

“Animals are disappearing. The oceans are full of plastic. The human population is so numerous, the planet may not be able to support it indefinitely,” said Ms. PerryMiller, 29. “This doesn’t paint a very pretty picture for people bringing home a brand-new baby from the hospital.”

The people thinking about these issues fit no single profile. They are women and men, liberal and conservative. They come from many regions and religions. . .

Continue reading.

Written by LeisureGuy

5 February 2018 at 11:08 am

Posted in Daily life, Education, Go

AlphaGo Zero: Learning from scratch

leave a comment »

A very good (and very interesting) post at DeepMind.com:

Artificial intelligence research has made rapid progress in a wide variety of domains from speech recognition and image classification to genomics and drug discovery. In many cases, these are specialist systems that leverage enormous amounts of human expertise and data.

However, for some problems this human knowledge may be too expensive, too unreliable or simply unavailable. As a result, a long-standing ambition of AI research is to bypass this step, creating algorithms that achieve superhuman performance in the most challenging domains with no human input. In our most recent paper, published in the journal Nature, we demonstrate a significant step towards this goal.

The paper introduces AlphaGo Zero, the latest evolution of AlphaGo, the first computer program to defeat a world champion at the ancient Chinese game of Go. Zero is even more powerful and is arguably the strongest Go player in history.

Previous versions of AlphaGo initially trained on thousands of human amateur and professional games to learn how to play Go. AlphaGo Zero skips this step and learns to play simply by playing games against itself, starting from completely random play. In doing so, it quickly surpassed human level of play and defeated the previously published champion-defeating version of AlphaGo by 100 games to 0.

It is able to do this by using a novel form of reinforcement learning, in which AlphaGo Zero becomes its own teacher. The system starts off with a neural network that knows nothing about the game of Go. It then plays games against itself, by combining this neural network with a powerful search algorithm. As it plays, the neural network is tuned and updated to predict moves, as well as the eventual winner of the games.

This updated neural network is then recombined with the search algorithm to create a new, stronger version of AlphaGo Zero, and the process begins again. In each iteration, the performance of the system improves by a small amount, and the quality of the self-play games increases, leading to more and more accurate neural networks and ever stronger versions of AlphaGo Zero.

This technique is more powerful than previous versions of AlphaGo because it is no longer constrained by the limits of human knowledge. Instead, it is able to learn tabula rasa from the strongest player in the world: AlphaGo itself.

It also differs from previous versions in other notable ways.

  • AlphaGo Zero only uses the black and white stones from the Go board as its input, whereas previous versions of AlphaGo included a small number of hand-engineered features.
  • It uses one neural network rather than two. Earlier versions of AlphaGo used a “policy network” to select the next move to play and a ”value network” to predict the winner of the game from each position. These are combined in AlphaGo Zero, allowing it to be trained and evaluated more efficiently.
  • AlphaGo Zero does not use “rollouts” – fast, random games used by other Go programs to predict which player will win from the current board position. Instead, it relies on its high quality neural networks to evaluate positions.

All of these differences help improve the performance of the system and make it more general. But it is the algorithmic change that makes the system much more powerful and efficient.

After just three days of self-play training, AlphaGo Zero emphatically defeated the previously published version of AlphaGo – which had itself defeated 18-time world champion Lee Sedol – by 100 games to 0. After 40 days of self training, AlphaGo Zero became even stronger, outperforming the version of AlphaGo known as “Master”, which has defeated the world’s best players and world number one Ke Jie. . .

Continue reading.

Written by LeisureGuy

22 October 2017 at 2:14 pm

Posted in Go, Software, Technology

One more step toward the Singularity: Artificial Intelligence Learns to Learn Entirely on Its Own

leave a comment »

In science-fiction about the Singularity, the point at which AI could instruct itself was a big step, leading to a rapid increase in capabilities as a result of positive feedback. Reality is not, of course, science-fiction, but the report of AI instructing itself does catch the eye.

Kevin Hartnett writes in Quanta:

A mere 19 months after dethroning the world’s top human Go player, the computer program AlphaGo has smashed an even more momentous barrier: It can now achieve unprecedented levels of mastery purely by teaching itself. Starting with zero knowledge of Go strategy and no training by humans, the new iteration of the program, called AlphaGo Zero, needed just three days to invent advanced strategies undiscovered by human players in the multi-millennia history of the game. By freeing artificial intelligence from a dependence on human knowledge, the breakthrough removes a primary limit on how smart machines can become.

Earlier versions of AlphaGo were taught to play the game using two methods. In the first, called supervised learning, researchers fed the program 100,000 top amateur Go games and taught it to imitate what it saw. In the second, called reinforcement learning, they had the program play itself and learn from the results.

AlphaGo Zero skipped the first step. The program began as a blank slate, knowing only the rules of Go, and played games against itself. At first, it placed stones randomly on the board. Over time it got better at evaluating board positions and identifying advantageous moves. It also learned many of the canonical elements of Go strategy and discovered new strategies all its own. “When you learn to imitate humans the best you can do is learn to imitate humans,” said Satinder Singh, a computer scientist at the University of Michigan who was not involved with the research. “In many complex situations there are new insights you’ll never discover.”

After three days of training and 4.9 million training games, the researchers matched AlphaGo Zero against the earlier champion-beating version of the program. AlphaGo Zero won 100 games to zero.

To expert observers, the rout was stunning. Pure reinforcement learning would seem to be no match for the overwhelming number of possibilities in Go, which is vastly more complex than chess: You’d have expected AlphaGo Zero to spend forever searching blindly for a decent strategy. Instead, it rapidly found its way to superhuman abilities.

The efficiency of the learning process owes to a feedback loop. Like its predecessor, AlphaGo Zero determines what move to play through a process called a “tree search.” The program starts with the current board and considers the possible moves. It then considers what moves its opponent could play in each of the resulting boards, and then the moves it could play in response and so on, creating a branching tree diagram that simulates different combinations of play resulting in different board setups.

AlphaGo Zero can’t follow every branch of the tree all the way through, since that would require inordinate computing power. Instead, it selectively prunes branches by deciding which paths seem most promising. It makes that calculation — of which paths to prune — based on what it has learned in earlier play about the moves and overall board setups that lead to wins.

Earlier versions of AlphaGo did all this, too. What’s novel about AlphaGo Zero is that instead of just running the tree search and making a move, it remembers the outcome of the tree search — and eventually of the game. It then uses that information to update its estimates of promising moves and the probability of winning from different positions. As a result, the next time it runs the tree search it can use its improved estimates, trained with the results of previous tree searches, to generate even better estimates of the best possible move.

The computational strategy that underlies AlphaGo Zero is effective primarily in situations in which you have an extremely large number of possibilities and want to find the optimal one. In the Nature paper describing the research, the authors of AlphaGo Zero suggest that their system could be useful in materials exploration — where you want to identify atomic combinations that yield materials with different properties — and protein folding, where you want to understand how a protein’s precise three-dimensional structure determines its function.

As for Go,  . . .

Continue reading.

Written by LeisureGuy

18 October 2017 at 4:12 pm

Posted in Go, Software, Technology

n the Age of Google DeepMind, Do the Young Go Prodigies of Asia Have a Future?

leave a comment »

Dawn Chan in the New Yorker:

Choong-am Dojang is far from a typical Korean school. Its best pupils will never study history or math, nor will they receive traditional high-school diplomas. The academy, which operates above a bowling alley on a narrow street in northwestern Seoul, teaches only one subject: the game of Go, known in Korean as baduk and in Chinese as wei qi. Each day, Choong-am’s students arrive at nine in the morning, find places at desks in a fluorescent-lit room, and play, study, memorize, and review games—with breaks for cafeteria meals or an occasional soccer match—until nine at night.
Choong-am, which is the product of a merger between four top Go academies, is currently the biggest of a handful of _dojang_s in South Korea. Many of the students enrolled in these schools have been training since they were four or five, perhaps playing informally at first but later growing obsessed with the game’s beauty and the competitiveness and camaraderie that surround it. (Indeed, the word “dojang” more commonly refers to a martial-arts academy.) Lee Hajin, the secretary-general of the International Go Federation, told me that she left home when she was nine. With only her clothes and a stuffed-toy poodle backpack that her parents gave her for Christmas, she moved across the country, into the home of a Go master and his wife.
The aim of all serious Go pupils is ultimately to be designated a professional. This makes them eligible to compete in Asia’s pro tournaments, which are broadcast on TV and sponsored by companies such as Samsung, LG, and the instant-noodle maker Nongshim. At the highest-level tournaments, first-place winners can win as much as three hundred thousand dollars. But the competition is fierce. It is estimated that, of South Korea’s three hundred and twenty pros, only around fifty are able to earn a living on tournament winnings. Sometimes, after losing an especially important match, players joke about drowning themselves in the Han River. Lee Hajin recalls having such bad insomnia before important games that her teacher’s wife would bring her a shot of whiskey, diluted in a cup of water, to help her fall asleep.
Go itself is simple in design but complex in its possible outcomes: two players, one using white stones and the other black, take turns placing their pieces on a square board, capturing territory and boxing each other out. If a child dedicating her life to such a game seems unfathomable elsewhere in the world, it makes more sense in East Asia, where Go has a twenty-five-hundred-year cultural history. Through the centuries, princes, generals, monks, and farmers have played the game, not only to win but to build character and develop mental acumen. “It’s also psychology, philosophy—it’s art,” Fan Hui, the reigning European Go champion, told me. In Tang-dynasty China, the game was considered one of the four arts that a cultivated gentleman ought to master, along with calligraphy, painting, and playing the lute. So many East Asian leaders have studied it that political scientists are wont to identify traces of Go strategy in the continent’s real-world conflicts. Henry Kissinger, for instance, argued that during the Taiwan Strait crisis of the nineteen-fifties, “both sides were playing by wei qi rules.” Today, Seoul’s Myongji University even offers degrees in Go studies. According to Daniela Trinks, a professor in the department, one in four Koreans knows how to play the game.
But recent events could pose a threat to Go’s cultural supremacy. Earlier this week, one of the world’s top players, Lee Sedol, lost two high-profile matches—the first of a planned five—to AlphaGo, an artificial-intelligence program created by Google DeepMind. The same program beat Fan Hui, 5–0, back in October. Until then, Go had been considered the only popular two-player board game that humans would continue to dominate for the foreseeable future, its array of outcomes still too dizzyingly vast for even increasingly smart machines to pick out the best moves. That, of course, has now changed. Even if Lee miraculously comes back to win his remaining three games, the first of which takes place on Saturday, in Seoul, AlphaGo promises to grow even more formidable. (“If there’s a limit to improvement, we haven’t hit it yet,” Demis Hassabis, DeepMind’s founder and C.E.O., told me.) What’s notable, too, is how quickly AlphaGo improves compared with humans. The program lost two quick, unofficial matches with Fan Hui that were scheduled between longer, official ones, which the computer won. Five months later, it is capable of defeating Lee, who is ranked far higher than Fan. According to Ben Lockhart, one of the best amateur Go players born outside East Asia, Fan “could have trained his whole life and would never have gotten close to where Lee Sedol is.”
Lockhart, as it happens, is the lone American pupil currently enrolled at Choong-am. He is an anomaly at the dojang, not just because he is a foreigner but also because he has memories of a life without intensive Go. When he was in high school, in Brooklyn, playing the game but also “smoking a lot of weed and listening to Noam Chomsky in Prospect Park,” his peers in Seoul were already deep into their training regimens. Now, however, Lockhart is more disciplined. Last Friday, he began his morning by trying to make progress through a book of six hundred Go problems. These exercise books are a common component of Go pedagogy, as are actual matches and occasional lectures by professionals. Students sometimes memorize parts of games, or even whole games, from the canon. They also practice specific skills, such as “reading,” or peering into the future at branching paths of possibility—an activity that’s not dissimilar to the so-called tree-search components of AlphaGo and many other game-playing A.I.s.
In the long course of their training, students may play upwards of ten thousand games, developing intuitions about which moves tend to work out well and which don’t. AlphaGo, analogously, improves by playing itself, with the added advantage that it can flit through games quickly, while humans take time to think and place stones on a board. (In January, the DeepMind team published a paper in Nature noting that one of AlphaGo’s neural networks had played more than a million games in a single day.) But there is one particularly interesting difference between a dojang’s pedagogical program and AlphaGo’s: human students receive active guidance from teachers, who can draw attention to specific mistakes or suggest generalized patterns to seek out or avoid. According to DeepMind’s most recent account, although AlphaGo’s learning is shaped by observations of expert human games, it doesn’t receive targeted advice from any outsiders.
Although some Go players are eager to see whether computers will unlock undiscovered moves and strategies, others seem despondent. . .

Continue reading.

Written by LeisureGuy

9 August 2017 at 3:26 pm

Someone Is Destroying Online Go, And Nobody Knows Who It Is

leave a comment »

Only, as the article explains, now they do. Alex Walker reports on Kotaku.com:

Right now, there’s a player lurking in the depths of the online Go scene that is laying waste to some of the best players in the world. It’s called Master, and nobody knows who it is.

Update: The identity of the mystery account has finally been revealed – you can read all about it here.

The account is simply called “Master”, and since the start of the new year it has made a habit out of trashing some of the world’s best Go professionals. It’s already beaten Ke Jie twice, who is currently the highest ranked Go player in the world. AlphaGo, incidentally, is #2.

Not that the ranking stopped him from being battered, mind you. A European professional Go player, Ali Jabarin, wrote on Facebook that Ke Jie was “a bit shocked … just repeating ‘it’s too strong'”. Jabarin wasn’t sure whether the player was AlphaGo or not, but he was certain that an AI was behind the mystery account.

By January 3, the number of probably-but-we-can’t-officially-say AI sanctioned beatings had risen to 41-zip. There’s a few signs that it might not be an all-AI account, though. Jabarin received a polite message on New Year’s declining a match, and a post appeared offering around $US14,000 to any professional player who could beat it. . .

Continue reading.

More at the link, including one of the games Master played.

Written by LeisureGuy

7 January 2017 at 5:27 pm

Posted in Games, Go, Technology

Move 37 vs. Move 78: A commentary on Go-playing AI

leave a comment »

Very interesting article by Cade Metz in Wired.

Written by LeisureGuy

21 May 2016 at 5:09 pm

Posted in Games, Go, Technology

AlphaGo and Deep Blue: Two very different approaches

leave a comment »

Michael Nielsen writes in Quanta:

In 1997, IBM’s Deep Blue system defeated the world chess champion, Garry Kasparov. At the time, the victory was widely described as a milestone in artificial intelligence. But Deep Blue’s technology turned out to be useful for chess and not much else. Computer science did not undergo a revolution.

Will AlphaGo, the Go-playing system that recently defeated one of the strongest Go players in history, be any different?

I believe the answer is yes, but not for the reasons you may have heard. Many articles proffer expert testimony that Go is harder than chess, making this victory more impressive. Or they say that we didn’t expect computers to win at Go for another 10 years, so this is a bigger breakthrough. Some articles offer the (correct!) observation that there are more potential positions in Go than in chess, but they don’t explain why this should cause more difficulty for computers than for humans.

In other words, these arguments don’t address the core question: Will the technical advances that led to AlphaGo’s success have broader implications? To answer this question, we must first understand the ways in which the advances that led to AlphaGo are qualitatively different and more important than those that led to Deep Blue.

In chess, beginning players are taught a notion of a chess piece’s value. In one system, a knight or bishop is worth three pawns. A rook, which has greater range of movement, is worth five pawns. And the queen, which has the greatest range of all, is worth nine pawns. A king has infinite value, since losing it means losing the game.

You can use these values to assess potential moves. Give up a bishop to take your opponent’s rook? That’s usually a good idea. Give up a knight and a bishop in exchange for a rook? Not such a good idea.

The notion of value is crucial in computer chess. Most computer chess programs search through millions or billions of combinations of moves and countermoves. The goal is for the program to find a sequence of moves that maximizes the final value of the program’s board position, no matter what sequence of moves is played by the opponent.

Early chess programs evaluated board positions using simple notions like “one bishop equals three pawns.” But later programs used more detailed chess knowledge. Deep Blue, for example, combined more than 8,000 different factors in the function it used to evaluate board positions. Deep Blue didn’t just say that one rook equals five pawns. If a pawn of the same color is ahead of the rook, the pawn will restrict the rook’s range of movement, thus making the rook a little less valuable. If, however, the pawn is “levered,” meaning that it can move out of the rook’s way by capturing an enemy pawn, Deep Blue considers the pawn semitransparent and doesn’t reduce the rook’s value as much.

Ideas like this depend on detailed knowledge of chess and were crucial to Deep Blue’s success. According to the technical paper written by the Deep Blue team, this notion of a semitransparent levered pawn was crucial to Deep Blue’s play in the second game against Kasparov.

Ultimately, the Deep Blue developers used two main ideas. The first was to build a function that incorporated lots of detailed chess knowledge to evaluate any given board position. The second was to use immense computing power to evaluate lots of possible positions, picking out the move that would force the best possible final board position.

What happens if you apply this strategy to Go?

It turns out that you will run into a difficult problem when you try. The problem lies in figuring out how to evaluate board positions. Top Go players use a lot of intuition in judging how good a particular board position is. They will, for instance, make vague-sounding statements about a board position having “good shape.” And it’s not immediately clear how to express this intuition in simple, well-defined systems like the valuation of chess pieces.

Now you might think it’s just a question of working hard and coming up with a good way of evaluating board positions. Unfortunately, even after decades of attempts to do this using conventional approaches, there was still no obvious way to apply the search strategy that was so successful for chess, and Go programs remained disappointing. This began to change in 2006, with the introduction of so-called Monte Carlo tree search algorithms, which tried a new approach to evaluation based on a clever way of randomly simulating games. But Go programs still fell far short of human players in ability. It seemed as though a strong intuitive sense of board position was essential to success.

What’s new and important about AlphaGo is that its developers have figured out a way of bottling something very like that intuitive sense. . .

Continue reading.

Written by LeisureGuy

29 March 2016 at 12:31 pm

Posted in Games, Go, Software, Technology

Wow! Computer running AlphaGo program defeats a professional Go player!

leave a comment »

The video is from the post “AlphaGo: Mastering the ancient game of Go with Machine Learning” on the Google Research blog.

David Silver and Demis Hassabis, of Google DeepMind project, write:

Games are a great testing ground for developing smarter, more flexible algorithms that have the ability to tackle problems in ways similar to humans. Creating programs that are able to play games better than the best humans has a long history – the first classic game mastered by a computer was noughts and crosses (also known as tic-tac-toe) in 1952 as a PhD candidate’s project. Then fell checkers in 1994. Chess was tackled by Deep Blue in 1997. The success isn’t limited to board games, either – IBM’s Watson won first place on Jeopardy in 2011, and in 2014 our own algorithms learned to play dozens of Atari games just from the raw pixel inputs.

But one game has thwarted A.I. research thus far: the ancient game of Go. Invented in China over 2500 years ago, Go is played by more than 40 million people worldwide. The rules are simple: players take turns to place black or white stones on a board, trying to capture the opponent’s stones or surround empty space to make points of territory. Confucius wrote about the game, and its aesthetic beauty elevated it to one of the four essential arts required of any true Chinese scholar. The game is played primarily through intuition and feel, and because of its subtlety and intellectual depth it has captured the human imagination for centuries.

But as simple as the rules are, Go is a game of profound complexity. The search space in Go is vast — more than a googol times larger than chess (a number greater than there are atoms in the universe!). As a result, traditional “brute force” AI methods — which construct a search tree over all possible sequences of moves — don’t have a chance in Go. To date, computers have played Go only as well as amateurs. Experts predicted it would be at least another 10 years until a computer could beat one of the world’s elite group of Go professionals.

We saw this as an irresistible challenge! We started building a system, AlphaGo, described in a paper in Nature this week, that would overcome these barriers. The key to AlphaGo is reducing the enormous search space to something more manageable. To do this, it combines a state-of-the-art tree search with two deep neural networks, each of which contains many layers with millions of neuron-like connections. One neural network, the “policy network”, predicts the next move, and is used to narrow the search to consider only the moves most likely to lead to a win. The other neural network, the “value network”, is then used to reduce the depth of the search tree — estimating the winner in each position in place of searching all the way to the end of the game.

AlphaGo’s search algorithm is much more human-like than previous approaches. For example, when Deep Blue played chess, it searched by brute force over thousands of times more positions than AlphaGo. Instead, AlphaGo looks ahead by playing out the remainder of the game in its imagination, many times over – a technique known as Monte-Carlo tree search. But unlike previous Monte-Carlo programs, AlphaGo uses deep neural networks to guide its search. During each simulated game, the policy network suggests intelligent moves to play, while the value network astutely evaluates the position that is reached. Finally, AlphaGo chooses the move that is most successful in simulation.

We first trained the policy network on 30 million moves from games played by human experts, until it could predict the human move 57% of the time (the previous record before AlphaGo was 44%). But our goal is to beat the best human players, not just mimic them. To do this, AlphaGo learned to discover new strategies for itself, by playing thousands of games between its neural networks, and gradually improving them using a trial-and-error process known as reinforcement learning. This approach led to much better policy networks, so strong in fact that the raw neural network (immediately, without any tree search at all) can defeat state-of-the-art Go programs that build enormous search trees.

These policy networks were in turn used to train the value networks, again by reinforcement learning from games of self-play. These value networks can evaluate any Go position and estimate the eventual winner – a problem so hard it was believed to be impossible.

Of course, all of this requires a huge amount of compute power, so we made extensive use ofGoogle Cloud Platform, which enables researchers working on AI and Machine Learning to access elastic compute, storage and networking capacity on demand. In addition, new open source libraries for numerical computation using data flow graphs, such as TensorFlow, allow researchers to efficiently deploy the computation needed for deep learning algorithms across multiple CPUs or GPUs.

So how strong is AlphaGo? To answer this question, we played a tournament between AlphaGo and the best of the rest – the top Go programs at the forefront of A.I. research. Using a single machine, AlphaGo won all but one of its 500 games against these programs. In fact, AlphaGo even beat those programs after giving them 4 free moves headstart [i.e., a four-stone handicap, roughly equivalent to Queen odds in chess – LG] at the beginning of each game. A high-performance version of AlphaGo, distributed across many machines, was even stronger. . .

Continue reading.

Here’s the paper (“Mastering the game of Go with deep neural networks and tree search”)
in Nature. The abstract:

The game of Go has long been viewed as the most challenging of classic games for artificial intelligence owing to its enormous search space and the difficulty of evaluating board positions and moves. Here we introduce a new approach to computer Go that uses ‘value networks’ to evaluate board positions and ‘policy networks’ to select moves. These deep neural networks are trained by a novel combination of supervised learning from human expert games, and reinforcement learning from games of self-play. Without any lookahead search, the neural networks play Go at the level of state-of-the-art Monte Carlo tree search programs that simulate thousands of random games of self-play. We also introduce a new search algorithm that combines Monte Carlo simulation with value and policy networks. Using this search algorithm, our program AlphaGo achieved a 99.8% winning rate against other Go programs, and defeated the human European Go champion by 5 games to 0. This is the first time that a computer program has defeated a human professional player in the full-sized game of Go, a feat previously thought to be at least a decade away.

Written by LeisureGuy

27 January 2016 at 12:26 pm

Posted in Games, Go, Software, Technology

%d bloggers like this: