Later On

A blog written for those whose interests more or less match mine.

Archive for the ‘Software’ Category

A different way of viewing ballet

leave a comment »

Via this article in Aeon:

Written by LeisureGuy

17 October 2017 at 9:42 am

Posted in Art, Software, Technology, Video

The World Once Laughed at North Korean Cyberpower. No More.

leave a comment »

David Sanger, David Kirkpatrick, and Nicole Perlroth report in the NY Times:

When North Korean hackers tried to steal $1 billion from the New York Federal Reserve last year, only a spelling error stopped them. They were digitally looting an account of the Bangladesh Central Bank, when bankers grew suspicious about a withdrawal request that had misspelled “foundation” as “fandation.”

Even so, Kim Jong-un’s minions still got away with $81 million in that heist.

Then only sheer luck enabled a 22-year-old British hacker to defuse the biggest North Korean cyberattackto date, ransomware attack last Maythat failed to generate much cash but brought down hundreds of thousands of computers across dozens of countries — and briefly crippled Britain’s National Health Service.

Their track record is mixed, but North Korea’s army of more than 6,000 hackers is undeniably persistent, and undeniably improving, according to American and British security officials who have traced these attacks and others back to the North.

Amid all the attention on Pyongyang’s progress in developing a nuclear weapon capable of striking the continental United States, the North Koreans have also quietly developed a cyberprogram that is stealing hundreds of millions of dollars and proving capable of unleashing global havoc.

Unlike its weapons tests, which have led to international sanctions, the North’s cyberstrikes have faced almost no pushback or punishment, even as the regime is already using its hackingcapabilities for actual attacks against its adversaries in the West.

And just as Western analysts once scoffed at the potential of the North’s nuclear program, so did experts dismiss its cyberpotential — only to now acknowledge that hacking is an almost perfect weapon for a Pyongyang that is isolated and has little to lose.

The country’s primitive infrastructure is far less vulnerable to cyberretaliation, and North Korean hackers operate outside the country, anyway. Sanctions offer no useful response, since a raft of sanctions are already imposed. And Mr. Kim’s advisers are betting that no one will respond to a cyberattack with a military attack, for fear of a catastrophic escalation between North and South Korea. . .

Continue reading.

Written by LeisureGuy

15 October 2017 at 3:28 pm

The Real Threat from Kaspersky Security Software

leave a comment »

Herb Lin writes in Lawfare:

The Washington Post and the Wall Street Journal report that Russian government hackers obtained details of U.S. cyber capabilities from the personal computer of a National Security Agency employee who had taken classified material home. He was running Kaspersky antivirus software. Apparently, the compromised secrets could enable the Russian government to thwart U.S. cyber operations, both defensive and offensive.

News reports regarding this story have understandably focused on the damage to U.S. cyber capabilities. I have no particular inside knowledge of the specific information leaked to the Russians, but if these reports are true, the compromise was particularly severe. However, as concerned as I am about the compromised information, I observe that such information is often of transient value to an adversary, or at least should be treated that way.

Of more concern to me is the idea that Kaspersky software has the capability to inspect the media of any computer running it for interesting files and to forward such files to Russian intelligence. This raises at least two groups of questions.

First, what is the nature of the algorithm that searches stored files on my computer? For example, does it look for documents that have the phrase “Top Secret” on them? Does it seek to decrypt my encrypted files? Does it go after my deleted files? Does it do keyword searches for documents containing the word “nuclear”? Is it looking for pornography stored on my computer so the Russians can blackmail me? Reading my email? And so on.

Second, how widely deployed is Kaspersky software on non-U.S.-government computers? This includes personal computers of U.S. government employees, of course, but also the work and/or personal computers of many in the private sector. What kinds of information have been taken from those computers? And what is the potential for mischief or malfeasance with that information being compromised?

Taken together, these questions speak to an even more serious compromise: the fact that the Russians are able to mine and are mining the documents, one by one, on the computers of every single Kasperksy user. Kaspersky software is used by 400 million individuals and is the most popular European security software vendor. I suspect the information derived from that scale of operation is much more significant than what they got from one user, important though he may be.

Lastly, no public information has been revealed about what Kaspersky anti-virus software actually does once installed, despite the fact that . . .

Continue reading.

Written by LeisureGuy

12 October 2017 at 8:42 am

How Israel Caught Russian Hackers Scouring the World for U.S. Secrets

leave a comment »

Nicole Pearlroth and Scott Shane report in the NY Times:

It was a case of spies watching spies watching spies: Israeli intelligence officers looked on in real time as Russian government hackers searched computers around the world for the code names of American intelligence programs.

What gave the Russian hacking, detected more than two years ago, such global reach was its improvised search tool — antivirus software made by a Russian company, Kaspersky Lab, that is used by 400 million people worldwide, including by officials at some two dozen American government agencies.

The Israeli officials who had hacked into Kaspersky’s own network alerted the United States to the broad Russian intrusion, which has not been previously reported, leading to a decision just last month to order Kaspersky software removed from government computers.

The Russian operation, described by multiple people who have been briefed on the matter, is known to have stolen classified documents from a National Security Agency employee who had improperly stored them on his home computer, on which Kaspersky’s antivirus software was installed. What additional American secrets the Russian hackers may have gleaned from multiple agencies, by turning the Kaspersky software into a sort of Google search for sensitive information, is not yet publicly known.

The current and former government officials who described the episode spoke about it on condition of anonymity because of classification rules.

Like most security software, Kaspersky Lab’s products require access to everything stored on a computer in order to scour it for viruses or other dangers. Its popular antivirus software scans for signatures of malicious software, or malware, then removes or neuters it before sending a report back to Kaspersky. That procedure, routine for such software, provided a perfect tool for Russian intelligence to exploit to survey the contents of computers and retrieve whatever they found of interest.

The National Security Agency and the White House declined to comment for this article. The Israeli Embassy declined to comment, and the Russian Embassy did not respond to requests for comment.

The Wall Street Journal reported last week that Russian hackers had stolen classified N.S.A. materials from a contractor using the Kaspersky software on his home computer. But the role of Israeli intelligence in uncovering that breach and the Russian hackers’ use of Kaspersky software in the broader search for American secrets have not previously been disclosed.

Kaspersky Lab denied any knowledge of, or involvement in, the Russian hacking. “Kaspersky Lab has never helped, nor will help, any government in the world with its cyberespionage efforts,” the company said in a statement Tuesday afternoon. Kaspersky Lab also said it “respectfully requests any relevant, verifiable information that would enable the company to begin an investigation at the earliest opportunity.”

The Kaspersky-related breach is only the latest bad news for the security of American intelligence secrets. It does not appear to be related to a devastating leak of N.S.A. hacking tools last year to a group, still unidentified, calling itself the Shadow Brokers, which has placed many of them online. Nor is it evidently connected to a parallel leak of hacking data from the C.I.A. to WikiLeaks, which has posted classified C.I.A. documents regularly under the name Vault7.

For years, there has been speculation that Kaspersky’s popular antivirus software might provide a back door for Russian intelligence. . .

Continue reading.

Written by LeisureGuy

10 October 2017 at 7:31 pm

Fascinating application: Teaching a Machine What Members of Congress Care About

leave a comment »

Jeremy Merrill reports in ProPublica:

If you asked congressional experts what legislative subjects, say, Sen. Patty Murray of Washington specializes in, they’d have a few pretty good guesses: maybe education and health care — because she’s the ranking member on a key committee that oversees those issues. If you asked who else in the Senate shares her interests, you might hear Sen. Michael Bennet of Colorado. Why? Because he is a former school superintendent and a member on that same committee.

You could ask them the same question about more members of Congress, but before you got through all 535 lawmakers, they’d probably hang up on you.

But what if we could teach a computer what specific topics are distinctive to each member? We did just that. We trained a computer model to extract what phrases a Congress member uses more than the rest, using hundreds of thousands of press releases from 2015 to the present.

We hope this addition to Represent’s member pages will give constituents new insight into what the people who work in their names specialize in, whether it’s hot-button national issues or local happenings.

Many of the results are intuitive: Rep. Jared Polis, a Democratic representative from Colorado who is known as a civil libertarian, has “email privacy” as a topic; the model also knows Sen. Mitch McConnell, the Kentucky Republican, talks often about “coal miners.”

But the model’s strength is not in making obvious observations, but spotting things others might not. The model has picked up on New Jersey Democrat Rep. Josh Gottheimer’s use of the phrase “moocher states,” for example, a phrase more closely associated with libertarian groups than his own party. And the model recognizes Rep. Yvette Clarke’s interest in “confederate generals,” as it relates to street names in Fort Hamilton, near her Brooklyn, New York, district.

The model notices issues that aren’t quite on the national radar, like the “wotus rule” — AKA, the Waters of the United States Rule, a change in who regulates water pollution that has raised the ire of Republicans such as Rep. Bob Gibbs of Ohio. Or widespread interest among representatives of the rural West, including Sen. Mike Enzi of Wyoming and Rep. Rob Bishop of Utah, about whether to add the sage grouse to the endangered species list, triggering rules that could limit farming and industry near the bird’s habitat.

Just because a topic appears on one member’s list but not another’s doesn’t mean the second Congress member don’t care about it. There may simply be more distinctive topics that they talk about. And for now, that means big topics that lots of representatives and senators talk about, such as education or crime, aren’t included in each member’s list. But we’re working on ways to reflect those, too.

Along with identifying discrete topics, the model finds which members of Congress’ press releases are most similar, in topic or turns of phrase, in essence calculating who “sounds like” whom.

The representative whose press releases are closest to Rep. John Lewis’ is Rep. A. Donald McEachin, another African-American Democrat from a southern state. Rep. Thomas Massie, the model says, puts out releases similar to Sen. Rand Paul, his fellow Kentuckian who also leans libertarian.

How the Model Works

Our code relies on an approximation of what English words mean created by mathematically representing the context in which they occur. The theory that this would give you an idea of words’ meanings is called “Distributional Semantics.”

Why the particular technique we use, called Word2Vec, works so well is a bit of a mystery — especially if you, like me, never studied linear algebra — but it does work. Without being explicitly programmed to know anything about U.S. politics, the model has learned a lot about how our country works:

  • It knows that “death tax” and “estate tax” refer to the same thing.
  • If you ask the model who has the same kind of relationship to Senate Majority Leader Mitch McConnell that Rep. Nancy Pelosi has to Rep. Paul Ryan, its answer is Sen. Chuck Schumer — the Democratic minority leader in the Senate. (Well, it’s a tie: the model suggests Schumer and his predecessor in that position, Harry Reid.)

A related technique, Doc2Vec, assigns a value to individual press releases or a member’s entire body of press releases from the sum of the meanings of the words. Similar to the way in which DW-Nominate, a powerful statistical technique used to characterize where politicians stand along a political spectrum, transforms a congressperson’s voting record into a location in two dimensions, Doc2vec transforms what the Congress member says into a location in 100 dimensions. (However, unlike DW-Nominate, there’s no good way to translate those dimensions into anything that makes analytical sense to humans.) Finding Congress members who sound alike is as easy as finding each member’s “nearest neighbor” in this imaginary 100-dimensional space.

The topics are generated in a way that uses the same software, called Gensim, but relies less on linear algebra and more on counting. It finds the phrases that occur most often in each member’s statements but rarely in everyone else’s — a statistical technique called term-frequency (over) inverse-document-frequency (often shortened to “TF-IDF”) that is a useful proxy for importance. More concretely, it finds that Sen. Enzi’s statements contain the phrase “sage grouse” a lot, but that phrase appears frequently in only a few other members’ statements. A more general topic like “environment” would not show up, since it’s relatively common and only one word long.

The results of the TF-IDF algorithm are not presented verbatim; we do some manual filtering to exclude, say, the name of the member’s contact person for press releases or the phrasing of their “contact me” button.

There’s more in store. Stay tuned for a way to see what bills are related to a given topic — in a way that’s more powerful than just a keyword search. We’re also planning to throw floor statements into the model, as part of the relaunch of the CapitolWords project we inherited from Sunlight Labs earlier this year.

So how did our algorithm do on Murray? . . .

Continue reading.

Written by LeisureGuy

7 October 2017 at 10:25 am

The Equifax Aftermath – We Need More Hacking

leave a comment »

An interesting perspective by Ido Kilovaty at Lawfare:

The Equifax data breach that compromised the Social Security numbers and other personal information of more than 145.5 million Americans prompted calls from authorities and consumers for more federal regulation to protect sensitive personal data. The Federal Trade Commission (FTC), Congress, and the Justice Department, among others, are investigating aspects of the incident. Some experts frame the cause of the breach in terms of market failure, arguing that companies have insufficient economic incentive to secure the data they handle. Others have suggested that the breach reveals no legal or regulatory failures because there is no legislation or regulation that effectively addresses the spate of very public  cybersecurity incidents against sensitive systems.

These approaches all have an ex post nature. But what’s needed, in fact, is to prevent such breaches before they happen. These intrusions occur because software and systems are imperfect and companies’ efforts to secure their systems are often ineffective (which could have roots in a perceived lack of economic incentive or the absence of binding regulation). Regulation and enforcement against cyberbreaches could work only if the cost of being compromised is so high that compliance is the only viable course of action. Of course, law enforcement has limited deterrence value against foreign hackers. Consequently, there must be a  shift in thinking about cybersecurity, particularly when sensitive data is at stake. Regulation is prone to inflexibility, which could quickly leave it ineffective as technology evolves and develops. Hackers could figure out ever more sophisticated ways into consumers’ data, and security methods prescribed by regulations would soon become outdated.

That’s why, if we really want to secure our systems and prevent future Equifax-like breaches, the answer is counterintuitive: We should encourage more hacking against these systems. This would not be malicious hacking but, rather, the “ethical” type: cyber intrusions seeking to help secure systems by identifying security vulnerabilities before they can be exploited. If Nietzsche was right that chaos comes out of order, this is precisely the way to approach cybersecurity challenges. Incentivizing ethical hacking (or “white-hat hacking”) could be much more efficient than traditional forms of regulation. Put another way, encouraging a minimal degree of chaos could help prevent a major one.

Data breaches happen daily, and sectors are targeted indiscriminately. Americans’ credit card details, addresses, security clearances and voter data are just some of the sensitive information that malicious hackers constantly target, often successfully. Rarely discussed after initial reports of data being compromised are the ways breached companies handle vulnerabilities and whether they’re welcoming vulnerability disclosure efforts from the broader hacking community. While many tech companies including Google and Microsoft have vulnerability disclosure systems, companies that monitor credit, such as Equifax, do not have the same level of private-public cooperation with the community of ethical hackers. This often prevents these hackers from probing potentially vulnerable systems. HackerOne, a well-known platform aggregating vulnerability disclosure policies, has no reported guidelines for Equifax.

Although some companies have developed vulnerability disclosure programs, others are not particularly friendly toward ethical hackers who try to help. Cisco threatened legal action against researchers who disclosed vulnerabilities about its internet routers, and HP behaved similarly over a vulnerability in its Tru64 operating system. These inconsistent reactions about vulnerabilities reported by the broader hacking community point up an issue thoughtful legislation or regulation could address. Another foundational issue: making hacked entities directly accountable for data breaches.

Consider: Equifax’s breach might have been prevented if ethical hackers had more freedom vis-à-vis sensitive systems. In the aftermath of the breach, evidence of vulnerabilities was quickly publicized. Brian Krebs reported that Equifax used the default username and password—“admin/admin”—for at least one database. Had good actors been given more incentives to operate, or if vulnerability disclosure programs were mandated at every company that handles large amounts of sensitive information, this might have been spotted and secured earlier.

The issue here is not solely one of law and policy but also of democratic governance. A substantial number of companies process personal data without consumers having any form of influence over what these companies do and how they conduct themselves when it comes to that personal data. More freedom to ethically hack would create an oversight system that involves and engages the broader public and allow ethical hackers’ engagement in security research. In the arms race between malicious and benevolent hackers, law should create a  . . .

Continue reading.

Written by LeisureGuy

6 October 2017 at 10:48 am

Computer Scientists Take Road Less Traveled

leave a comment »

Erica Klarreich writes in Quanta:

Not long ago, a team of researchers from Stanford and McGill universities broke a 35-year record in computer science by an almost imperceptible margin — four hundredths of a trillionth of a trillionth of a trillionth of a trillionth of a percent, to be exact. The advance — made to that poster child for hard-to-solve computer science quandaries, the “traveling salesman” problem — was too minuscule to have any immediate practical significance, but it has breathed new life into the search for improved approximate solutions.

The traveling salesman problem asks: Given a collection of cities connected by highways, what is the shortest route that visits every city and returns to the starting place? The answer has practical applications to processes such as drilling holes in circuit boards, scheduling tasks on a computer and ordering features of a genome.

The traveling salesman problem is easy to state, and — in theory at least — it can be easily solved by checking every round-trip route to find the shortest one. The trouble with this brute force approach is that as the number of cities grows, the corresponding number of round-trips to check quickly outstrips the capabilities of the fastest computers. With 10 cities, there are more than 300,000 different round-trips. With 15 cities, the number of possibilities balloons to more than 87 billion.

Christofides’ Algorithm

In the early days of computers, mathematicians hoped that someone would come up with a much better approach to large traveling salesman problems — some algorithm that would allow computers to solve them in a reasonable amount of time. But while computer scientists have made progress with specific scenarios — identifying the shortest round-trip route for a 49-city map in the 1950s, a 2,392-city map in the 1980s and a 85,900-city map in 2006 — no one has devised an algorithm that can efficiently solve every traveling salesman problem. According to a landmark paper published in 1972, such a solution might not even be possible. The computer scientist Richard Karp, of the University of California at Berkeley, showed that the traveling salesman problem is “NP-hard,” which means that it has no efficient algorithm (unless a famous conjecture called P=NP is true — but the majority of computer scientists now suspect that it is false).

After Karp’s paper was published, many computer scientists set their sights on creating an efficient algorithm to find approximate solutions to the traveling salesman problem — round-trip routes whose lengths come within striking distance of that of the best possible route. Here, they had better luck: In 1976, Nicos Christofides, a professor at Imperial College London, developed an algorithm that produces routes guaranteed to be at most 50 percent longer than the shortest route.

When it was created, many computer scientists assumed that Christofides’ algorithm, which is simple enough to teach to computer science undergraduates in an hour, was a placeholder that would eventually give way to a more sophisticated algorithm able to better approximate the true solution. Yet for decades, that algorithm failed to materialize.

“Almost every graduate student in theoretical computer science has at some point spent a few futile weeks or months trying to improve upon Christofides’ algorithm,” said Sanjeev Arora, a computer scientist at Princeton University.

Finally in 2011, the Stanford-McGill team edged past Christofides’ 50 percent guarantee for certain types of traveling salesman problems, showing that its algorithm’s solutions would be at most 49.99999999999999999999999999999999999999999999999996 percent longer than the true answer.

A string of improved approximation algorithms have since emerged, after computer scientists began looking at the problem with fresh eyes. Although these approximations apply only to certain types of traveling salesman problems, the approach they embody holds great promise, said Michel Goemans, a computer scientist at the Massachusetts Institute of Technology.

“We’ve barely scratched the surface,” he said. “I’m a big believer that, maybe five years down the road, there will be a much more powerful result.”

The Shortest Tree

Christofides’ algorithm starts by looking not for the shortest round-trip route, but the shortest “spanning tree” — a collection of branches linking the cities, with no closed loops. Unlike the shortest round-trip route, the shortest spanning tree is easy to construct efficiently: Start by finding the shortest highway connecting two cities; that’s the first branch. To add the next branch, find the shortest highway connecting a new city to one of those two — and so on until all the cities have been reached.

The resulting tree is not a possible solution to the traveling salesman problem because it does not create a round-trip route. But it can be converted into a round-trip by visualizing the branches as walls and then imagining walking along the tree, with your finger touching the wall, until you get back to where you started. The resulting trip visits every city and returns to the starting point, but it is usually far from the shortest way to do so because it typically involves retracing steps many times — every wall in the tree is traversed twice, once on each side.

However, this round-trip route is, at worst, twice as long as the best solution to the traveling salesman problem. By adding some carefully chosen highways to this tree and taking some clever shortcuts, Christofides showed how to trim this round-trip to one that is at most 50 percent longer than the shortest route.

The shortest spanning tree was a natural starting point for efforts to build a short round-trip tour. But this approach also offered an opening for researchers trying to whittle down Christofides’ 50 percent guarantee. For although the shortest spanning tree seems effective at first, other trees may be better when it comes to the short-cutting process that converts the tree into a round-trip — for example, a tree that never branches needs only one added highway to become a round-trip.

So the goal was to find a spanning tree that strikes the perfect balance between length and easy conversion into a round-trip. No efficient algorithm can uncover this perfect tree (unless P=NP), but a successful approximation algorithm only needs to find a pretty good one.

A Fractional Trip

The path toward that “pretty good” spanning tree has involved the widely used but counterintuitive technique of allowing fractional solutions to certain types of problems. A fractional round-trip, for example, might involve going on half the highway from Minneapolis to Detroit and half the highway from Cleveland to Detroit. Such a route is, of course, nonsense from a practical perspective. But it can be formulated in precise mathematical terms, and, unlike the standard traveling salesman problem, this fractional version can be solved efficiently.

Many approximation problems in computer science can be tackled by calculating the solution to the fractional version of the problem and then finding a smart way to round off the fractions to produce an approximate solution to the original problem. But until recently, no one had figured out a good way to do this for the traveling salesman problem.

In 2009, Amin Saberi of Stanford University and Arash Asadpour, then a graduate student, developed a general rounding technique that uses randomness to try to pick a whole-number solution that retains as many properties of the fractional solution as possible. Saberi saw this new rounding technique as “a strong hammer looking for a nail.” The right nail, he suspected, might be the traveling salesman problem.

A few months later, Saberi, Asadpour, Goemans, Stanford graduate student Shayan Gharan, and Aleksander Madry, now of the École Polytechnique Fédérale de Lausanne in Switzerland, showed that the new rounding technique could produce a good approximation algorithm for a variation of the traveling salesman problem, the “asymmetric” case. The following year, Saberi, Gharan and Mohit Singh of McGill University used the same approach to develop a new approximation algorithm for the ordinary traveling salesman problem.

Given a map of cities and highways, the new algorithm starts by calculating the exact fractional solution to the traveling salesman problem. Next, it rounds off that fractional solution to a spanning tree that will hopefully come close to striking the sought-after balance between length and easy conversion to a round-trip. Finally, the algorithm plugs that spanning tree — rather than the shortest spanning tree — into Christofides’ framework.

The new algorithm was “an idea we could describe in an hour or two, but proving that it actually worked took more like a year,” Saberi said.

After a lengthy analysis, the Stanford-McGill team was finally able to beat out Christofides’ algorithm by . . .

Continue reading.

Written by LeisureGuy

6 October 2017 at 10:20 am

Posted in Math, Software

%d bloggers like this: