Turning reviews into ratings
The proliferation of websites such as Yelp and CitySearch has made it easy to find local businesses that meet common search criteria -- moderately priced seafood restaurants, for example, within a quarter-mile of a particular subway stop. But what about the not-so-common criteria? How big are the portions? Are diners packed too closely together? Does the bartender make a good martini?
That kind of information often turns up in reviews posted by site users, but finding it can mean skimming through pages of largely irrelevant text. A new system from the Computer Science and Artificial Intelligence Laboratory鈥檚 Spoken Language Systems Group, however, automatically combs through users鈥� reviews, extracting useful information and organizing it to make it searchable.
The first thing the system does is determine the grammatical structure of the sentences that compose the reviews and sort the words used into adjective-noun pairs. If, for instance, someone has written, 鈥淚 found the martinis to be excellent,鈥� the algorithm extracts the phrase 鈥渆xcellent martinis.鈥�
As the group鈥檚 name might imply, its principal area of research is computer systems that respond to spoken language, and indeed, the interface for the new system is speech-based: A user looking for seafood restaurants, for instance, simply says 鈥淪how me seafood restaurants鈥� into the microphone of either a computer or a cell phone. Likewise, the algorithm that does the grammatical analysis is one that Stephanie Seneff, a senior research scientist with the group, began developing 20 years ago as a component of speech-recognition systems. Seneff and her grad student Jingjing Liu applied the algorithm to the substantially different problem of parsing written text with very little modification and even less certainty about how it would fare. 鈥淲e ran it, and we were absolutely delighted with how well it worked,鈥� Seneff says.
Seeing sense
The algorithm produces its adjective-noun pairs 鈥� like 鈥渆xcellent martinis鈥� or 鈥渇riendly vibes鈥� 鈥� based purely on the words鈥� positions in sentences; it has no idea what the words mean. Fortunately, many review sites allow users to provide numerical scores for some aspects of their customer experience. In work presented at several different conferences sponsored by Association for Computational Linguistics, Liu and Seneff developed a second set of algorithms that use numerical ratings to infer adjectives鈥� meanings. If people who describe food as 鈥渆xcellent鈥� consistently give it five out of five stars, and people who describe food as 鈥渉orrible鈥� consistently give it one out of five stars, then the system deduces that 鈥渆xcellent鈥� probably indicates greater customer satisfaction than 鈥渉orrible.鈥�
Once the system has calibrated a set of adjectives against numerical scores, it uses them to infer the meanings of still other words. For instance, if the service at enough restaurants is consistently described as both 鈥渉orrible鈥� and 鈥渞ude,鈥� the system concludes that 鈥渞ude,鈥� like 鈥渉orrible,鈥� is a term of opprobrium. Similarly, if the adjective 鈥渞ude鈥� is frequently paired with nouns like 鈥渟ervice,鈥� 鈥渨aiters鈥� and 鈥渟taff鈥� 鈥� but not with nouns like 鈥渧iew鈥� or 鈥減arking鈥� 鈥� then the system deduces that 鈥渟ervice,鈥� 鈥渨aiters鈥� and 鈥渟taff鈥� are thematically related terms.
As a consequence, if a user asks the system to identify restaurants with nice ambiance, its list of search results will include restaurants described as having, say, a 鈥渇riendly vibe.鈥� The system can also use information gleaned from the sites of the businesses under review to expand its semantic repertory. If, for instance, the foie gras and bisque at some restaurant are consistently praised, and they both turn up, on the restaurant鈥檚 website, under the menu heading 鈥渁ppetizers,鈥� then the system will include the restaurant among those with good appetizers, even if the word 鈥渁ppetizer鈥� never appears in any of its reviews.
Xiao Li of Microsoft鈥檚 Speech Research group says that extracting quantitative ratings from unstructured reviews is a hot research topic both in the academy and in industry and that several commercial products already offer some version of the same functionality. 鈥淏ut you can always do it better,鈥� she says. The MIT researchers鈥� work is distinct, she says, in that 鈥渢hey do a lot of linguistic analysis.鈥� Other systems, for instance, might try to infer relationships between words without first determining their parts of speech. Which approach will prevail remains to be seen, she says, but she adds that the abundance of research in the area demonstrates that the work has obvious practical import.
Two prototypes of the MIT system, both with speech interfaces, can currently be found online. One takes and contains information on businesses in Taipei, Taiwan, and the other takes and includes information on businesses in Boston.
Another grad student in the group, Alice Li, has used similar techniques to extract information from online discussions of patients鈥� experiences with pharmaceuticals. In a yet-unpublished paper, Li, Seneff and Liu present evidence that certain types of cholesterol-lowering drugs may pose a significantly higher risk of some neurological side effects than their alternatives.
This story is republished courtesy of MIT News (), a popular site that covers news about MIT research, innovation and teaching.
Provided by Massachusetts Institute of Technology
