BlackHat360

Google has just launched a new “search options” feature on its main search page. When you click on “Search options” you can filter your search by different types of results (videos, forums, and reviews), by time (recent, past 24 hours, past week, past year), as well as seeing related searches, a “wonder wheel” view, or a timeline view.

At Google’s Searchology event, which is going on right now, Marissa Mayer listed the following as the hardest unsolved problems in search:

- Finding the most recent information
- Expressing that you want just one type of result
- Assessing which results are best
- Knowing what you’re looking for
- Expressing your searches in keywords

Notice that real-time search is the No. 1 problem. (Twitter and a bunch of startups from OneRiot to Tweetmeme are also working on it, with the latter two launching their own real-time search efforts today). And it certainly is a problem for Google, even with the new recent results option. Try searching for any of teh top trending results on Twitter right now like Miss California (vs. Twitter search results) or Star Trek (vs. Twitter results), and you don’t even get any Twitter results on Google.

While real-time search is still a big problem, it is not the only problem. Some of the new options address the difficulty of searching back through time. The recent results get as real-time as Google can get, but you can also expand the timeframe. And you can look at an actual timeline of results, which looks for dates within results and then places them chronologically (this is sort of hit or miss—just because a date is mentioned in a text does not mean the entire result is about or from that period of time). Google now also lets you see related searches as an option. And the Wonder Wheel is more of a visual aid to see how different related topics are clustered together. When you click on any spoke of the wheel, it then causes that search term to be at the center. We’ve seen many of these techniques in the past, but Google is giving them a higher profile by putting them in its main search page..

One of the next frontiers of search is taking all of the unstructured data spread helter-skelter across the Web and treat it like it is sitting in a nice, structured database. It is easier to get answers out of a database where everything is neatly labeled, stamped, and categorized. As the sheer volume of stuff on the Web keeps growing, keyword search keeps getting closer to its breaking point. Adding structure to the Web is one way to make sense of all that data, and Google is starting the tackle the problem with a Google Labs project called Google Squared, which Marissa Mayer mentioned earlier today at the company’s Searchology briefing.

Google Squared extracts data from Web pages and presents them in search results as squares in an online spreadsheet. Michael was at the event and got a personal demo (see video below). From Michael’s Searchology notes:

Google Squared is launching later this month in labs. Google Squared returns search results in a spreadsheet format. It structures the unstructured data on web pages. So a search for Small Dogs returns results with names, description, size, weight, origin, etc., in columns and rows.

Google is looking for data structures on the web that imply facts, and then grabbing it for Squared results. “It takes an incredible amount of compute power to create one of those squares,” she says.

This type of technology has obvious applications for many types of targeted searches, including product search, health search, scientific searches, you name it. There are dozens of semantic search startups trying to impose structure on the Web to perform similar tricks. Another high-profile search startup which is launching on Monday, Wolfram Alpha, takes a slightly different approach in that it simply ingests massive amounts of information into its own databases where it can query it to its heart’s delight. Already there is a bit of a rivalry between Google and Wolfram because getting back structured results is a major new direction for search.

Wolfram does a pretty good job parsing the information in its own databases, but those databases will never match what is available on the Web. Wolfram’s databases currently store only 10 terabytes of information, a tiny fraction of what is on the Web. (I will be posting my impressions of Wolfram’s search engine soon). Google Squared is an early attempt to take the messy data which exists on the Web and place it into simple tables. It is still very experimental and isn’t always on target, but you can see where this is going. Turning the Web into a giant database will crush any attempt to segregate the “best” information into a separate database so that it can be processed and searched more deeply.

In the video demo below, a search for “camera” sorts the results in different columns by images, description, and manufacturer, resolution, etc.. You can refine results by clicking on a particular column such as manufacturer. A search for “rollercoasters” sorts results by name, image, description, height, length, and number of inversions. But sometimes it gets confused. A search for “spaceships” turns up a Corvette and a missile carrier. It is going to be a while before this makes it out of Google Labs