The Semantic Imperative

Words NOT meaning. Semantics NOT comprehension.

This is a hugely important distinction. Our searches are not understood. The results show us a string to string match of words.

A page relevant by way of meaning, but not semantics will never appear in the search results, a clear failing of today's search engines.

But the semantic imperative has significant negative consequences, especially if the goal is relevance. Google refers to a term 'stop words' when evaluating a search. Stop words are considered noise, and are completely ignored. 'By', 'to', 'for', 'from', 'at', 'and' and every other short preposition and article are all 'stop words', ignored by the algorithm. While that makes things faster, it also generates confusion.

Because stop words include all the short prepositions. And the prepositions impart MEANING. Google's use of stop words is one huge compromise away from being able to understand the search. For convenience sake, some words are ignored.

Here's how this harms the search: If you're looking for books written by girls, you might search for "books by girls." But if you throw away the 'by' you don't understand the search, and it becomes the same search as 'books FOR girls' or 'books WITH girls' or 'books ABOUT girls' - none of which yield books BY girls.

That's the crux of the problem with our current search engines: The search is not understood, it's merely a semantic match. This suggests the main area of improvement for the next generation of search tools. Because when the search is understood, the doors open to offerings much more valuable than semantics - related products and services, and the ability to suggest meaningfully.

But for the enterprise operating in the current environment, respect must be paid to the philosophical underpinnings of the semantic imperative. Operating in a flawed search environment requires over-optimization, because the search is not really understood, so every semantic iteration of any valuable target must be addressed as a separate target. All these efforts create the very conditions Google preaches against, yet the enterprise will be not make productive use of rank if it is not optimizing for these seemingly redundant targets.

The semantic imperative requires the competitive enterprise to optimize not only for 'blue widgets,' but also for 'widgets blue,' 'blue widget.' widget blue,' etc. If the search were understood, this would clearly not be necessary, and the web would have much less intentional redundancy, ie. garbage. That's not the world the Enterprise inhabits.

The goal of the next generation search engines will be the elimination of the semantic imperative, and that comes automatically with the ability to understand the search.