I noticed a few days ago that "site:domain.com" searches no longer go any deeper than 1,000 urls - many sites show significantly less, even when the number of indexed urls is high. Not yet seeing any push back from the seo community, but this is really huge for those of us responsible for the ranks of our clients. If your site has a large number of urls, this change means that you no longer can view what Google has cached beyond a limited set that is no greater than 1,000 results. This is not a good thing.
Over time, Google has become less and less transparent regarding the data that it chooses to make available to us. The recent uproar over Google's announcement to prevent access to referral data from logged-in searches has appropriately angered the seo community and is only the latest in a string of harmful pullbacks.
"Last week, the search giant said that it would begin encrypting logged-in searches that users do by default, when they are logged into Google.com. This further integration of a Secure Sockets Layer (SSL) will prevent search marketers from receiving referral data from the websites consumers click on from Google search results." seos were not buying google's privacy motive for encrypting search 2011-10
So Google's walling off and withholding the data on the search behavior of its user base, and claiming they're doing it for security/privacy reasons. They're downplaying the small numbers this involves, but at the same time, they're making it easier, and encouraging users to remain logged in - you can choose to do this from every Google property (eg. Gmail, YouTube, Google Maps, Google News, Google Video, etc.). So these numbers are not going to stay small, and the more it grows the more valuable the data you're not getting. Apparently, you can get access to the https data by advertising. How can you buy the privacy excuse when the changes do not impact advertisers? I see this as a way for Google to compete with Facebook - isolating the community into a vertical, then monetizing the data. It's their right, but it's bad for us.
In July 2010, Google removed our ability to see all inbound links they had discovered. Webmaster Tools now shows us only a 'sample' of the total they store. I have 2 posts on this: Caffeine May Have A Hidden Cost & Coping With The Loss Of Link Metrics. So Google penalizes sites for certain kinds of links, yet we may not be able to discover those issues when our ranks are harmed.
The withholding of data has become a pattern over the recent past. Here's a summary of the important ones I'm aware of:
- Index searches now incomplete - limited to 1,000
- Logged-in search metrics no longer available
- Link data from Webmaster Tools incomplete
- Supplemental results still exist but are no longer labeled as such
And while we're on the subject of harmful decisions, a client had an experience with Google that is the epitome of unethical and unjust behavior from the enterprise that claims to do no evil. His site was penalized and in order to keep his business functioning had to turn to Adwords (Google undoubtedly gains advertisers by penalizing sites). After a short time, the Adwords account was suspended, with no explanation. It turns out that a competitor had complained falsely that my client was selling counterfeit merchandise. Google would not reveal who the complainant was, and the only way to recover the account was to have the manufacturers confirm that my client was indeed a legitimate distributor of their goods. In Google's eyes, my client was guilty based on the claims of unnamed accusers, and the burden was on him to prove his innocence.
I'm aware that Google has the right to do all the things being discussed here, and in spite of its pr machine touting how people friendly the place is, it is a corporation that has morphed into a huge monopoly with agendas that reveal them to be more of an adversary than ever before. And lets not forget that every major industry, oil, telephone, railroads, steel, etc. all evolved via a free market to the point that one giant monopolized the market, and regulation was required to force ethical behavior upon them. Google is that monopoly right now, without the regulation.
Occupy Wall Street is the result of people finally saying, "Enough!" to the bad behavior. Wondering when that threshold will be reached with Google's continuing slide, and will enough people be willing to camp out to motivate change. Kind of scary to think about.