In this quick video, I’m going to show you how to install a free Chrome plugin known as Link clump to easily copy Google web page results for any URL you wish. Most crawlers (Search Engine Optimization robots) do not pull up Google results for URLs that are not listed in Google’s index.
So, what you need is a Google extractor to grab these URLs from the Google index. It will grab all of the links in your site and copy them into a new Google document or spreadsheet. You’ll be able to see Google web page data being pulled out of the Google index from your site. The reason this is such a simple process is that the Google index is one big file of thousands of web pages.
Now, with the link clump web scraper, it can easily grab a copy of each Google page for any URL on your site. This allows you to easily extract web pages from sites like Squid, Hub Pages, or WordPress. There’s no need to manually grab page data manually.
The other big advantage of using a Link clump Google extractor is the ability to automatically get the page title, meta tags and other metadata for every web page on your site scrape google search results. This is so important to the search engine spiders. The more metadata you have, the easier it is for Google to index your site. A good quality web scraping program will give you full metadata and HTML source code for every page on your website. This means you won’t have to manually go through and manually type in all of this info.
For anyone wondering if this is a hack, I can assure you this is 100% legal as long as you’re only pulling up Google pages. If you attempt to scrape other pages, you may be breaking the terms of service of many of the major search engines as well. So, you may want to keep your scrape Google pages to only pull up Google page data.
To get a free Google page extractor, check out Link clump at the link below. Download and install it now!
Now that you’ve had a quick look at how easy it is to grab a Google page from a site, you may want to go ahead and download a scraper for other major search engines like Yahoo, Bing, and YouTube. These will work exactly the same way as Link clump, but they will work for millions of searches, not just for Google.
So, the next time someone asks how to scrape Google results, I encourage you to give this method a try. I know it may sound simple, but you will be surprised at how many sites don’t have all the pages that Google pulls.