Chrome Web Scraper Tutorial From Semalt Expert
If you are utilizing Google Chrome, there is an extension for your browser which can help to scrape web pages. It is known as ''Scrapper,'' and it can be utilized without problems. Scrapper will aid in scraping a website content and uploading the results to Google documents.
How to scrap a website using Scraper extension?
1. Select Chrome Web Store in Google Chrome;
2. In extensions, carry out a search for ''Scrapper'';
3. The first search result is the extension known as ''Scrapper'';
4. Select the button listed as ''Add to Chrome";
5. Get back to the UK MPs listing;
6. Click the following link;
7. Now look for one MP and ensure the entry is marked;
8. Right-click to choose the "Scrape Similar..." option;
9. The console for scrapper will pop up in another window;
10. View the scraped content in the scraper console;
11. To ensure the content is saved as a Google Spreadsheet, select "Save to Google Docs..."
Extended scraping
Before sticking to this recipe, it is useful to understand the basics of HTML. For example, you can read a short introduction to HTML via this link
Let's imagine we are interested in all movies which starred Asia Argento, a famous Italian actress.
1. There is a very detailed archive of actors in IMDB. Asia Argento site is: https://www.imdb.com/name/nm0000782/;
2. Here, you can view all roles played by the actress. Let's begin scrapping the information we are interested in;
3. Try to scrape it the way it was described above;
4. You'll see that the list is a bit distorted. This is due to the fact that the list here can be structured differently;
5. Head to the scraper console. Top left, you'll see the small box which saying XPath;
6. Xpath is a sort of query language which works for XML and HTML;
7. XPath can help to locate the parts of the page you are interest in. The next thing is to find an appropriate element and write the XPath for it;
8. Now let's arrange our table;
9. You'll see that our existing XPath, which has all the data needed is "//div[3]/div[3]/div[2]/div";
10. XPath informs the System to view the HTML doc and choose the third element, then the second element and then all of them;
11. But, we would like to have our data separated out;
12. Utilize the columns section in the console for scrapper to get this done;
13. Let's first find our title – Use Inspect Element to view the title;
14. Check the title within a tag. Add the tag to the XPath;
15. The expression appears to function appropriately, so make it our first column;
16. In the section "Columns", replace the first column's name to "title";
17. Add the XPath to it;
18. In the column section, the XPaths are relative and it means that "./b" will choose the element
19. In the XPath for the title column, add "./b" and select "scrape";
20. Now let's keep going for a year. Years can be found within one span;
21. Create a new column by selecting the small plus next to the column for your title;
22. Using XPath "./span" create a column for "year";
23. Click scrape and view how the year was added;
24. Done!
Post a comment
