How to use the list scraper

Advanced

The list scraper finds list elements on a website and captures them. 


To build your list scraper model, run the following command: 

Do create new list scraper model


Then select elements with our visual element selector. 


❗️ First, select two similar list items so that Bardeen recognizes the list that you want to scrape. Those elements will show inside a blue box. 


Second, you can select additional elements inside the first list item.


When ready, click “Done.”


Now, run the following command: 

Get data using scraper [last model] with pagination [number of pages]


You can run the DO command to add the data to Sheets, Airtable, or Notion. 


Click on “New Playbook” in the compose tab to create a custom playbook to make this automation run with a single click. 


Pagination

You have two options to scrape larger lists: infinite scroll and click pagination.


Websites such as Facebook or Instagram load new list elements, when you scroll to the bottom of a page. So pick “infinite scroll” on them. 


Other websites such as Google or LinkedIn, require you to click a button to go to the next page. For those sites, pick “click pagination” and use the visual element selector on the element that take you to next page (not a specific page).


You can specify how many pages you want to scrape in the “Get data using scraper model with pagination limit.”