I want to scrape all the options under the select element to scrape product variants available on the eCommerce website which I am scraping. Web scraping drop-down lists become a little easy with Agenty.
Let’s take an example…
Example HTML
I tried to use the “.sizes” selector and it’s not scraping anything. And I don’t want full text, instead the option value of the select item.
Here “sizes” is not a class, so you cannot use it with dot (.) to match this element. You should be using the attribute selector select[name="sizes"] option and value as your attribute to scrape all values.
select dropdown list options scraping1103×526 22.5 KB
If you want to know more about scraping selected options from the dropdown menu, you can go through this how to extract selected options from dropdown list!
Here is Github example to try out - Forum #16