Web Scraping in PHP using Goutte - part 2
Tags: webscraping, goutte, phpscraping
Published: 2018-12-28 18:05:17

---
title: Web Scraping in PHP using Goutte - part 2
published: true
description: we use Goutte to scrape a news website for the latest headlines  
tags: web scraping, goutte, php scraping 
series: PHP Web Scraping 
---
# Web Scraping in PHP using Goutte II

In the [last article](https://dev.to/sayopaul/web-scraping-in-php-using-goutte--4p6h), we got introduced to web scraping and we looked into [Goutte](https://github.com/FriendsOfPHP/Goutte), a wonderful PHP web scraping library . In this article, we would be putting our knowledge to practice by scraping the website of the [Punch](https://punchng.com) . To be more specific, we would be scraping the punch to get the lastest news [https://punchng.com/topics/news](https://punchng.com/topics/news) headlines  😎 .

 Let's get right into it 💪 !

**NB : This is for testing purposes only, I do not in any way intend to reproduce the material gotten from the Punch and I do not advice you to do so as that would be copyright infringement** .


First things first, we set up Composer autoloading, import the Goutte namespace and we instantiate a new Goutte Client:
```php
    require "vendor/autoload.php";
    use Goutte\Client;
    $client = new Client();
```
                        
The next step is to send a request via the $client object . The $client object returns a crawler instance . It is this instance that we use to apply our filters .
```php
     $crawler = $client->request('GET',"https://punchng.com/topics/news");
```
                        
On the front page of the Punch news page are article boxes . Each article has its own box and a heading ( The headline ) with the class ".seg-title" . We want to select all the headlines (.seg-title) on the page and then take each of them one by one . We do it with this:
```php
     $crawler->filter('.seg-title')->each(function ($node){


     });
```
Notice the method `each()` ? The `each()` method allows us to iterate over the current selection(node list) when it contains more than one node . As we mentioned above, we are selecting each of the headlines (.seg-title) hence we have more than one node and we want to iterate through them . Underground, the `each()` method accepts an instance of an anonymous function, loops through the current node list and then passes a node on each iteration to the closure thus allowing us to access the current node ( `$node` ) in the closure .
```php
     public function each(\Closure $closure)
     {
          $data = array();
          foreach ($this->nodes as $i => $node) {
              $data[] = $closure($this->createSubCrawler($node), $i);
          }

          return $data;
      }
```
Alright, the next thing we want to do is extract the text from the current node .
```php
     $crawler->filter('.seg-title')->each(function ($node){
         $headline = $node->text();
         echo $headline;
     });
```
We get the textual content of the node by calling the method `text()` . The next thing we do is print out the headline and there we have it ! We would always get the latest 10 news headlines on the punch printed out to us whenever we run this script . Like I said in the previous article, when it comes to scraping, almost anything is possible ( even logging in and filling forms ) . The limit is your mind 😊 . I honestly wish we could go deeper but sadly that's all for now 😅 . 

For more information, please do well to read the docs of [DomCrawler](https://symfony.com/doc/current/components/dom_crawler.html), [CssSelector](https://symfony.com/doc/current/components/css_selector.html) and [Goutte](https://github.com/FriendsOfPHP/Goutte) .

Do you have any web scaping needs ? You can hire me to help you out [here](https://www.fiverr.com/sayopaul/scrape-webpage-for-you-and-save-it-in-any-format-in-php)