# Web communities network mapping with Hyphe

Benjamin Ooghe-Tabanou

► To cite this version:

Benjamin Ooghe-Tabanou. Web communities network mapping with Hyphe. NodeXL Academy: Social Media Research Winter School 2022, Social Media Research Foundation, Jan 2022, online, France. hal-03904193

HAL Id: hal-03904193

https://sciencespo.hal.science/hal-03904193

Submitted on 16 Dec 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
<figure><img src="image_2.png" /></figure>

# Web communities network mapping with Hyphe

NodeXL Academy '22

January 19th 2022

Benjamin Ooghe-Tabanou (@boogheta)

Sciences Po médialab (@medialab_ScPo)

SciencesPo
MÉDIALAB
* Pluridisciplinary Research Lab created by Bruno Latour in May 2009, led by Dominique Cardon since 2017

* Social Sciences, Engineering & Design

* Articulate qualitative & quantitative methods through a digital approach

* Work with digital traces

* Deploy an ecosystem of tools
http://tools.medialab.sciences-po.fr

* METAT: a monthly Open Support Workshop
https://www.sciencespo.fr/recherche/fr/content/metat-latelier-de-methodes

<figure><img src="image_3.png" /></figure>

<figure><img src="image_4.png" /></figure>
<figure><img src="image_5.png" /></figure>

« Google is nice, but we need something better »

The Indian Express, 2011
# Webcrawlers? What is that?

> « Web crawling is the process of building a collection of webpages by starting with an initial set of URLs and recursively traversing the corresponding pages to find additional links. »

Gabe Ignatow & Rada Mihalcea, in: Text Mining: A Guidebook for the Social Sciences

But why would anyone want to do that?

- create an index for a search engine (like... you know...)

- extract contents or structured data (like scraping)

- perform some text mining

- study network communities through hyperlinks
# Crawling the web for social sciences?

> « **Hyperlink** » is at the core of the Web's architecture
> → charged with meaning and structure

> « The texts are **linked together** in a way that one can go from one concept to another to find the information one wants.

> The network of links is called a **web**. [...]

> The texts are known as **nodes**.

> The process of proceeding from node to node is called **navigation**. »

Tim Berners-Lee, 1990, WorldWideWeb: Proposal for a HyperText Project

<figure><img src="image_6.png" /></figure>

> « A hyperlink is a **manifestation of intention**. **By linking** one page to another, one piece of text to another, **people intend** to do particular things. »

Ryfe, Mensing, & Kelley, 2016, What is the meaning of a news link?
# The Web: a place of controversies and debates

<figure><img src="image_7.png" /></figure>

CC-BY-NC - Randall Munroe - XKCD
# A bottom-up hierarchy emerged from hyperlinks

« Matthew effect » : preferential attachment

→ new web pages tend to cite the already most cited ones

<figure><img src="image_8.png" /></figure>
# Hyphe: a research directed crawler

http://hyphe.medialab.sciences-po.fr/demo/

Build your own web documents corpus
to study social phenomena online

→ gather « web actors »

→ explore hyperlinks between them

<figure><img src="image_9.png" /></figure>

hyphe

<figure><img src="image_10.png" /></figure>

http://medialab.github.io/double-dating-data/

COP 21
Vie privée
Extrême droite
Tissu associatif
Produits laitiers
Cellules souches
Administrations culturelles
...

<figure><img src="image_11.png" /></figure>

http://utopies-concretes.org/
# There is no such thing as a « website » !

<figure><img src="image_12.png" /></figure>

→ « **WebEntities** » : bundles of webpages aggregating coherent actors to answer a specific research question
= set of URL prefixes
# Finely delimit the web territories of actors

<figure><img src="image_13.png" /></figure>

Manually setup prefix patterns to adjust the cursor of « WebEntities »

<figure><img src="image_14.png" /></figure>
# Which data structure to manage hypertexts?

https://medialab.github.io/hyphe-traph/fosdem2018/#/

- **Tree** of URLs

- **Graph** of hyperlinks

- Dynamic branches aggregates

→ Hyphe's Traph

<figure><img src="image_15.png" /></figure>
# Hyphe's crawling strategy: leverage hyperlinks

* Classical crawlers (DMI's IssueCrawler): Snowball
  → Top Layer attraction (Google, YouTube, Wikipedia...)
  → Topic drifts

<figure><img src="image_16.png" /></figure>

* Hyphe:

- crawl exclusively pages within the chosen WebEntities

- sort discovered entities by degree of citation

- humanly select new entities to include and crawl

<figure><img src="image_17.png" /></figure>
# Web prospection loop: curating a corpus iteratively

* Step by step iterative expansion & curation of entities

<figure><img src="image_18.png" /></figure>

* Human/Time cost

* How to know when to stop?

→ hyperlink citations threshold
# Qualify the corpus entities (tagging)

* Free notes

* Categories

<figure><img src="image_19.png" /></figure>
# HyBro: a web browser designed for corpus curation

https://github.com/medialab/hyphe-browser/releases/

<figure><img src="image_20.png" /></figure>

## « NaviCrawler » heritage : build a Hyphe web corpus while browsing

→ « in-situ » prospection & tagging (digital field work)

→ teach the web to students
# Many ways to use Hyphe

* Complete methodology includes:

sourcing, automatized collection, iterative corpus building
qualitative categorization, exploratory analysis,
network visualization, quantitative statistical analysis

* Diverse audiences:

- Research: help social scientists work on digital fields

- Pedagogy: teach students what the web is beyond Google & Facebook

* Possible small & large scale analyses:

- a website's internal structure

- a theme's ensemble of actors and their ties

- a controversy's alliances & oppositions

- etc.
# From above: clusters, opposition & affinity

## Corpus web sur le changement climatique : types d'activités soutenues

Le réseau représente les acteurs tels qu'ils sont reliés sur le web par des hyperliens. Les acteurs les plus cités dans le corpus ont une plus grande taille. La couleur des liens est celle de l'acteur qui en est à l'origine.

## Web corpus on climate change: type of supported policy

The network represents how actors hyperlink on the web. The most cited actors have the biggest representation. Links are colored according to their source.

## Double Dating Data: Climate change on the web

<figure><img src="image_21.png" /></figure>

TYPES D'ACTIVITES SOUTENUES / TYPE OF SUPPORTED POLICY

<figure><img src="image_22.png" /></figure>

ADAPTATION

<figure><img src="image_23.png" /></figure>

ADAPTATION

<figure><img src="image_24.png" /></figure>

MITIGATION

<figure><img src="image_25.png" /></figure>

NOT RELEVANT, UNKNOWN OR NOT CATEGORIZED
# From above: clusters, opposition & affinity

## Corpus web sur le changement climatique : le changement climatique est-il anthropique ?

Le réseau représente les acteurs tels qu'ils sont reliés sur le web par des hyperliens. Les acteurs les plus cités dans le corpus ont une plus grande taille. La couleur des liens est celle de l'acteur qui en est à l'origine.

## Web corpus on climate change: anthropogenic climate change

The network represents how actors hyperlink on the web. The most cited actors have the biggest representation. Links are colored according to their source.

## Double Dating Data: Climate change on the web

<figure><img src="image_26.png" /><figcaption>POSITIONNEMENTS DES ACTEURS / ACTORS' POSITION</figcaption></figure>

Le changement climatique est-il anthropique ? / Anthropogenic climate change

<figure><img src="image_27.png" /></figure>
# From within: explore webpages contents

EXPLORE WEB ENTITIES

## PRIVACY WEB CORPUS

SciencesPo
MÉDIALAB

<figure><img src="image_28.png" /></figure>

AXA
Research Fund
Through Research, Protection

<figure><img src="image_29.png" /></figure>

Dota
Innovation Lab

ABOUT

<figure><img src="image_30.png" /></figure>

2,313 ENTITIES

7,549 entities represented as a cloud

<figure><img src="image_31.png" /></figure>

<figure><img src="image_32.png" /></figure>

Search

<figure><img src="image_33.png" /></figure>

Apple FBI backdoor

**nakedsecurity.Sophos**

https://nakedsecurity.sophos.com/category/technologies/operating-s..
developers would rather quit than give FBI a backdoor A lead
developer for the Tor Project said

**Helpnetsecurity**

https://www.helpnetsecurity.com/tag/backdoor/
encryption backdoors a bad idea March 4, 2016 backdoor
cybercriminals encryption Apple and the FBI

**nakedsecurity.Sophos**

https://nakedsecurity.sophos.com/2016/03/23/tor-project-says-devel...
developers would rather quit than give FBI a backdoor 23 Mar 2016 1
Apple, iOS, Law & order, Privacy

**nakedsecurity.Sophos**

https://nakedsecurity.sophos.com/2016/03/23/tor-project-says-devel...
developers would rather quit than give FBI a backdoor 23 Mar 2016 1
Apple, iOS, Law & order, Privacy

**Sidstamm**

http://blog sidstamm com/2016_02_01_archive.html
their phones vulnerable is not the right approach. The current public
discourse on the Apple vs. FBI "open

**Laquadrature**

https://mediakit.laquadrature.net/view.php?full=1&id=2374
20160219[AC] LCP Chiffrement Apple FBI Download: MP4, WebM,
How to embed ?

**Laquadrature**

https://mediakit.laquadrature.net/view.php?id=2374
20160219[AC] LCP Chiffrement Apple FBI Download: MP4, WebM,
How to embed ?

**Laquadrature**

https://mediakit.laquadrature.net/view.php?full=1&id=2374
20160219[AC] LCP Chiffrement Apple FBI Download: MP4, WebM,
How to embed ?

<figure><img src="image_34.png" /></figure>

Topics

Surveillance FR

Business & Media

Surveillance US

Cybersecurity

Big data & Analytics

Data Regulation FR

Cookies & Tracking

Telec Operators FR

Card and ID fraud

EXPLORE TOPICS

## Privacy Web Corpus Datascape
# From the sides: a hierarchy of directed hyperlinks

<figure><img src="image_35.png" /></figure>

## Conspiracy theorists

- Not reliable

- Not cited

- Cite many reliable sources

<figure><img src="image_36.png" /></figure>

## French national media

- Reliable (*in this qualification)

- Cited by everyone

- Cite only platforms and regional media

## Visual Network Exploration for Data Journalists
# Hyperlinks directionality: a bottom-up hierarchy

<figure><img src="image_37.png" /></figure>

<figure><img src="image_38.png" /></figure>

Most hyperlinks stem from the unreliable and aim at the reliable resources

Each bloc's surface is proportional to the count of websites. The color code is the same as the "Décodex".

<figure><img src="image_39.png" /></figure>

The thickness is proportional to the normalized link density*

<figure><img src="image_40.png" /></figure>
# Explore polarization dynamics

<figure><img src="image_41.png" /></figure>

Unfolding the multi-layered structure of the French Mediascape
Bibliography

Reference publications:

- Jacomy M., Venturini T., Heymann S., Bastian M. (2014), ForceAtlas2, a Continuous
  Graph Layout Algorithm for Handy Network Visualization Designed for the Gephi
  Software, PLos ONE, 9(6), 1-18
https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0098679

- Jacomy M., Girard P., Ooghe-Tabanou B., Venturini T. (2016), Hyphe, a Curation-
  Oriented Approach to Web Crawling for the Social Sciences, ICWSM 2016, Cologne
https://spire.sciencespo.fr/hdl://2441/6obemb2hsj9pboj9bbvc7sftne

- Plique G., Jacomy M., Ooghe-Tabanou B., Girard P. (2018), It's a Tree... It's a Graph... It's
  a Traph! Designing an on-file multi-level graph index for the Hyphe web crawler,
  FOSDEM 2018, Bruxelles
https://medialab.github.io/hyphe-traph/fosdem2018/#/

- Ooghe-Tabanou B., Girard P., Jacomy M., Plique G. (2018), Hyperlink is not dead!, ACM
  Proceedings of the 2nd International Conference on Web Studies (WS.2 2018) Paris.
http://hyphe.medialab.sciences-po.fr/docs/20181004-ACM-WebStudies-HyperlinkIsNotDead.pdf
Now let's play!

https://hyphe.medialab.sciences-po.fr/demo/

https://github.com/medialab/hyphe

Any question first?

benjamin.ooghe@sciencespo.fr

@boogheta @medialab_ScPo