Buying Fragment numbers with Go
Tags: blockchain, webscraping, ton, go
Published: 2023-06-07 11:12:25


This article is a combination of two things that I love to do. The first one is what I started doing when I was in Cuba. I used to [scrape](https://en.wikipedia.org/wiki/Web_scraping) the world's websites, mostly in Python. Using libraries like [Beautiful Soup](https://www.crummy.com/software/BeautifulSoup/bs4/doc/) and [Requests](https://requests.readthedocs.io/en/latest/), I used to fetch a lot of data. I really love to do that, it's been a while since I haven't done it, so it is time to practice that again. On the other hand, recently I'm starting to learn more about blockchain. Now that I'm unemployed I have free time to do it, which is awesome. Specifically I've been focusing on the [TON(Telegram Open Network)](https://ton.org/) blockchain.

Today I'm going to share with you a way to scrape [Fragment Marketplace](https://fragment.com/numbers?sort=price_asc&filter=sale), to look for the cheapest available [Anonymous Numbers](https://fragment.com/numbers) and to buy them, at least some of them. From the scraping we are going to get information about the numbers, and we are going to buy them interacting with the smart contract associated with this NFT. The source of this smart contract is public and can be found [here](https://github.com/TelegramMessenger/telemint). Let's go then.

## Source code

The source code for this script can be found in [Gealber/blog_recipes/fragment_buyer](https://github.com/Gealber/blog_recipes/tree/main/fragment_buyer)


## Goal
My goals with the script

1. Write a script that allows me to buy [Anonymous Numbers](https://fragment.com/numbers) at the cheapest price available.



## Requirements
This will only require Golang.



## Flow

The flow will be simple

![sketch flow](https://res.cloudinary.com/djjd0wi5i/image/upload/v1686132495/photo_2023-06-07_12-06-51_kb8ov7.jpg)



## Code
Let's see first how we can retrieve the phone numbers from Fragment Marketplace. For that we are going to use [go-colly](https://go-colly.org/docs/).

**File: utils/fragment.go**

```go
package utils

import (
	"fmt"
	"strconv"
	"strings"

	"github.com/gocolly/colly"
)

func ScrapeFragmentMarket() ([]AnonNumbers, error) {
	numbers := make([]AnonNumbers, 0)
	c := colly.NewCollector()

	c.OnHTML("tr td.thin-last-col a", func(e *colly.HTMLElement) {
		name := strings.Trim(e.Attr("href"), "/number/")
		valueStr := e.ChildText("div.table-cell-value.tm-value.icon-before.icon-ton")
		value, err := strconv.Atoi(valueStr)
		if err != nil {
			return
		}		

		numbers = append(numbers, AnonNumbers{Name: name, Value: value})
	})

	c.OnRequest(func(r *colly.Request) {
		fmt.Println("Visiting", r.URL)
	})

	err := c.Visit("https://fragment.com/numbers?sort=price_asc&filter=sale")
	if err != nil {
		return nil, err
	}

	return numbers, nil
}

```

Now that I have the numbers or **token name**, I will need to know its address in the blockchain somehow. Now is when it comes into play our knowledge of the [Telemint smart contract](https://github.com/TelegramMessenger/telemint/tree/main/func). Reading this smart contract you can notice that there's a method available for requesting the address of a given NFT item in this contract. The method is called [get_nft_address_by_index](https://github.com/TelegramMessenger/telemint/blob/c6e3d326376fa902c932299e6bfd710111ee2fed/func/nft-collection-no-dns.fc#LL99C7-L99C31), this method is a getter method so can be requested off-chain, without cost. I will need to compute the [item_index](https://github.com/TelegramMessenger/telemint/blob/c6e3d326376fa902c932299e6bfd710111ee2fed/func/nft-collection-no-dns.fc#LL48C9-L48C19), unfortunately we don't have a getter method for that. However, reading the contract you can notice that this [item_index](https://github.com/TelegramMessenger/telemint/blob/c6e3d326376fa902c932299e6bfd710111ee2fed/func/nft-collection-no-dns.fc#LL48C9-L48C19) is computed as the [hash](https://en.wikipedia.org/wiki/SHA-2) of the [token_name](https://github.com/TelegramMessenger/telemint/blob/c6e3d326376fa902c932299e6bfd710111ee2fed/func/nft-collection-no-dns.fc#LL48C9-L48C19) and that we do have it, so it's an easy piece.

**File: utils/address.go**

```go

package utils

import (
	"crypto/sha256"
	"math/big"

	// other imports as well …
)

// … other piece of code goes here

func itemIndex(tokenName string) *big.Int {
	h := sha256.New()
	h.Write([]byte(tokenName))
	hash := h.Sum(nil)
	index := new(big.Int)
	index.SetBytes(hash)

	return index
}

```

Fetching the address can be done in this way

```go

package utils

import (
	"context"
	"crypto/sha256"
	"fmt"
	"math/big"
	"time"

	"github.com/xssnick/tonutils-go/address"
	"github.com/xssnick/tonutils-go/ton"
)

const (
	MainnetConfig = "https://ton-blockchain.github.io/global.config.json"
	MaxRetries    = 3
	Timeout       = 50 * time.Millisecond
)

var (
	AnonymouAddress = address.MustParseAddr("EQAOQdwdw8kGftJCSFgOErM1mBjYPe4DBPq8-AhF6vr9si5N")
)

func GetAnonymousNumberAddress(ctx context.Context, api *ton.APIClient, tokenName string) (string, error) {
	index := itemIndex(tokenName)

	nftAddr, err := numberAddress(ctx, api, index)
	if err != nil {
		return "", err
	}

	return nftAddr, nil
}

func itemIndex(tokenName string) *big.Int {
	h := sha256.New()
	h.Write([]byte(tokenName))
	hash := h.Sum(nil)
	index := new(big.Int)
	index.SetBytes(hash)

	return index
}

func numberAddress(ctx context.Context, api *ton.APIClient, itemIndex *big.Int) (string, error) {
	tries := 0
START:
	block, err := api.CurrentMasterchainInfo(ctx)
	if err != nil {
		return "", fmt.Errorf("CurrentMasterchainInfo::%v\n", err)
	}
	tries++

	res, err := api.RunGetMethod(ctx, block, AnonymouAddress, "get_nft_address_by_index", itemIndex)
	if err != nil {
		if lsErr, ok := err.(ton.LSError); ok && lsErr.Code == 651 && tries <= MaxRetries {
			fmt.Println("Sleeping ", Timeout)
			time.Sleep(Timeout)
			goto START
		}

		return "", fmt.Errorf("RunGetMethod::%v\n", err)
	}

	cs := res.MustSlice(0)
	addr := cs.MustLoadAddr()	

	return addr.String(), nil
}

```

Given that [tonutils-go](https://github.com/xssnick/tonutils-go) implements connection pool, it can be the chance that some block is not yet applied on one of the nodes, so it can lead to errors. In order to handle that I set a max retries policy of three times. A better approach would be to stick your connection to one node, but that requires you to have a ton node running, that's too expensive for us.

_note: I'm poor feel free to donate to this address if you are going to use this script_

`EQC9n6aFb2oxQPMTPrHOnZDFcvvC2YLYIgBUms2yAB_LcAtv`

_Stop asking for money bro, no one will donate a shit…_

Ok, to summarize we now have the item address in the blockchain, so we can now interact with the smart contract to buy this NFT.

```go
package utils

import (
	"context"
	"fmt"
	"log"
	"strconv"
	"time"

	"github.com/xssnick/tonutils-go/address"
	"github.com/xssnick/tonutils-go/tlb"
	"github.com/xssnick/tonutils-go/ton"
	"github.com/xssnick/tonutils-go/ton/wallet"
	"github.com/xssnick/tonutils-go/tvm/cell"
)

const (
	// another declaration goes here…
	nanoTons     int64 = 1e9
)

func buyNumber(
	ctx context.Context,
	w *wallet.Wallet,
	tokenAddress string,
	value int,
) error {
	nftAddr, err := address.ParseAddr(tokenAddress)
	if err != nil {
		return err
	}

	transfer := wallet.SimpleMessage(nftAddr, tlb.MustFromTON(strconv.Itoa(value)), nil)

	return w.Send(ctx, transfer, true)
}

// code related to auction of numbers goes here…

func BuyNumbers(
	ctx context.Context,
	w *wallet.Wallet,
	api *ton.APIClient,
	numbers []AnonNumbers,
) error {
	// check available balance
	block, err := api.CurrentMasterchainInfo(ctx)
	if err != nil {
		return fmt.Errorf("CurrentMasterchainInfo::%v\n", err)
	}

	balanceCoins, err := w.GetBalance(ctx, block)
	if err != nil {
		return fmt.Errorf("GetBalance::%v+\n", err)
	}

	balanceStr := balanceCoins.TON()
	balance, err := strconv.ParseFloat(balanceStr, 64)
	if err != nil {
		return err
	}

	// check if we have enough balance
	if enoughBudget(numbers, int(balance)) {
		for _, number := range numbers {
			if !number.OnSell {
				err = buyNumber(ctx, w, number.Address, number.Value)
				if err != nil {
					return err
				}

				// add this number into db maybe
				log.Printf("Number %s bought\n", number.Name)
			}
		}
	}

	return nil
}

// more code go here…

```

Almost done, in order to see the whole code I recommend you to clone this [repo](https://github.com/Gealber/blog_recipes). As always I won't put the whole code here, because it doesn't make sense, as a developer you should get used to reading other people's code. So feel free to dig into that repository.



## Conclusion
The combination of these two tools can be quite amazing. Try it yourself, feel free to reach me out on any doubts. Bye 👋.