Investigating Scalping Options on the LTT Giveaway
Tags: Programming, JavaScript, Nodejs, Bots, Web Development

Photo by Alex Knight on Unsplash

On the 22nd November of 2020, LTT released a video about a Colorful RTX 3080, which they later were going to “give away”. It is in quotes because, in reality, they were going to sell it for one dollar in the next 24 hours. A nice and neat idea.

Linus did, however, say that bots won’t help because the page does not yet exist. This got me thinking — is that really true? If it were that simple then scalping would not be a problem.

The challenge

From the scarce information given it, all boils down to that their merch shop will have some change. Maybe it will be just a new product or perhaps some kind of banner will appear or something completely unimaginable. Spoiler — it was just a product, let us ignore that for now.

As it stands, what we are trying to find seems unpredictable, so a bot might have a hard time purchasing the item, but it can definitely help sense that the site has changed and notify us to take a look.

The solution

It breaks down to a few simple steps:

Go to the site Take a screenshot Compare to the previous screenshot Shoot an email if the screenshots are not the same

Using the NodeJS stack we can utilize a few libraries for completing these steps:

puppeteer — headless browser which can be driven programmatically pixelmatch — pixel-level image comparison library nodemailer — for sending us the emails

There are a few more helper libraries, but the ones mentioned are the gist of the solution.

The code

Ignoring package.json , it required only 175 lines of code for this notify bot to work. All it does is visits 3 pages of their store, save the screenshots, compare them, and shoot emails if something is a miss.

const puppeteer = require("puppeteer");

const fs = require("fs");

const PNG = require("pngjs").PNG;

const pixelmatch = require("pixelmatch");

const winston = require("winston");

const os = require("os");

const nodemailer = require("nodemailer");

require("winston-daily-rotate-file"); const mail = nodemailer.createTransport({

service: "gmail",

auth: {

user: "mail@gmail.com",

pass: "pass",

},

}); const transport = new winston.transports.DailyRotateFile({

filename: "application-%DATE%.log",

datePattern: "YYYY-MM-DD",

maxSize: "20m",

maxFiles: "2d",

}); const logger = winston.createLogger({

format: winston.format.json(),

transports: [transport],

}); const URLS = [

"https://www.lttstore.com/",

"https://www.lttstore.com/collections/all",

"https://www.lttstore.com/collections/all?page=2",

]; const urlToFileMap = {

[URLS[0]]: "home",

[URLS[1]]: "all-p1",

[URLS[2]]: "all-p2",

}; const tDuration = 1000 * 60 * 10;

let counter = 0; async function autoScroll(page) {

await page.evaluate(async () => {

await new Promise((resolve, reject) => {

var totalHeight = 0;

var distance = 100;

var timer = setInterval(() => {

var scrollHeight = document.body.scrollHeight;

window.scrollBy(0, distance);

totalHeight += distance;

if (totalHeight >= scrollHeight) {

clearInterval(timer);

resolve();

}

}, 100);

});

});

} const compare = (url) => {

return new Promise((resolve) => {

try {

const img1 = fs.createReadStream(`./images/${urlToFileMap[url]}-${counter - 1}.png`).pipe(new PNG()).on("parsed", doneReading); const img2 = fs.createReadStream(`./images/${urlToFileMap[url]}-${counter}.png`).pipe(new PNG()).on("parsed", doneReading); let filesRead = 0; function doneReading() {

if (++filesRead < 2) return; if (img1.width !== img2.width || img1.height !== img2.height) {

resolve(0);

} const diff = new PNG({ width: img1.width, height: img2.height });

const numDiffPixels = pixelmatch(

img1.data,

img2.data,

diff.data,

img1.width,

img1.height,

{ threshold: 0.1 }

); resolve(numDiffPixels);

}

} catch (e) {

console.log(e);

}

});

}; const getScreenshotAndCompare = async () => {

const promises = URLS.map(async (url) => {

let browser;

try {

browser = await puppeteer.launch({

executablePath: os.arch() === "arm" ? "chromium-browser" : undefined,

headless: true,

}); const page = await browser.newPage();

page.setDefaultNavigationTimeout(0); await page.goto(url); await page.setViewport({

width: 1200,

height: 800,

}); await autoScroll(page); await page.waitForFunction(

async () => {

return await new Promise((resolve) => {

setTimeout(() => resolve(true), 3000);

});

},{

timeout: 0,

}

); await page.screenshot({

path: `./images/${urlToFileMap[url]}-${counter}.png`,

fullPage: true,|

}); if (counter == 0) {

return;

} const comparison = await compare(url); if (comparison > 0) {

logger.info(url);

await mail.sendMail({

to: "my_email@gmail.com",

subject: "LTT Store",

attachments: [

{

filename: "Previous.png",

path: `./images/${urlToFileMap[url]}-${counter - 1}.png`,

},

{

filename: "Current.png",

path: `./images/${urlToFileMap[url]}-${counter}.png`,

},

],

});

}

} catch (e) {

console.log(e);

} finally {

browser && browser.close();

}

}); Promise.all(promises).then(() => {

counter++;

});

}; getScreenshotAndCompare();

setInterval(getScreenshotAndCompare, tDuration);

This could be improved quite a bit as well, this is somewhat of a rough and fast take on things, a lot of the code is just taken from the examples of the corresponding libraries. The full code can be found here, though the full code already is here.

Result

Deployed the script to my Pi 3 overnight, and in the morning I had quite a lot of emails from myself. Some were false positives, the Pi loads images a lot slower so sometimes they just weren’t fully loaded.

the default

Eventually, the correct change was picked up, yay! If only it weren’t already sold.

the Spanish inquisition

Final thoughts

Given that it turned out that the change was just a new product, this is far from the best approach. Just loading HTML and then doing some xpath magic would have found which product to buy and would have been a lot faster than saving screenshots, especially on the Pi.

The ten-minute interval does not help as well, the first image I got with the GPU was already with the sold out badge, but this was more of an experiment than real participation, I would have been deep asleep when the email hit anyway, so I’m fine with that.

I do not approve of scalpers, but the only real thing we can do is to control ourselves and not buy their goods. Websites have a lot of guidelines to follow, be it for SEO or accessibility, but the guidelines can be easily used by bots to do their evil bidding.

Now it is time to delete all the pics of Linus from my Pi, g’day.