uBot

uBot is a drag-and-drop scriping application with which you can easily build Web robots.

Scraping ISP Data Usage With uBot Studio

bot_01.jpg

This is a pretty easy way of scraping data from a website, but it does cost about $300. You can accomplish the same thing in Python, the catch being is that you have to know Python... so for those of you who don't, uBot is great.

In this tutorial I will be using AT&T as an example, but with some adjustment it should be easy enough to adapt it for any ISP that gives you a running total of your monthly data usage.

By the end of this tutorial you will have a bot that can be fire automatically to log into your AT&T control panel, scrape the current data usage, and write that data into a file called "data.txt" on your computer's hard drive.

You MUST convert the data.txt file to UNIX before InfluxDB will accept it.

This is the first of four sub-tutorials that will enable you to pull data from a web page and post it into InfluxDB.

This is a Windows specific tutorial.

Once you have completed this tutorial, please see the following tutorial:

Dependencies:

1. Download uBot Studio

2. Create the uBot script

Since I'm using the Community edition of uBot studio, I can't share my projects or export the script to an executable file, but I can give you the steps that I used to pull my data usage.

In this example, when I began creating the bot, I had already opened uBot, navigated to AT&T and logged in (https://www.att.com/my/#/login). This reason for this is that uBot's built in Chromium browser will remember my e-mail address, a feature that we can use to make the script shorter. Once you've manually visited and logged into AT&T, log out again.

At this point you should now have a text file on your hard drive titled "data.txt". This is the file that you can import into InfluxDB via the command line, however you CANNOT import this file in its current state on a Windows computer - you MUST convert it to UNIX format before InfluxDB will accept it.

You MUST convert the data.txt file to UNIX before InfluxDB will accept it.

Once you have completed this tutorial, please see the following tutorial: