time between submission

General Discussion Area
Post Reply
superxdude
Census Taker
Posts: 76
Joined: Tue Nov 18, 2008 10:14 am
Location: Hawaii

time between submission

Post by superxdude »

Just wondering if the minimum time between submission has been lowered or will be with demon hunters coming online

Kanegasi

Post by Kanegasi »

Why would you need it lowered? Just upload once a day.

User avatar
bringoutyourdead
Forums Admin & general flunky
Posts: 1432
Joined: Fri Nov 07, 2008 1:11 pm
Location: Texas, USA

Post by bringoutyourdead »

There are a couple of misunderstandings with the above posts.
And having finished this post I realize that I have gone into too much info mode.
If you are not interested in site history, go to the bottom of the page where the BOLD text answers the above questions.

While I am not the final decider on how this site is run, I do have some insight into issues presented to sites like www.warcraftrealms.com

In the old days of Vanilla Wow and on through Wrath, Rollie had this site on minimal hardware.
I don't know for sure, but I expect he had a business level fixed IP Internet account and ran the site from his home. He might have had a machine at his local ISP, but I doubt that due to the extra costs of having hardware in the ISP hardware cages. (I did the same thing in those days.)

Either way, with limited disk storage and bandwidth availability, Rollie had to protect against abusive denial of service attacks.
Considering we have the ability to push megabytes of data up to the website on every upload, having nearly unlimited uploading from the same IP addresses could and would in Rollie's day have filled the upload storage area and blocked the site from valid uploads, and might have crashed the site.

As such Rollie had implemented a set of rules that no IP address could tie up more storage then one upload file every 60 minutes.
This protected storage overflow but not bandwidth issues, all he could do for that was after the fact blocking of offending IP addresses.
There are many addresses that are blocked from communicating with this site.

When Rollie handed over the ownership of Warcraftrealms.com to Metalbeast, he did so because he knew the new owner had the financial standing and ISP technical connections to keep the site running.
Under Metalbeast some of the rules and constraints changed.

Metalbeast is the owner of an American muliti-regional ISP, as such he has access to ISP owned and controlled machines at data centers in a number of areas of the country.
Running Warcraftrealms.com is not cheap. The book price assigned to our class of machine is in the hundreds of dollars per month.
The only reason we still have a website is that Metalbeast agrees with us that a site of this nature is needed and that he can limit the real cost of the machine by having us share the machine with , development, marketing, community (non-profit, charity, etc.) sites.
Looking at the stats our site consumes more then 80% of the total bandwidth for all the sites associated with the machine, and over 85% of all the data storage.

Metalbeast has all the same time constraints that we all have. Real Life does impact our available free time.
He also has the time constraint of running a business (which is not a 9 to 5 time frame), as such time is his most limited resource.

In the last three years, Metalbeast after seeing the real resource requirements of Warcraftrealms.com has moved our site multiple times.
The site has been housed in at least three data centers, and currently is on a machine that Rollie could never have envisioned.
The ram memory alone is greater then the total ram and disk storage of the best machine Rollie had for the site.

The processing of upload is handled by a system level job submitter (cron for those who understand unix/linix stuff.)
The cron job has always been setup to run once per hour.
There were a few times in the bad old days where the current job was still processing uploads when the next cycle was set to trigger.
With the better/larger multiprocessor server and the massive ram memory, I have never seen upload processing to take more the a few minutes. Often times less then one minute.

As to the question from superxdude.. we have lowered the time period between submissions.
As noted in the announcements, I have asked that you submit no more frequently then once every 5 minutes.. 12 times more often then what Rollie could allow.

This was a off the cuff decision and will be fine tuned by Metalbeast in consultation with his Network Operation Center (NOC) people who monitor our machine along with all the others 24/7.

As to the question from Kanegasi.. some of our uploaders run multiple instances of World of Warcraft just to run CensusPlus.
They literally can not wait and upload only once per day as they would run close to the 8MB web server software limit for uploading.
Some of us use UniUploader (3rd party software) to get around that limit, but others prefer to not use that extra software.

And to be honest, if you are interested in being on the power uploaders list you want to upload more frequently then once a day.
The first person who uploads data that indicates a new character or a new level for an existing character gets the credit.

superxdude
Census Taker
Posts: 76
Joined: Tue Nov 18, 2008 10:14 am
Location: Hawaii

Post by superxdude »

Thank you BYOB. I use 4 accounts to gather census data.

I will not abuse it.

SuperX

Post Reply