WarcraftReamls.com
  FAQFAQ    SearchSearch    MemberlistMemberlist    UsergroupsUsergroups   RegisterRegister 
  ProfileProfile    Log in to check your private messagesLog in to check your private messages    Log inLog in 
Census display here on site.

 
Post new topic   Reply to topic    WarcraftRealms.com Forum Index -> WarcraftRealms.com Bugs
View previous topic :: View next topic  
Author Message
bringoutyourdead
Forums Admin & general flunky


Joined: 07 Nov 2008
Posts: 1261
Location: Silicon Valley, CA, USA
WR Updates: 9,204,147
bringoutyourdead WR Profile

PostPosted: Mon Aug 22, 2016 3:11 pm    Post subject: Census display here on site. Reply with quote

The display files have been updated to full Legion settings
Demon Hunter class added
Levels adjusted 20 -> 110

selecting via the top 50 guilds will provide data... but I haven't a clue if that data will be accurate.. maybe yes, maybe no.


Last edited by bringoutyourdead on Tue Apr 17, 2018 3:48 pm; edited 1 time in total
Back to top
View user's profile Send private message
bringoutyourdead
Forums Admin & general flunky


Joined: 07 Nov 2008
Posts: 1261
Location: Silicon Valley, CA, USA
WR Updates: 9,204,147
bringoutyourdead WR Profile

PostPosted: Tue Sep 06, 2016 5:49 pm    Post subject: Reply with quote

Another good news, bad news, other news day.

I found (again) the process that updates the data that feeds Census.php pages.
This data currently gets updated once a day early AM hours site time.
After discussion with Metalbeast, I have gone from gentle nag mode to active fix mode.
I intend to cut the lag time for this data down from the as much as 24 hours to hopefully 3 to 6 hours.
This final update process is time consuming and some what computer resource intensive.

I have three conflicting goals, Metalbeasts wants and needs concerning site performance and requirements, Users wants and needs for more and better data then we can provide, and my absolute limits of what I will and won't do to balance the other needs.

The processes that update the Census.php page also cleans out data, but not in a way I find acceptable.. Nor that Metalbeast found acceptable (i.e. currently it is broken burning cpu cycles and doing no real work Rolling Eyes )

The changes that are going to be made will be gradual, but probably faster then the two very old level 20 characters that were removed from the data today. (After seven hours of research and testing.)

Note old data that has already been removed can not be replaced.. just can not happen .. sorry about that.

All characters level 20 and below will be gone some day.
My current testing has a last seen cut off date of the day before Cataclysm release.
This will very slowly cycle forward up to the current expansion release date.

There are currently over 2 million characters at level 20, I expect many of them are starter accounts and will never go higher.
I do not want them in the database.
Even if most of my characters are in that group.

Blizzards decision to allow inactive character names to be grabbed and reused is one of my clean out points.
If someone on a realm grabs an inactive name and uses it on the same faction/race/class it is hard to know that it is a new player behind the curtain.
Levels 21 through (yet to be determined) that have not been active since the cutoff Blizzard uses above will be removed based on a level-time curve (also yet to be determined.)

When I get to it, the 293 millions rows of history data will also be shrunk.
Again it will be done based upon a level-time curve (nothing more then a vague idea so far.)
At a minimum the existing x5/x0 records will be kept.
More active characters will have more current records kept.
Less active characters will more likely find lower non-x5/x0 records that still exist going away.
Same level records where the character goes in guild/out of guild/ back in same guild.. will have the out of guild records removed.
And I have no clue what else will happen, but I hope it ends up being the least displeasing to all parties.
Back to top
View user's profile Send private message
bringoutyourdead
Forums Admin & general flunky


Joined: 07 Nov 2008
Posts: 1261
Location: Silicon Valley, CA, USA
WR Updates: 9,204,147
bringoutyourdead WR Profile

PostPosted: Wed Nov 02, 2016 5:19 pm    Post subject: Reply with quote

The first batch of level 20's that have not been seen since mid 2010 have finally been purged.

Almost 718,000 dead characters were removed, while at the same time around 1.5 million new characters were added.

the 2nd pass where level 20 (and below {if any are left}) characters that have not been seen since before the last Mists of Pandaria patch, and who appear to never have been in a guild (a good sign these were abandoned start account characters), has started.

There may be a 3rd pass, depending on Blizzards next announcement for name reclaiming.

The now over 299 million history records will soon start shrinking as we start clearing out zombie records for characterIDs that no longer exist.

I figured out fairly quickly how to clear out all those dead records... but since I do not want to lock users out of the site for minutes to hours at a time.. it took a bit of work to come up with a viable solution.

Starting later this week or next week, you will start to see the history record number start to shrink, slowly.
I expect the full first pass on this to take as much as 45 days.

Since this is just to remove worthless records from the table, no valid characters will see any reduction in their history information.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    WarcraftRealms.com Forum Index -> WarcraftRealms.com Bugs All times are GMT - 6 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
WarcraftRealms.com  


Powered by phpBB © 2001, 2005 phpBB Group