Stats page not updating. [Sorted]

[Kisai]

Google Analytics is actually much better than what we had before. I like it.

This is what my HTML looks like that got it to work:

<div class="keen_ad" style="display:inline;">
***advertisement***
<script type="text/javascript">
var pageTracker = _gat._getTracker("XX-XXXXXXX-X");
pageTracker._initData();
pageTracker._trackPageview();
</script>
</div>

Obviously this has the HTML around the script (which is only the section labeled script, naturally) and you would have to replace the Xs with you actual numbers. This is just showing where it fits in the HTML.
Is this the correct placement or should it be somewhere else?

Yes that will work. Those three lines in the javascript are the important ones.

[Drsaltine2]

Okay. I've ignored commenting about the stats not updating for a while. I was willing to let this drop because of Comicgenesis's reputation of unreliabilty and bugs. I was willing to shrug it off and let the a few weeks pass because I am too lazy to figure out how to do my own hosting and learn CSS and Comicspress and actually run my webcomic as if I was serious about it.

I don't care how accurate CG stats are, but they were at least consistent stats that I could follow. And, despite the admin's estimation, there are plenty of people who use the stats system to monitor their stats. It is not accuracy I am looking here, but precision.

And to drop all these changes without warning...

[Kisai]

Okay. I've ignored commenting about the stats not updating for a while. I was willing to let this drop because of Comicgenesis's reputation of unreliabilty and bugs. I was willing to shrug it off and let the a few weeks pass because I am too lazy to figure out how to do my own hosting and learn CSS and Comicspress and actually run my webcomic as if I was serious about it.

I don't care how accurate CG stats are, but they were at least consistent stats that I could follow. And, despite the admin's estimation, there are plenty of people who use the stats system to monitor their stats. It is not accuracy I am looking here, but precision.

And to drop all these changes without warning...

There is no accuracy or precision when the data may be served from a different server or even by ipv6. (Yes CG is available by ipv6, the version of webalizer on CG doesn't recognize ipv6, and the newest version would require backporting all the xhtml changes again to make it work in the siteadmin.)

But overall keeping the logs necessary to support webalizer has been at the expense of being able to maintain the site. If you haven't noticed, there were sometimes 30 seconds of lag before CG was reachable if you hadn't recently visited a cg site, this was due entirely do the logging and webalizer process. Once I turned it off, tada, ezspeedy.

A site as large as CG either has to spread out to multiple servers, at the expense of tracking logs. If we use edge caching, all hits to the edge servers won't even register a log event. (Akamai anyone?) If we buy additional servers ala wikipedia or google, the logs would have to be turned off anyway.

The log files really should have been turned off in 2007 (Which was almost entirely responsible for the outages.) But I figured the problem would go away after getting a new server. No it just delayed the inevitable.

Here's an outline of what goes into webalizer:
1. Visitor visits the site
2. Apache logs this to a single file, eg webserver.log, apache does not turn over logs, and does not create log files after it has started.
3. This requires a plugin script to split split the log file into the virtual hosts, or a piped log script to split the logs on demand
(Unfortunately no script in existence does this properly on a multi-threaded environment, I wrote my own script for it.)
4. Then those files must be aggregated and the content sorted by date (especially on a multithreaded environment) which requires a lot of RAM on the system
5. Then those files must be run through webalizer, chronologically. Or the pipe must be run through webalizer. We've tried both, and webalizer uses hundreds of MB's of ram in doing this. But it's single threaded.
6. The output data is then saved and html files made available in siteadmin.

The setup on keenspot is nearly the same, except that webalizer runs every 30 minutes to keep the size of the log files down. This isn't even remotely possible on CG, as the overhead required just to run webalizer on 15000 sites with no log files to process takes more than 30 minutes. Hence the statement earlier that webalizer isn't completing before it has to be run again.

All large sites eventually throw away conventional logging and eventually switch to sampling instead. Even google analytics will switch to sampling on you if you exceed a certain amount of traffic.