Stating the Obvious...

TheSuburbanLetdown · Post by **TheSuburbanLetdown** » Wed Jun 06, 2007 7:35 am

Server's broken.

Webkilla · Post by **Webkilla** » Wed Jun 06, 2007 9:20 am

theSuburbanLetdown wrote:Server's broken.

oh snap...

why?

Sphix · Post by **Sphix** » Wed Jun 06, 2007 9:32 am

lgo wrote:Ok, I just went to update my comic, and there are nearly 4000 pending requests. What's going on?

webkilla wrote:oh snap...

why?

AutoKeen got blocked and backed up. Think of it as comic pubic hair that has clogged the drain of ComicGenesis, and now the Drano-Admins must dissolve said pubes to get everything flowing again.

Kirb · Post by **Kirb** » Wed Jun 06, 2007 11:10 am

NOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO

Dice · Post by **Dice** » Wed Jun 06, 2007 12:29 pm

Sphix wrote:
webkilla wrote:oh snap...

why?
AutoKeen got blocked and backed up. Think of it as comic pubic hair that has clogged the drain of ComicGenesis, and now the Drano-Admins must dissolve said pubes to get everything flowing again.

Ah, you have such a way with words, Sphix.

Lol! At least it gets the message across.

Shield · Post by **Shield** » Wed Jun 06, 2007 12:38 pm

Sphix wrote:AutoKeen got blocked and backed up. Think of it as comic pubic hair that has clogged the drain of ComicGenesis, and now the Drano-Admins must dissolve said pubes to get everything flowing again.

That is the best metaphor for the sever getting backed up, ever.

Nathalie · Post by **Nathalie** » Wed Jun 06, 2007 1:56 pm

Alright, the updating server is down... But could I at least change my index.html file manually ?
I don't know if any of you tried, but it won't let you modify it. And I can't access my .htaccess file to change that either.

I understand that since many newly registered might not read instructions and change that index file instead of using properly the updating server and all, by default the index file is locked.
But since the server is down, I'm stuck with whatever thing is on the index page, and I really don't like it at all.

Is there anybody who could give me control to my index.html file around here ? I swear I won't mess things up

Webkilla · Post by **Webkilla** » Wed Jun 06, 2007 3:40 pm

Sphix wrote: AutoKeen got blocked and backed up. Think of it as comic pubic hair that has clogged the drain of ComicGenesis, and now the Drano-Admins must dissolve said pubes to get everything flowing again.

why i do i keep thinking of THIS when I read that?

Jackhass · Post by **Jackhass** » Wed Jun 06, 2007 5:26 pm

*Siiiiiigh*...here we go again.

Perk_daddy · Post by **Perk_daddy** » Wed Jun 06, 2007 5:34 pm

I think the Drano-Admins are working; it's down from 4000 to 3700+

LauraSeabrook · Post by **LauraSeabrook** » Wed Jun 06, 2007 8:45 pm

* SIGH *

Sphix · Post by **Sphix** » Wed Jun 06, 2007 9:06 pm

Kirb wrote:NOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO

Yup!

dice wrote:Ah, you have such a way with words, Sphix. Lol! At least it gets the message across.

I try!

Shield wrote:That is the best metaphor for the sever getting backed up, ever.

Thankee! ^_^

webkilla wrote:why i do i keep thinking of THIS when I read that?

Because that page was made of WIN. <3

And now it's flowing again! All it takes is a little humor, eh?

Mycohl · Post by **Mycohl** » Wed Jun 06, 2007 11:14 pm

Nathalie wrote:could I at least change my index.html file manually ?
I don't know if any of you tried, but it won't let you modify it.

Using FTP, first delete the file in public_html, then upload a new one. You won't get any tag substitutions, but you can put those in manually if you like.

Oarfly · Post by **Oarfly** » Thu Jun 07, 2007 1:11 am

The queue's now unclogged, I think. Woo.

Dr Neo Lao · Post by **Dr Neo Lao** » Thu Jun 07, 2007 5:01 am

Mycohl wrote:Using FTP, first delete the file in public_html, then upload a new one. You won't get any tag substitutions, but you can put those in manually if you like.

Close.

To manually change the index: (Temporary fix that shouldn't be done)

Use FTP to connect to your account.
Save the current index.html file from the public_html folder to your computer.
Edit index.html - you should only change news items and not add new comics (except maybe one-off filler or guest art) as doing it manually can potentially create a lot of problems.
Upload the edited index.html file to the public_html folder.
You do not trigger any updates, it'll start showing up straight away.

If you want to put filler / guest art / apology image on the index then upload the image file to the public_html/images folder and put in the link as:

Code: Select all

<img src="/images/filename.jpg">

But be very wary of trying to update your comic manually this way, especially if you're trying to maintain the integrity of your archives. Also note that if you change your index in this manner it will revert back on the next update.

Post by **Kisai** » Thu Jun 07, 2007 10:18 am

To answer your questions:
1. There has been no server move, I have no idea what Kelly is talking about.
2. All these problems started cropping up exactly the same time every day
3. We've been demanding a new server for years, and still have not received one.
4. You can't put a warning message up on the front page when the server can not be reached or is timing out. Think about it.

Okay smartypants types.

http://guide.comicgenesis.com/images/cpuload.png
Compare with
http://guide.comicgenesis.com/images/cpuavg.png

CPU Load is how heavily the cpu is loaded, 100% = bad
CPU Average is how much time is spent loaded. So a 1.0 means one processor is fully loaded, a 2.0 means both processors are fully loaded, and any number above 2.0 means that the system is over loaded.

Anyone care to guess why or what is happening? As of 11:18AM PDT

Nathalie · Post by **Nathalie** » Thu Jun 07, 2007 10:56 am

Thanks a lot Mycohl and Dr Neo Lao !
The server was back, but I still tried it anyway

Mycohl · Post by **Mycohl** » Thu Jun 07, 2007 12:04 pm

Dr Neo Lao wrote:Close.

No. Completely factual, accurate and an answer to the question asked.

Dr Neo Lao wrote:
Code: Select all
<img src="/images/filename.jpg">

Since we are being condescending and nitpicking, that doesn't conform to XHTML/XML. You need to close it with />.

Kisai wrote:All these problems started cropping up exactly the same time every day [...] Anyone care to guess why or what is happening? As of 11:18AM PDT

What happens when you "ps -e --sort %cpu" at 11:19am? Is it all httpd freaking out?

Post by **Kisai** » Thu Jun 07, 2007 12:46 pm

Mycohl wrote:
Dr Neo Lao wrote:Close.
No. Completely factual, accurate and an answer to the question asked.
Dr Neo Lao wrote:
Code: Select all
<img src="/images/filename.jpg">
Since we are being condescending and nitpicking, that doesn't conform to XHTML/XML. You need to close it with />.

Kisai wrote:All these problems started cropping up exactly the same time every day [...] Anyone care to guess why or what is happening? As of 11:18AM PDT
What happens when you "ps -e --sort %cpu" at 11:19am? Is it all httpd freaking out?

Well the problem prior today was that at 4:55AM on the dot, it would go into a race condition. I'm really want to blame postfix.

I was able to replicate the race condition when I resynced the replication, as mysql on the main server just pretty much stopped responding. Seeing as how it's a multithreaded server on a not very multithreaded friendly OS, there was no way to get the logs to work properly, so it was either throw the logs away or write them to mysql and then pull them out of mysql by domain later. Guess what runs at 4AM? The log splitter/webalizer.

When viewing message logs, all I could see were errors from postfix, rapidly, so I poked Kelly several times to get that fixed. That stopped happening sometime on the 5th. No race condition at 4:55AM today.

No instead a breif outage was caused by resyncing replication, which requires locking the tables, and hence locking apache out of writing logs. I've made a few modifications to the mysql logging wrapper so it just throws away the log entry if it can't reconnect to the database, that should stop apache from freezing solid, but that won't do anything for autokeen, ftp and siteadmin if it's mysql that's being broken.

And no, upgrading to Mysql 5.x is not a possibility due to the OS still running FreeBSD 4.x Both mysql and freebsd are running depreciated versions that are no longer in active development, and as such we're stuck till we get a new server or at the very least the OS is replaced. Replacing the OS will take the server down for a VERY LONG time. Likewise replacing the server would take a very long time.

Mycohl · Post by **Mycohl** » Thu Jun 07, 2007 4:53 pm

Kisai wrote:No instead a breif outage was caused by resyncing replication, which requires locking the tables, and hence locking apache out of writing logs. I've made a few modifications to the mysql logging wrapper so it just throws away the log entry if it can't reconnect to the database, that should stop apache from freezing solid, but that won't do anything for autokeen, ftp and siteadmin if it's mysql that's being broken.

Why is that mysql being broken? Autokeen should sleep if it can't touch the db for however long it takes to sync and unlock; if that is what is happening, I wouldn't call that broken. I haven't experienced ftp or siteadmin blocking at all, even when CG's httpd was hung/down. I'm sorry I can't be of any help.