Server Performance Issues
View previous topic | View next topic >
Post new topic Reply to topic
M.A.I.L. Forum Index -> Knitting Circle
   
Author Message

Joined: April 29, 2002
Posts: 3212
Submissions: 93
Location: Albany, New York

Server Performance Issues
Reply with quote
Posted on Thu Nov 10, 2005 4:01 pm || Last edited by Blaise on Mon Jan 30, 2006 9:37 pm; edited 2 times in total
Link to Post: Link to Post

Hi Folks,

there have been a lot of posts on this, so I'm going to stick this for visibility.

Here, hopefully for the last time, I will explain the issue:

1) Our site is horribly complex, and overuses server resources significantly.

2) Our database (including discussion posts, gallery submissions, etc) is very large, and we can't really reduce its size without throwing away posts or submissions.

3) Our database server is very slow, and every page on the site has to access it, so our entire site is slow.

4) Our webhost is completely uninterested in helping us discover the problem. Their final word on this issue, after three weeks of arguing, is "It must be something your site is doing.".

5) We would very much like to move to a new webhost, but certain issues stand in our way:
a) See point 1 - this site is built from a frighteningly convoluted amalgam of php,perl,mysql,and html code. Trying to move it without losing any data is almost impossible.
b) Even if we knew HOW to move it, the amount of work involved is stupendous (I'd estimate 3 man-weeks), and since I'm pretty much the only tech-guy around, and I can really only manage an hour or two a week of real technical work on the site, that means we'd be ready to go in 60 weeks, or just over a year.
c) Even if we knew how AND had the manpower, without redesigning the site to stop it from doing whatever it's doing "wrong", it would just start doing it again at the new host, and we'd end up in exactly the same boat, a year down the line.

6) Given 1-5, the only solution I can see is to find an actual, professional database expert who can troubleshoot our database for us, so I can find the root cause of the problem. I am currently attempting to find said expert, and trick, erm, I mean persuade, them to volunteer the time to look into it for us...

I apologize for the hardships in the meantime, but there IS work being done on the problem, even if there's been no percieved change in the site's performance, or lack thereof.

In the meantime, if you get page denied, zero length document, or connection refused errors from MAIL, please try again in a minute or two, and you should get in. It seems to make getting into the fora easier if you go directly to the index, instead of via the M.A.I.L. homepage, and then move around from there. - http://www.mailleartisans.org/board/index.php


"The fact that a believer is happier than a skeptic is no more to the point than the fact that a drunken man is happier than a sober one."

George Bernard Shaw
___________________________________

Maille Code V1.0 T5.7 R5.1 Fhd MCu Wc Cd G2.03/.56 I9.75/3.25 Pn Dacdjs S97 CCi

Joined: March 3, 2002
Posts: 4372
Submissions: 79
Location: tres piedras, new mexico

Reply with quote
Posted on Fri Nov 11, 2005 2:12 pm
Link to Post: Link to Post

i'll ask around and see if i can't rustle up some sort of expert.. maybe other members know someone who might be able to help on a volunteer basis also.

thanks for being here for us, mr. runsitall.
Smile

kim


PSA: remember to stretch.
3.o is fixing everything.

Joined: April 15, 2002
Posts: 1819
Submissions: 1
Location: Calgary, AB. Canada.

Reply with quote
Posted on Sat Nov 12, 2005 5:16 am
Link to Post: Link to Post

Yeah, thanks a lot for all the support Blaise, and for fighting to solve this. A big kick in the nuts to our host.

While your shanghai'd help is at it, maybe someday, someday, we could talk him/her into cleaning up the code or making some changes, so that things run better Smile

I know some people, but they're very.. profit.. oriented.

*sigh*

If we had a better bank account it would be nice to be able to pay someone for this, but, we'd be looking at $40-$80/hour.. and, like you said, 60 hours just to move the site... that'd take $10 from practically every member Sad

Joined: December 29, 2004
Posts: 896
Submissions: 19
Location: North jersey

Reply with quote
Posted on Sat Nov 12, 2005 4:04 pm
Link to Post: Link to Post

Here's an idea: Try and raise money so we can get the profenssional in here. I'm sure that many members would be more than willing to give a couple bucks to a good cause. Smile


http://chainmaillebookproject.ninjatails.com/index.php --Please contribute! Existing members, put this in your signature to increase publicity!

Joined: March 22, 2005
Posts: 12
Submissions: 3

Reply with quote
Posted on Sat Nov 12, 2005 6:25 pm
Link to Post: Link to Post

Quote:
Here's an idea: Try and raise money so we can get the profenssional in here. I'm sure that many members would be more than willing to give a couple bucks to a good cause. icon_smile.gif


I'm glad to see someone else thinks getting a professional web geek is a good idea. Might want to look at the mirror thread @ TRL forums, it's been discussed more heavily there.

Joined: March 3, 2002
Posts: 4372
Submissions: 79
Location: tres piedras, new mexico

Reply with quote
Posted on Sun Nov 13, 2005 1:14 am
Link to Post: Link to Post

a pro is being sought out, from my understanding..

kim


PSA: remember to stretch.
3.o is fixing everything.

Joined: July 13, 2004
Posts: 97
Submissions: 6

Reply with quote
Posted on Mon Nov 14, 2005 4:39 pm
Link to Post: Link to Post

TipO MasTr wrote:
Here's an idea: Try and raise money so we can get the profenssional in here. I'm sure that many members would be more than willing to give a couple bucks to a good cause. Smile

There happens to be at least one professional web programmer in this group. (points to self) I wouldn't mind the work, or could justify some time to add this site to my resume.
As a start, processing images to improve compression or shrink oversize images could probably help.

Joined: April 29, 2002
Posts: 3212
Submissions: 93
Location: Albany, New York

Reply with quote
Posted on Mon Nov 14, 2005 9:22 pm
Link to Post: Link to Post

Actually, our images are no problem at all, we are well within our limits both for disk quota and bandwidth. Our issues are strictly database related (Well, that and the overall design of the site). What we need is someone who could first determine what is slowing down the DB so much (so we don't do that any more), and then redesign the site with that information in mind.

I myself simply speak a little PERL, PHP, html, and MySQL, so while I might be able to manage the re-design/implementation, I can't figure out the damned database problem! What we need is a MySQL wizard who speaks a bit of PERL, PHP, and html.

If you are that wizard, by all means, let's talk...


"The fact that a believer is happier than a skeptic is no more to the point than the fact that a drunken man is happier than a sober one."

George Bernard Shaw
___________________________________

Maille Code V1.0 T5.7 R5.1 Fhd MCu Wc Cd G2.03/.56 I9.75/3.25 Pn Dacdjs S97 CCi

Joined: July 13, 2004
Posts: 97
Submissions: 6

Reply with quote
Posted on Tue Nov 15, 2005 6:04 pm
Link to Post: Link to Post

Blaise wrote:
...Our issues are strictly database related...

Much more a perl wizard than an SQL wizard, but get me a look at the database table layout and queries and I might come up with something. Probably no need for me to see actual data. Making a blind guess, the tables don't have key fields or keys aren't used well in queries. mySQL tends to be much faster when a query is based on primary keys rather than generic fields.

Joined: April 29, 2002
Posts: 3212
Submissions: 93
Location: Albany, New York

Reply with quote
Posted on Tue Nov 15, 2005 8:07 pm
Link to Post: Link to Post

Funny you should mention that, I just spent the morning adding column indices to the db tables to try to speed things up. Please, everyone, report any behaviours you see that are WORSE than they have been, but also let me know if this makes things BETTER...


"The fact that a believer is happier than a skeptic is no more to the point than the fact that a drunken man is happier than a sober one."

George Bernard Shaw
___________________________________

Maille Code V1.0 T5.7 R5.1 Fhd MCu Wc Cd G2.03/.56 I9.75/3.25 Pn Dacdjs S97 CCi

Joined: March 31, 2004
Posts: 254
Submissions: 8

Reply with quote
Posted on Wed Nov 16, 2005 2:07 am
Link to Post: Link to Post

Hopefully it isn't just the time of day, but things are loading a LOT faster. Smile Smile Smile

Joined: February 29, 2004
Posts: 1898
Submissions: 8
Location: Behind Enemy Lines in Marxistchussetts

Reply with quote
Posted on Wed Nov 16, 2005 5:59 am
Link to Post: Link to Post

Never seen things load so fast. Coif Cool Smiley We need the :cheers: smily they have over at TRL! Thanks Blaise!


"Do not go where the path may lead,
go instead where there is no path and leave a trail."
Ralph Waldo Emerson

My blog: The (New) Eclectic Dragonfly. Some of my stuff at CafePress!

Joined: September 02, 2003
Posts: 2016
Submissions: 8
Location: Sydney, Australia

Reply with quote
Posted on Wed Nov 16, 2005 11:03 am
Link to Post: Link to Post

Last couple of days have been fine for me...no problems at all. Whatever you did it worked.

Siege


John Lennon once sang a song where he imagined a world with no religion at all, and how all the people would be living life in peace, which sounds like a pretty terrific philosophy to follow - but then again, John Lennon also sang a song about how he was an eggman and a walrus goo goo g'joob, so i'm not sure he was always 100 per cent reliable.

Joined: April 15, 2002
Posts: 1819
Submissions: 1
Location: Calgary, AB. Canada.

Reply with quote
Posted on Wed Nov 16, 2005 11:14 am
Link to Post: Link to Post

Yeah, I'll also have to say that, hot damn, it loads fast now! *clap clap* Well done Blaise. Assuming this, and continuing this procedure, fixes the problems, is there still a plan to move the site to a new host that doesn't suck as much?

Hanelyp - To play the role of recruiter here for a bit, we *really* need coders badly to make improvments to MAIL. Not sure if you've been around long enough, but there's a big wishlist of ways we wish the site worked, but our current coders can't handle that kind of workload.

I'm not sure if you were interested in the work for pay only, but, there's the possibility of using MAIL funds to pay a coder to improve the site too, not just to pay someone to repair it so it can be moved. Of course, free, or discounted would be fantastic too. Would that be something you are interested in helping out with?

Joined: May 07, 2002
Posts: 29
Submissions: 0
Location: Alpharetta, GA U.S.A.

Reply with quote
Posted on Wed Nov 16, 2005 12:57 pm
Link to Post: Link to Post

Sure seems faster to me. Surprised
Great job!

Post new topic Reply to topic
Jump to:  
Page 1 of 3. Goto page 1, 2, 3  Next
All times are GMT. The time now is Thu Oct 23, 2014 9:18 pm
M.A.I.L. Forum Index -> Knitting Circle
Display posts from previous: