[ADMIN] Future hosting of ILX (Part 8080)

Message Bookmarked
Bookmark Removed

Sorry for the info vacuum of the past few days; i have been trying to get things together to work out some clear options. Here's where we are so far:

- The hosts recommend that "basically you should junk your server; it's old and inefficient and the disks are shot". They're offering us a virtual server for the same price which has less CPU but the same amount of memory and disk. It's not a bad deal -- and ILX is unbelievably light on CPU anyway -- but I also think we should get other quotes, so...

- Have asked a number of colo and virtual server places to quote us for enough power to run ILX. The big issue is memory -- the DB is heading towards 4gb now, most of which the code tries to keep in memory. But virtual hosts are usually quite sparing with the memory. Given that ...

- The memory requirement we're at now is already pretty expensive, and the ILX DB has doubled in size since we switched to Keith-code. If it keeps on growing like that we could eventually hit some relatively expensive memory requirements. Changing this setup would be a big chunk of work, and it might not be necessary, but would also dictate a bit what kind of hosting we're looking for. Surprisingly, among the more reasonable big-memory online hosts are Amazon, so:

- Also looking at running on Amazon's server, using their EC2 service. That's what this sandbox is running on right now (on a smaller, free, server -- the full-size one would cost a fair bit more.) There's some real advantages here, as they've also got great tools and monitoring and can do real-time backups and I can spin up a backup "server" in seconds and it's generally great. But:

- it'll take some code changes to run properly here, as there are some things we do which are incompatible with the Amazon setup, and it could also be expensive. That said:

- Virtually all the options are going to cost some cash. When I get the quotes back (day or two) we should know how much those options will cost, but won't really know how much the Amazon setup will be until we try it out for real. At an guess, I think an upper limit for basic running on any of these options would be $100 a month, which is what we pay now for hosting.

So: Right now I think Amazon's my preferred option, although our hosts have given us really personal service so don't want to rule their virtual server offer out either. I think either of them is better than buying or renting another server and just shifting all our eggs from one basket to another.

I'd like to keep investigating Amazon for at least this week. A couple of people have offered to help with the work I need to get done, which is v. appreciated and I've sent them details of what's still required to get proper ILX running on this setup.

If we can get that done, hopefully in a week or two at most, we'll know pretty quickly what the costs are likely to be. Then, depending which option we go with, it may be cap-in-hand time again. Will keep you posted on what's going on.

stet, Monday, 5 December 2011 21:14 (five years ago) Permalink

<3

mookieproof, Monday, 5 December 2011 21:17 (five years ago) Permalink

thx stet!

surely we can raise whatever money we need fairly quickly

Shakey Mo Collier, Monday, 5 December 2011 21:18 (five years ago) Permalink

thanks for the update!

sleeve sandbox, Monday, 5 December 2011 21:18 (five years ago) Permalink

is sb extra

Sad Banter (p much resigned to deems), Monday, 5 December 2011 21:19 (five years ago) Permalink

Brilliant. I'm away during the cap-in-hand period but am more than willing to stump up before or after.

(^_^) (^_^) (^_^), Monday, 5 December 2011 21:21 (five years ago) Permalink

open ended fundraising pls

Sad Banter (p much resigned to deems), Monday, 5 December 2011 21:22 (five years ago) Permalink

I will definitely donate to the cause when necessary. I has a jobz now!

Johnny Fever, Monday, 5 December 2011 21:22 (five years ago) Permalink

Is the DB plate inevitably going to get too big to keep spinning on a stick?

William (C), Monday, 5 December 2011 21:25 (five years ago) Permalink

Apols for asking that in a dumb way

William (C), Monday, 5 December 2011 21:25 (five years ago) Permalink

great work, thx much stet!

slandblox goole, Monday, 5 December 2011 21:27 (five years ago) Permalink

presumably this kinda space will be cheaper in the future?

iatee, Monday, 5 December 2011 21:27 (five years ago) Permalink

WmC, I think that was the perfect way to ask.

Making like Melusine (Pyth), Monday, 5 December 2011 21:27 (five years ago) Permalink

whatever price he asks, we shall pay

dayo, Monday, 5 December 2011 21:30 (five years ago) Permalink

apprec your work, stet.

Dr Morbius, Monday, 5 December 2011 21:35 (five years ago) Permalink

Wmc: We're being outrun by falling prices for now, but there are other complications -- the bigger it gets the more unwieldy it gets too. Running backups was more of a pain than it needed to be on the old server, for instance.

(In database terms, it's not that huge -- there are databases of hundreds of GBs and more happily chuntering along. Those tend to have more dedicated stewards and need care and watering, tho.)

stet, Monday, 5 December 2011 21:38 (five years ago) Permalink

Thanks, stet. Sorry for being crochety on the other thread. Really been missing/needing ILM of late.

Naive Teen Idol, Monday, 5 December 2011 21:41 (five years ago) Permalink

WmC, I think that was the perfect way to ask.

― Making like Melusine (Pyth), Monday, December 5, 2011 3:27 PM (9 minutes ago)

heh, thanks. I keep thinking "would it be possible to reduce the load on the DB by making the first several years of it read-only and sending it off to live quietly with relatives or something?" but really, there's too much value in revisiting and reviving old threads. And having it accessible to read doesn't reduce the load anywhere, I guess. It's like having to lug around a copy of The Riverside Shakespeare all the time instead of just the one play you're reading at the time.

William (C), Monday, 5 December 2011 21:42 (five years ago) Permalink

Amazon seems like a pretty sound option IMO

Ed, Monday, 5 December 2011 21:43 (five years ago) Permalink

I'm not gonna read all that cause it's long and I'm trying to leave work but I'm happy to donate and thank you stet for your hard work. Also, the sandbox is pretty good for the time being imo.

ᶘ ᵒᴥᵒᶅ (~curious orange~), Monday, 5 December 2011 21:44 (five years ago) Permalink

Thanks stet for handling this, much appreciated.

xp Amazon option seconded.

yes this is the real (snoball), Monday, 5 December 2011 21:49 (five years ago) Permalink

+1 Amazon. I can't help in any meaningful way other than by injecting funds. William (C), you should buy a Kindle.

(^_^) (^_^) (^_^), Monday, 5 December 2011 21:51 (five years ago) Permalink

"would it be possible to reduce the load on the DB by making the first several years of it read-only and sending it off to live quietly with relatives or something?" but really, there's too much value in revisiting and reviving old threads.

I have been thinking similar things myself. The quick and easy thing to do is make everything that hasn't been updated in say a year just a read-only HTML thread. That'd be super-easy to serve, and we could do it from a tiny server like this one.

But it's also a bit crap -- wouldn't be properly integrated with search, and you couldn't just bump an old thread. I'm trying to see if there isn't something cleverer we could do that would fix those problems -- perhaps the threads live in cold storage as static HTML and then if someone tries to bump one we "defrost" it and insert all its posts into the DB again? Can foresee problems doing that, too.

Either way, by far the simplest option for the immediate future given funding (eg at least another 2/3 years the way things have been going) is just to get a bit more memory.

stet, Monday, 5 December 2011 21:51 (five years ago) Permalink

I can live with the sandbox for a while. As an example of Amazon EC2 performance it's pretty impressive.

yes this is the real (snoball), Monday, 5 December 2011 21:53 (five years ago) Permalink

^^ yeah, definitely

would our existing host folks be willing to allow us to spin up another sandbox using their virtual solution so we can compare performance?

OH NOES, Monday, 5 December 2011 21:54 (five years ago) Permalink

ah, got it -- those were the same questions and reservations I had from a layman's perspective -- just wasn't sure if I was asking the right questions or the programming equivalent of "can we turn this block of solid granite inside out?"

xp to stet

William (C), Monday, 5 December 2011 21:56 (five years ago) Permalink

is sb extra

― Sad Banter (p much resigned to deems), Monday, 5 December 2011 21:19 (34 minutes ago) Bookmark Permalink

this, this should be like carbon tax, we should get a credit & have to buy more

good work stet!, let us know when you need $$$. when your head explodes from the continuous effort dedicated to server maintenance &c we will chip in for replacement parts for that, also

Never translate German (schlump), Monday, 5 December 2011 21:57 (five years ago) Permalink

That's probably not a bad idea to ask about. I would guess they'd want to put the existing server out of its misery before they do that though, and I want to keep it zombieing on until I have a live running copy of proper ILX on another server.

(I have a copy running on my Mac, and the DB all checks out, but it's not quite the same)

xp thanks everybody btw <3 u all

stet, Monday, 5 December 2011 21:59 (five years ago) Permalink

I am in favor of a sb tax - we would never need a funding drive

dayo, Monday, 5 December 2011 22:02 (five years ago) Permalink

frogbs would keep ilx running for 100 years

dayo, Monday, 5 December 2011 22:02 (five years ago) Permalink

stet, there is one way I might be able to help depending on (a) how you go about doing what you intend to go about doing and (b) how busy I am when you get to that point. Bung me an email if you so desire.

(^_^) (^_^) (^_^), Monday, 5 December 2011 22:08 (five years ago) Permalink

Yes, adding to the thank-yous to stet for service above and beyond!

William (C), Monday, 5 December 2011 22:13 (five years ago) Permalink

I want to volunteer to help but this is about to become silly crazy busy season for me and I don't want to be the flake who backs out

OH NOES, Monday, 5 December 2011 22:14 (five years ago) Permalink

can't believe you're already flaking dan. sorry everyone - it seems like the orphans won't get into their new home in time for christmas after all.

Never translate German (schlump), Monday, 5 December 2011 22:16 (five years ago) Permalink

so unthoughtful, jeez

dayo, Monday, 5 December 2011 22:17 (five years ago) Permalink

awesome, adding my thanks here, and hopefully some $ at some point.

nuhnuhnuh, Monday, 5 December 2011 22:17 (five years ago) Permalink

hey stet, thanks for all yr work!

your voice of treason, Monday, 5 December 2011 22:23 (five years ago) Permalink

fwiw i support using amazon too, best to use the futuristic hosting styles since we r a cutting edge community after all

Cooper Chucklebutt, Monday, 5 December 2011 22:31 (five years ago) Permalink

can't believe dan is being such a huge jerk about this

river wolf, Monday, 5 December 2011 22:38 (five years ago) Permalink

hey stet, thanks for all yr work!

― your voice of treason, Monday, December 5, 2011 5:23 PM (15 minutes ago)

echoing frogbs here, thanks stet

k3vin k., Monday, 5 December 2011 22:39 (five years ago) Permalink

dan needs to take some time off and think abt his priorities

Cooper Chucklebutt, Monday, 5 December 2011 22:40 (five years ago) Permalink

xpost

http://i.imgur.com/xsUWy.gif

your voice of treason, Monday, 5 December 2011 22:41 (five years ago) Permalink

\(^o^)/

Cooper Chucklebutt, Monday, 5 December 2011 22:42 (five years ago) Permalink

\(^o^)/

your voice of treason, Monday, 5 December 2011 22:42 (five years ago) Permalink

Thank you, stet! I have no skills to contribute but would love the opportunity to throw money at the problem.

thejenny, Monday, 5 December 2011 22:43 (five years ago) Permalink

btw in case I sounded too much like a huge wanker I was offering to develop the sort of supporting documents that would be megaboring for an ilx thread.

(^_^) (^_^) (^_^), Monday, 5 December 2011 22:44 (five years ago) Permalink

ditch the db, keep the sb

cccccc, Monday, 5 December 2011 22:44 (five years ago) Permalink

GOOGLE ADS HAVE WE TREID GOGGLE ADS

regional pug (dealwithit.gif), Monday, 5 December 2011 23:31 (five years ago) Permalink

Amazon affiliate setup where every word on ILX is a link to a product.

milo z, Monday, 5 December 2011 23:34 (five years ago) Permalink

loll yes

Cooper Chucklebutt, Monday, 5 December 2011 23:40 (five years ago) Permalink

My sense is that, when ILX is reopened with the intention of long term stability, that about 3 or 4 days' worth of the pre-crash threads ought to be displayed to anyone who arrives there. This should cut down considerably on the new arrivals bashing the search function.

Aimless, Friday, 23 December 2011 18:24 (five years ago) Permalink

would it work if it was open but if jimmy riddle was banned? if you open it real quick the first 51 in can help out

Never translate German (schlump), Friday, 23 December 2011 18:24 (five years ago) Permalink

maybe we should just not do ilm could be a solution

Cooper Chucklebutt, Friday, 23 December 2011 18:25 (five years ago) Permalink

haha
i didnt search anything. I just started 2 threads

Anyone need a (Jimmy Riddle), Friday, 23 December 2011 18:26 (five years ago) Permalink

and were they really worth it

Never translate German (schlump), Friday, 23 December 2011 18:27 (five years ago) Permalink

Yeah this is still on Amazon. The setup is largely the same as before -- mysql, tomcat and apache -- but it's somehow consuming massive amounts of memory. It's not the actual caches, they're staying about 500mb (which is what I'd expect), but I am seeing free memory steadily dwindle to nowt.

There's two big differences from the old box -- one is that we're running on the OpenJRE instead of the Oracle one (we used to be on the Sun JRE), and the other is that the recreated Lucene search index is now a massive 4gb file (it was previously chunks built up over time).

I am first of all going to try disabling the search and see how it goes with that, and failing that will shift to the Oracle JRE.

It's hugely annoying that the next size of server up is 8gb of memory and $150/month but if that's what it takes so be it.

stet, Friday, 23 December 2011 18:28 (five years ago) Permalink

as a hilarious behavioural experiement can we try an 'essential ilx' where people only try to post things that are worthwhile and important

Never translate German (schlump), Friday, 23 December 2011 18:29 (five years ago) Permalink

^ lol

Never translate German (schlump), Friday, 23 December 2011 18:29 (five years ago) Permalink

^rite

Never translate German (schlump), Friday, 23 December 2011 18:31 (five years ago) Permalink

well one was nominations thread for metal poll

Anyone need a (Jimmy Riddle), Friday, 23 December 2011 18:31 (five years ago) Permalink

lol

HOOS aka driver of steen, Friday, 23 December 2011 18:43 (five years ago) Permalink

https://www.youtube.com/watch?v=s6pLn_rF4_c

PROVEN BY BOOZE SCIENCE, Saturday, 24 December 2011 02:22 (five years ago) Permalink

as a hilarious behavioural experiement can we try an 'essential ilx' where people only try to post things that are worthwhile and important

― Never translate German (schlump), Friday, 23 December 2011 18:29 (Yesterday) Bookmark Permalink

ilf reduced to the gareth bale v. theo walcott thread
ilm reduced to rolling gun sounds and rolling r&b thread and maybe something metal and steely dan
ile just that thread about trucker hats on couches and any thread surmounter ever started

big popppa hoy, Saturday, 24 December 2011 09:14 (five years ago) Permalink

just let me know if we won't be back by the Oscars

Dr Morbius, Tuesday, 27 December 2011 15:54 (five years ago) Permalink

^^ itchy thread-trigger finger

Aimless, Tuesday, 27 December 2011 18:51 (five years ago) Permalink

i seriously miss ilxor

Bee, Friday, 30 December 2011 06:03 (five years ago) Permalink

I didn't even know there was a Miss Ilxor contest!

God is great. God is good. And people are crazy. Amen. (Johnny Fever), Friday, 30 December 2011 06:12 (five years ago) Permalink

We're gonna bring a case of wine
Hey, let's go mess and fool around
You know, like we used to

Bee, Friday, 30 December 2011 06:43 (five years ago) Permalink

thanks for everything stet

my new year resolution is: i want my ilx

Bee, Saturday, 31 December 2011 07:44 (five years ago) Permalink

+1

Thanks for everything.

Naive Teen Idol, Sunday, 1 January 2012 05:59 (five years ago) Permalink

Thanks for getting ILX up again before the Oz Open tennis, and for all your hard work generally.

Will all the old threads from the old site ever be accessible? Sorry if this ? has already come up....

Fred Nerk, Sunday, 1 January 2012 10:31 (five years ago) Permalink

They are OK, they are in cryogenic freeze next to Ted Williams's head

I'm in the sandbox with the tombstone HOOS (James Redd), Sunday, 1 January 2012 21:40 (five years ago) Permalink

Have tweaked this and fiddled that and turned on ILX again. It looks a lot healthier this time, so here's hoping. If it conks again, will need to get the big guns in.

stet, Sunday, 1 January 2012 22:29 (five years ago) Permalink

Sweeet, so far so good. Thanks, stet!

William (C), Sunday, 1 January 2012 22:41 (five years ago) Permalink

Dumb question, but how do I set zing back to ilx (not sandbox)?

remy bean in exile, Sunday, 1 January 2012 23:40 (five years ago) Permalink

Just change the instance setting to "ILX"

stet, Sunday, 1 January 2012 23:50 (five years ago) Permalink

ilxor.com is still redirecting to the sandbox btw. direct link http://www.ilxor.com/ILX/index.jsp

caek, Monday, 2 January 2012 00:03 (five years ago) Permalink

Yes -- want to keep it that way. I want to add load back slowly so I can be sure it's working. Right now it seems OK, and has been up longer than any previous attempt, but it's a quiet time generally.

stet, Monday, 2 January 2012 00:19 (five years ago) Permalink

ok cool.

got a tomcat error (which i didn't save sorry) when i logged in, but i manually visited /ILX after that and i was logged in ok.

caek, Monday, 2 January 2012 00:22 (five years ago) Permalink

I never reset zing to sandbox for that very reason

I'm in the sandbox with the tombstone HOOS (James Redd), Monday, 2 January 2012 01:47 (five years ago) Permalink

Also had trouble over there posting from zing for some unknown reason

I'm in the sandbox with the tombstone HOOS (James Redd), Monday, 2 January 2012 01:48 (five years ago) Permalink

It Is A Mystery

I'm in the sandbox with the tombstone HOOS (James Redd), Monday, 2 January 2012 01:56 (five years ago) Permalink

OK, problem was I had logged myself off fooling around with the settings.

I'm in the sandbox with the tombstone HOOS (James Redd), Monday, 2 January 2012 02:11 (five years ago) Permalink

THANK YOU!

Bee, Monday, 2 January 2012 03:08 (five years ago) Permalink

are we back now?

Dranke, the German Drake Impersonator (forksclovetofu), Monday, 2 January 2012 04:19 (five years ago) Permalink

http://www.ilxor.com/ILX/

clay, Monday, 2 January 2012 04:22 (five years ago) Permalink

are we back now?

YES

Bee, Monday, 2 January 2012 04:24 (five years ago) Permalink

so old skool ilx is kinda slow right now?

Dranke, the German Drake Impersonator (forksclovetofu), Monday, 2 January 2012 05:09 (five years ago) Permalink

seems fine to me

iatee, Monday, 2 January 2012 05:10 (five years ago) Permalink

Slow for me too -- 3-4 seconds for SNA page load. But it's better than it was a couple of hours ago.

William (C), Monday, 2 January 2012 05:18 (five years ago) Permalink

As soon as I say that, now it's loading instantly.

William (C), Monday, 2 January 2012 05:18 (five years ago) Permalink

yah its been intermittently slow for me

Cooper Chucklebutt, Monday, 2 January 2012 05:19 (five years ago) Permalink

slow for me,as has sandbox

Anyone need a (Jimmy Riddle), Monday, 2 January 2012 05:19 (five years ago) Permalink

I no longer even have a working login for "real" ILX - can someone who's managed to log in pls be a dear and tell me if it's still got SB before I bother going to the trouble of reregistering? I've enjoyed it here but I'm not going back to that stress.

Kurun a Spern (Fotherington Thomas), Monday, 2 January 2012 09:30 (five years ago) Permalink

Yeah, the new ILX does seem to be quite slow. After I clicked "login", it took half a minute before it actually logged in.

Tuomas, Monday, 2 January 2012 09:35 (five years ago) Permalink

FT, regular ILX hasn't changed at all, and so, yes, does still have SB.

If Villasboas was named Mike Jones and looked like a tool. (c sharp major), Monday, 2 January 2012 10:24 (five years ago) Permalink

OK, thanks for letting me know.

Kurun a Spern (Fotherington Thomas), Monday, 2 January 2012 10:26 (five years ago) Permalink

yeah, prohibitively slow in both spaces now.

Dranke, the German Drake Impersonator (forksclovetofu), Monday, 2 January 2012 18:11 (five years ago) Permalink

two years pass...

fundraiser

lxy, Wednesday, 23 April 2014 16:50 (three years ago) Permalink

fundraiser.

sanbox (step hen faps), Thursday, 1 May 2014 13:29 (three years ago) Permalink


You must be logged in to post. Please either login here, or if you are not registered, you may register here.