All better now

…although I should know better than to say that, given how things have been going around here lately.
OK, this morning’s scans completed successfully, and things appear to be functioning normally. Here’s what happened, best as I can tell:
– Last week, I had a database corruption issue, which caused me to have to drop and re-add a key table of the Ecosystem. When I did that, I forgot to re-add the indexes on the table.
– This caused the Ecosystem routines to hammer the database, and the Hosting Matters server. Apparently, HM decided to put a job in place to kill my processes. They did not, however, decide to actually notify me of this.
– I’ve now fixed the indexing problem, which hopefully will resolve the resource-usage problem for HM. I’ve emphasized to HM that I do want to be a good citizen, and am perfectly willing to work with them to not put an unreasonable load on their systems.
So was this one my fault, or Hosting Matters fault? A little of both, would be my conclusion.
Anyway, that’s the story for now; hopefully this is the end of the troubles for a while….
-NZB

And the culprit is…

…Hosting Matters. Again.
I just received confirmation that yes indeedy, Hosting Matters implemented a change a few weeks back that rendered part of my Ecosystem code inoperable.
Turns out that they have now made it illegal to execute ‘exec’ calls from php scripts, which is a crucial part of how the Ecosystem routines work. They did this change — in general terms — on their support forum, but I didn’t see it.
I’ve posted a request to the HM support forum asking for help from other HM users on how to fix the problem — as customer support has declined to offer an alternative. Suggestions from the peanut gallery here are of course most welcome as well.
I do apologize for the long delay in even attempting to fix this problem, but the demands of my non-blogging life have been high lately. It is frustrating to me as well, I assure you — I’ve put quite a bit of effort into ensuring that the Ecosystem code is reliable and keeps working as long as I don’t mess with it — beacuse I know that I don’t have time to be futzing around and fixing it every day. But it is difficult for me to react quickly to unexpected changes like this…
UPDATE: Annette from Hosting Matters has now replied to my issue, and has re-examined the problem. She’s found that ‘exec’ was not the issue, rather, the ‘jailshell’ which I believe HM defaults users to was causing the problem. It appears that I’ve now been reset to a normal, non-restricted shell.
In addition, Annette has pointed out that apparently some of my routines are not terminating properly, causing problems for HM’s server resources, and so they put a job in place to kill them for me. (I don’t blame them at all for doing so, although I’m puzzled why they didn’t contact me on the issue instead, and have indicated that in the future, I’d be happy to address this kind of problem if I’m told about it).
Anyway: I need to do some further investigation to clean up my own mess, and Annette has removed the restrictions while I do so, which seems fair. So with a little luck, I’ll have things cleaned up by the end of the weekend one way or another…
-NZB

Broken Again

Yet another mysterious issue has cropped up which has broken the Ecosystem — now, my cron job is simply aborting part-way through the run.
This means no Showcase winner today — I’ll have to figure out something later in the week. Sorry.

Hitchens Opens Fire on Hope; Shoots Own Foot

When Hitchens is right, he’s brilliant. But when he misfires, he does it just as big. See his latest Slate piece, Hopeless, in which he both speaks ill of the dead, and makes an ass of himself — the former of which is forgiveable; the latter, lamentable.
Hitchens has made a name for himself by, among other things, attacking those personages who have become larger than life — whose behavior has come to be judged by their reputation, as opposed to their reputation flowing from their behavior. He has rightfully trashed the hypocrisy of Mother Theresa’s views on poverty and the poor, savaged Henry Kissinger’s amorality and the public media’s refusal to acknowledge it, and along the way also taken time out to deflate visions of Princess Diana’s good works — not to mention dismissing Bill Clinton as the common degenerate his behavior showed him to be.
But Hitchens veers wildly off his generally-steady rails when he applies his favorite hammer to the nail of Bob Hope’s celebrity. His critique is simple: that Hope just wasn’t funny.
Hope is “a truly unfunny man”, according to Hitchens, and “never even remotely a comedian”. This is fascinating stuff, and he bangs on about it for a whole column, educating us all about precisely how and why Bob Hope isn’t actually funny.
The one little problem here is that, unlike whether Mother Theresa actually helped the poor or not, and whether Henry Kissinger actually violated U.S. and international laws now and again, the question of whether Bob Hope is funny is not a question of fact. It is a question of opinion — more to the point, a matter of personal taste.
Hitchens was right to challenge the easy celebrity of his previous targets, first because they were operating under false pretenses. Mother Theresa was envisioned to be a friend of the poor, when it would be more accurate to describe her as a friend of poverty; Kissinger operated (and still operates) under a comfortable haze of respectability when any inspection of his behavior and history shows him to be anything but.
More importantly, the continued good reputation of these targets had consequences. People giving money to Mother Theresa might well send it somewhere more worthy if not for the whiff of sainthood that surrounds her. And American relations worldwide continue to be influenced — albeit indirectly — by Kissinger’s wide-reaching associations. Combatting these public perceptions arguably serves the greater good.
But what, exactly, is the purpose of even trying to deflate Bob Hope’s reputation as an entertainer; as simply a funny man? Does Hitchens truly think the world will be a better place; that moral justice will have been served in some sense if he can convince large groups of people that, in fact, the enjoyment they gained from seeing Hope perform was some kind of an error of judgment on their part?
The fact is, I’ve never found Hope to be particularly funny myself. But unlike Hitchens, I don’t consider my own personal taste to be the final arbiter of such matters. Millions of people worldwide seem to feel differently than I on this question. And while such a popular landslide against my chosen opinion will rarely make me reconsider a moral or ethical position, it will get me to acknowledge that if millions of people think that an individual was entertaining, well, then he probably was — even if I don’t find him to be to my own liking.
Hitchens has blazed an honorable path as a contrarian who is willing to challenge orthodoxy when it needs it, and he retains my respect as a man with fierce beliefs and an equally fierce intellect to drive them. But sometimes — particularly in matters of popular culture — the orthodoxy is right by definition, and will do just fine without any challenging, thank you very much.
PS – It serves Hitchens right that Slate’s editors ran his piece with the deeply lame headline “Hopeless” — which I refuse to believe he chose himself.

Fixed

All TTLB functions appear to be back to normal. Let me know if you still see any anomolies, but methinks everything should be OK now…

Almost fixed?

OK, I’m pretty sure I’ve identified and fixed the problem.
How’s this for obscure: The way the Ecosystem works, at a high level, is that a job is submitted for each weblog which goes out and downloads the text of the blog’s front page, and then parses it to extract links. In debugging that routine, I traced the root problem to the fread command in PHP: instead of downloading the whole page, it appeared to be getting only the first few hundred bytes. Hence: only a handful of links would be found.
Now, I am fairly certain that when Hosting Matters moved servers, the version of PHP being used was updated to 4.3.2. (I can’t prove it, as I don’t have access to the old server now — but it’s definitely 4.3.2 on the new one).
Interestingly, if you read the PHP manual, there’s note that the behavior of fread has been changed in version 4.3.2. I can’t quite match what I’m seeing with what the note describes, but it seems awfully suspicious.
Especially since I replaced the fopen/fread combination with the new function get_file_contents, and shazam: all appeared back to normal.
Anyway; I’m running a one-off catchup run of the scan routines now, and if all goes well, things should be back to normal in a few hours. Will update again later…

Blog On!

2003 is on!
So go visit Amish Tech Support, A Small Victory, and Meryl Yourish and throw some pledge dollars in to support Magen David Adom .
You might as well, ’cause nothing works around here. I’ve now debugged the source of the Ecosystem problems down to the actual link-scanning job that hits each blog. It is running perfectly — except for some unknown reason, it is now only returning a tiny fraction of the actual links each blog has. Utterly frustrating…

Everything is Wrong

In case you are wondering why nothing is working around here, my friends at Hosting Matters moved my server (their choice, not mine) over last weekend, and I only just now realized that this meant I needed to reset all of the cron jobs that make the Ecosystem and other TTLB functions work. (To their credit, they did mention that this might be necessary, although not exactly prominently, in their notification of the change).
However, they’ve also blown away the old server’s cron listing, so I need to go figure out from memory what was running when. And that ain’t going to happen this morning.
Sorry — we’ll be back up and running in a day or so.
Update 7/23 pm: OK, I think I’ve got all the cron jobs re-established. If all goes well, the Ecosystem & all our other goodies should be back up to snuff by tomorrow morning. Stay tuned…
Update 7/24 am: Great. Cron jobs ran, but for some reason, the scan job is running — but not picking up everyone’s links. Sorry folks, but it’ll be another day — my day job calls…
Update 7/25 am: No, it’s not fixed yet. And I can’t debug it, because Hosting Matters has disabled my shell access. We’ll see how long it takes them to re-enable it this time.
Update 7/25 am: Much better this time! My shell access was re-enabled within an hour or so of my entering the ticket. Thanks to the HM support folks for the swift response. Debugging to commence … hell, I don’t know when. It’s a busy few days coming up folks, so bear with me…

Hitchens Does Stand Up

Hitchens doing stand-up comedy?
Be afraid. Be very afraid.
Starts slow, in my opinion, but stick around for the Q&A session; Hitchens is at his best going off-the-cuff working from the crowd.

The Poison Kitchen Wins

Congratulations to Poison Kitchen for winning this past week’s New Weblog Showcase with the post Now It Can Be Told: Saddam’s Secret Weapons.
Full results are:
The Poison Kitchen: Now It Can Be Told: Saddam’s Secret Weapons
( 10 links)
Sadly, No!: Only 16 words you say?
( 7 links)
American Digest: The Sunday New York Times Lite
( 6 links)
One Little Victory: The Killing Urge
( 5 links)
Wince and Nod: Are You A Dung Beetle, Too?
( 3 links)
Passenger Pachyderms: The News on Tape
( 3 links)
Howard Lovy’s NanoBot: The Hulk, Prince Charles and other scary things
( 2 links)
dr.mani’s remarkably purple spots…: an experiment in selfishness
( 2 links)
In Sheeps Clothing: Liberty
( 2 links)
The Usurer: For want of a fly-swat
( 2 links)
the Federal Examiner: Why Not Tenet?
( 2 links)
Angry Liberal: Tucker Eats His Shoes
( 0 links)
Angry Liberal: Tucker Eats His Shoes
( 0 links)
RobMorris.NET featuring Baby Morgan: Episode 1
( 0 links)
JustCron Report: JustCron Report
( 0 links)
Sarcastic Southerner: Dean’s Hypocrisy
( 0 links)
Interested-Participant: Senator Lieberman and Elmer Fudd
( 0 links)

Daily News?

Is anybody actually reading the Daily News?
I thought it was pretty spiffy when I rolled it out a few weeks back, and still do, mostly. Still doesn’t function quite perfectly, but it does ok.
Haven’t heard much reaction to it, though… thoughts? Feedback? Don’t make me do my Ben Stein impression…

Priorities & Frivolities Wins the Showcase

Congratulations to & Frivolities for winning this week’s New Webblog Showcase!
Complete results are as follows:
Priorities & Frivolities: T3 in 2003: Rise of the Political Machine
( 15 links)
EconoPundit: Noted With Relief
( 12 links)
Who Tends The Fires: The Beauty of Being Mom
( 10 links)
SoonerThought: The General Who Would Be President
( 9 links)
Bad Money: Today’s Graffiti Currency: A Sea Story
( 7 links)
Johnny America: The Royal Tenenbaums
( 6 links)
What Would George Say!: Out of Respect for the 9/11 Victims please contact your Congress members
( 6 links)
BIROCO.COM ~ A way to look at things: A blatant attempt to influence Googlism
( 6 links)
Canucker: No Echo
( 6 links)
One Father For Dean: Many Are Listening
( 6 links)
Vantage Point: Fighting with Our Eyes Wide Shut
( 6 links)
DW-i: “We’re not going to beat Dean.”
( 5 links)
Firefive: Barking Moonbat of the Month Award
( 5 links)
Boots and Sabers: Idealism
( 5 links)
Frogs and Ravens: Finding the Still Point
( 5 links)
Writing in Orange: Living in Orange 1: From Your Window
( 3 links)
Dohiyi Mir: I Get It: Saddam Was Bad
( 3 links)
mythic flow: Individuality and Freedom
( 3 links)
stevedanforth.com – live a little: In pursuit of a dream
( 2 links)
Blog o’RAM: Home Alone
( 1 links)
Eric Poole’s Very Own Punditry: Court Affirms Bush’s Power to Detain Citizen as Enemy
( 0 links)
Dissento’s (Culture) War Journal: Tommy Franks pulls a Cartman
( 0 links)
PunditMania: Quo Vaids Blogs ?
( 0 links)
TerraFirmaDiaries: just because you can does not mean we will
( 0 links)

Ecosystem Redesign

Well, this day had to come: the Ecosystem is now split into multiple pages.
Having all 3,000+ weblogs on the front page was getting just too unwieldy — that single page was over 600 KB, which was killing my bandwidth costs, and not that much fun to wait for to load, either.
So now the front page shows the top 100 (Higher Beings, Mortal Humans, and Playful Primates), with every other category having its own individual page. Hopefully the navigation should be fairly straightforward; I’ve also added a search function which allows you to find weblogs by name.
Feedback is welcome as always…
-NZB

A close Showcase

It’s one of the tightest races we’ve seen yet over in the this week — any one of several blogs could come out on top.
So if you haven’t yet voted, do it! And if you don’t know how, read this.

New Feature: Traffic Statistics!

So a few days ago I pointed out the work Steven over at Poliblog did to blog traffic statistics with Ecosystem rankings, and I complained a bit that somebody should come up with a standard API for reporting traffic stats so that I could integrate them into the Ecosystem.
Then I thought about it a bit, and realized that, with limitations, it was possible to do such a thing even today (This kind of thinking comes naturally from the unofficial TTLB Ecosystem motto: “API? We don’t need no stinkin’ API!”) . So I did. Witness: the new traffic statistics page.
Now, the major caveat is, stats are only being gathered for weblogs that use SiteMeter, and then only if you make your statistics public. But to my surprise, a whole heck of a lot of weblogs fit into that category. (As of today, about 700 out of the total of about 3100).
As far as I know, this the first time anyone has gathered traffic statistics for this many weblogs in one place, though I could be mistaken. The scan routine is integrated into the normal daily Ecosystem job, so the statistics will be updated daily just like the normal Ecosystem listings.
As always, please let me know if you see anything funkylike, and enjoy!
-NZB

Rush Limbaughtomy Wins the Showcase

Congratulations to Barry at Limbaughtomy: his post GOD’s other Son George has brought him victory in this week’s New Weblog Showcase.
Complete results are as follows:
Rush Limbaughtomy: GOD’s other Son George
( 15 links)
under the fire star: Chitra Talkies
( 10 links)
Apropos of Nothing: Cat Hair Drama
( 7 links)
HD Blog: Ethics & Genetics
( 5 links)
All Day Permanent Red: Cruising for Howard Dean
( 5 links)
Catallarchy: Patriotism vs. Nationalism
( 5 links)
Mazes and Walls: Complaining about Marriage
( 5 links)
Cowgirl Cries: Carnal Conflict
( 3 links)
Geek Rants: Reverse logic
( 3 links)
UnPundit: A mighty wind?
( 2 links)
Vietnam through my eyes: Where Vietnamese people came from…
( 1 links)
The Populist: The De-Evolution of Warfare
( 1 links)
PunditMania: Quo Vaids Blogs ?
( 0 links)
Radio Free Newport: Rock and roll ain’t noise pollution
( 0 links)

Ecosystem: Self Service!

Finally have implemented (half of) a feature I should have added ages ago: self-service for the Ecosystem!
On the details page of any weblog, you can now click a link which takes you to an ‘edit details’ page, from which you can change any information on the blog, or request that it be deleted entirely (good for hunting down those dreaded duplicates!)
The changes will not take effect immediately — I will still review them manually and only execute those that seem to make sense (to prevent abuse, and to avoid having to implement some gawdawful authentication system).
I still need to write the back-end that will make my ‘accept change’ work, but y’all can start combing the Ecosystem for data errors and submit change requests to your hearts’ content. If you’ve sent me such requests in email, I’d greatly appreciate it if you’d re-submit them through the interface — I’ll still try to get to them if you don’t, but it’ll probably happen later.
Thanks!
-NZB
PS: No, this doesn’t mean Hosting Matters my shell access problem — it just happened that I was able to implement this feature without shell access. The goodies that you were going to get today were much spiffier than this — but it looks like that ain’t happening.