The Problem with Web Development…

Am I the only one who feels like web development is a tug of war? Nearly every project I’ve worked on seems to have:

• the “suits”, who have a vision for expanding the business but couldn’t give a flying hoot about the quality of their code
• developers, who often care deeply about the code but don’t give a rotten pizza about the business concept
• creatives, who care about making things pretty because screw everyone else

Methinks the web has got a serious communication problem to solve.

The Solution: Openness and Education

Enter Birth of a Site. This new project aims to educate the public about web development best practices in a holistic way, encompassing technology, marketing and design. And we are going to do it through the development of the site itself. The vision is to document every step of developing this static landing page into a full-fledged content portal, full of interviews with industry experts on the process of creating a quality website, from choosing a catchy name to hiring reliable developers to marketing your product. How meta is that?

We plan on trying some hotshot new technology (well hello there, NoSQL; HTML5, how you doin’?). And of course, the project and anything we build in the process will be open-source. The evolving source code will be posted to Github.

If you work with the web, we want to hear from you.

If you’re an expert in the web – be it development, design or marketing – please leave your suggestions in the comments, or contact me to set up an interview.

See you all over at Birth of a Site!

Want to make money off your blog?? I make $100000000 a week off my blog! Here’s How! Click here for this magic formula! Drive Traffic! Make dollars!

And of course, clicking takes you to an order page where you can shell out some hard-earned money for a PDF containing their “magic formula”.

Well I like you, dear readers, so I’m going to tell you their secrets, absolutely FREE! Here’s the trick to “monetizing your blog”:

1. Put up a newsletter subscription form
2. Write some nonsense about how much money you make off your blog, even if you don’t – this will drive traffic
3. Blast your users with stupid annoying emails promising them the secret to blog monetization
4. Put these steps 1-4 into a PDF and sell it for $30

It’s recursive!

That said, I haven’t changed my general opinion about Drupal, and this week I’ll be addressing some of the recurring points from the comments.

Let’s start with:

Q. What do you recommend in its place and under what circumstances?

A. If you’re building a web app with complex requirements, build from scratch.

I’m a programmer by trade so I’m usually inclined to “roll my own”. If your project is less “website” and more “web app” with many custom features, you need an architecture to work around your requirements, not the other way around. There are plenty of frameworks—Rails, Symfony, Django, to name a few—with enough plugins that you’ll get many of the same features that Drupal offers, such as user management and authentication, without sacrificing flexibility.

More importantly, rolling your own leaves you in charge of deployment procedures. My number one issue with Drupal is deployment. Storing configuration in the database makes deploying a Drupal site across multiple environments a huge PITA. There is a module dealing with this—of course—but it’s still in active development. And as far as I’m concerned, the sheer fact that this has to be dealt with through a third-party add-on is evidence of bad design.

In the original post I wrote that it’s possible to code up a custom CMS in a couple days, and I stick to this statement. No, it’s not possible to code up a Drupal replacement, i.e. a CMS that can be used for “anything”. But it’s definitely possible to code a basic custom CMS for *one* site. And custom-tailored suits, if well-sewn, are superior to off-the-rack jackets. This is hard to dispute.

Blogging? Use a blogging engine.

Use Wordpress, Typo or any of the other excellent ones out there. I happen to use WP but don’t have a particular bias.

Simple content site? Off-the-shelf CMS (including Drupal) is fine

If you’re aiming for something a tad more complex than a blog, but that is still limited to presenting content to a limited audience, and you don’t need much flexibility beyond the front end, and you don’t have complex deployment procedures, and you don’t have a team of developers working on the site, and there won’t be much post-launch development (that’s a lot of ifs) by all means use a general-purpose CMS. This can be Drupal (I said as much in the original post), but in the comments I’ve also seen people recommend these, in no particular order:

• Silverstripe
• BrowserCMS
• concrete5
• modx

(I’ve excluded Joomla, a big ball of spaghetti fail. Wonder when the hate mail will start for that one…)

In short, I stand by my statement. Use an off-the-shelf CMS if you don’t need to do anything fancy. And if you’ve tried ay of the Drupal alternatives I mentioned above, please write in with your opinions.

Two years ago I endorsed Multiple IE, but unfortunately they’ve dropped the ball on Vista and IE8 support. I recommend upgrading to DebugBar.

Are you choosing a Content Management System for your next site? Allow me to throw in my two cents against Drupal. In theory, Drupal is a CMS that lets you control your site out of the box. In practice, it’s a nightmare to configure and maintain.

I recognize that Drupal might work for small sites: install the software, upload your custom theme, and start adding content. But if you’re planning a serious site, with custom features and potential for high traffic, Droople, as I’ve christened it, might not be your best bet.

Apologists may protest that module X or hack Y might fix my gripes with Drupal. I think this is irrelevant. A CMS requiring a slew of third-party mods before it can be usable is useless to someone who can code a custom Rails CMS in a day or two. (Hint, hint. Build it in Rails.)

Without further ado, here is a breakdown of why Drupal is bad for the various parties involved, together with Why It’s Bad (WIB) notes for the less tech-savvy.

Gripe #1: Drupal Stores Just About Everything in the Database

Databases are great for storing passwords, content, and countless other things. These things do not include “views”, i.e. templates. That’s right, templates. In the database. Drupal stores templates in the database.

WIB: Best practices dictate using a version control tool like Subversion or Git which, among other things, lets two developers “download” the same code so they can work on it simultaneously. Unfortunately for Drupal developers, database settings are not typically stored in version control. This means that two developers have no easy way of coordinating their work. Further, it means you’ll have a heck of a time replicating your settings across multiple servers if you have a high-traffic site that requires load balancing, or a staging server where you want to preview changes before they go live.

Another thing Drupal stores in the database? Logs. In the database. Drupal stores logs in the database.

WIB: Logs record visits to your website, together with other useful details like errors. When you get lots of traffic, logs tend to get huge – we’re talking gigs. Logs are important for debugging, but, unless you’re a high-security outfit, they do not need to be permanently stored. If you back up your database regularly – and you should – you’ll be transferring all that unnecessary data every time you run a backup. Further, you might want to grant a sysadmin or programmer access to your logs to help diagnose a problem. You don’t want that person digging around in your database for this information: one bad search query and your data is gone.

Gripe #2: Drupal is Freaking Hard to Use and Has a High Learning Curve

In the good old days, building a website meant coding a page in HTML and uploading it to a server. If you wanted to update the page, you edited the HTML, and re-uploaded the file. Then people who didn’t know HTML started clamoring for a way to edit their own sites. Along came Content Management Systems, presumably to let an admin log in, edit, save, and call it a day.

Enter Drupal. This CMS, my friends, is so bloated that it takes days, if not weeks, for the layperson to learn their way around it. In fact, my Drupal clients are so confused by the interface that they still send me content and ask me to input it for them.

WIB: The whole point of a CMS is enabling a layperson to update a website. I think the average person can learn basic HTML and FTP within weeks. So if a CMS takes weeks to learn, and you still need a techie to help you find your way around it, the CMS is not doing its job of simplifying the website update process.

Why is Drupal so hard to use? It’s a combination of bad usability and  confusing navigation. An example of Drupal’s bad usability is requiring a user to confirm changes twice before they’re saved—often with the Save button located below the jump, i.e. out of view. As for confusing navigation:  creating a blog post is under “Create Content” but editing a blog post is under “Manage Content” and sidebar content is under “Site Building / Blocks”. What?

Gripe #3: Drupal’s Design is Piss-Poor

Drupal has a history of security vulnerabilities and is written in ugly spaghetti code (guys, have you heard of classes? Objects? Inheritance? Oh, nevermind).

It also has poor maintainability. In other words, upgrading to a newer Drupal version breaks templates and other code you may have written for an older version. Fixing this, of course, costs time and money. Other frameworks also deprecate older APIs, but in my experience, the changes between versions of other frameworks are not so drastic, and there is often a helpful process built in to migrate your code.

Gripe #4: Drupal’s None Too Friendly

There’s been quite a backlash over Drupal’s new trademark policy, which is rather contrary to the spirit of open-source software. Why protect your trademark so fiercely, Drupal? Your product is open, and great, so why the need to protect it against sites like DrupalSucks.com? Oh, right…

Conclusion: Drupal Sucks

If you are on a low budget and do not plan on hiring a developer, give Drupal a whirl (I guess). However, if you find yourself needing an oxymoronic “Drupal programmer” to do extensive configuration, consider hiring a Ruby on Rails developer instead. Chances are, you can get a custom CMS at the same price and with half the frustration.

Edit: Oh, and Drupal’s search also sucks. Thanks YCombinator.

Edit #2: I was unclear re. the templates in the database. I wasn’t referring to .tpl files, but to Drupal Views.

Edit #3: Oh, and for the templates you do store in the filesystem, the naming conventions for overriding a particular item are horrendous. For example, block-block-5.tpl.php contains the auto-generated id in the filename. So if the node/block has a different id in a different environment (dev vs. production) you’re out of luck, and the template won’t work.

Edit #4: Thanks for the amazing feedback everyone, both positive and negative. There have definitely been a lot of helpful suggestions. I’ll look into them all and write a followup with the results!

• File permissions (chmod)
• Regular expressions (Apache rewrite rules, Ruby regexes etc)

Does anyone else feel the same way? Do you have any good tutorials that can rid me of these handicaps once and for all?

• Working with the awesome JuicyOrange to launch the Global Debt Clock for the Economist.  Fun to play with, but it will depress you if you are American or British.
• Guest post over on Geek Feminism about sexism on social news sites. Please give it a read.

RubyOnRails.org actually showed this for a good few hours. TechCrunch has the full story.

SQLSTATE[HY000]: General error: 1005 Can’t create table ‘./mydb/#sql-xxxx.frm’ (errno: 150). Failing Query: ALTER TABLE cow ADD FOREIGN KEY (barn_id) REFERENCES barn(id)

I was using InnoDB, both tables were there, both columns were there, what else could MySQL want? Why didn’t it work?

Apparently, you must use the same data type for both columns. My cows.barn_id was an INT, while barns.id was a SMALLINT. This didn’t fly. Changing both to SMALLINT did the trick.

Hint: if you’re using Symfony with Doctrine, like I am, use “integer(4)” to get the MySQL INT datatype, or “integer(2)” for a SMALLINT data type. Plain old “integer” converts to BIGINT.

So how ’bout it, newspapers elsewhere? New York Times, Telegraph, anybody?